Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
YC-Bench is introduced as a new benchmark to evaluate long-term planning and execution in LLM agents by simulating a year-long startup management task. The benchmark tests agents' ability to handle uncertainty, learn from delayed feedback, and adapt to compounding consequences in a partially observable environment. Results show that only a few models, notably Claude Opus and GLM-5, consistently surpass initial capital, and that scratchpad usage is the strongest predictor of success, while failure to detect adversarial clients is the primary cause of bankruptcy.
Running a simulated startup for a year reveals that even frontier LLMs struggle with long-term strategic coherence, often failing due to over-parallelization and susceptibility to adversarial clients.
As LLM agents tackle increasingly complex tasks, a critical question is whether they can maintain strategic coherence over long horizons: planning under uncertainty, learning from delayed feedback, and adapting when early mistakes compound. We introduce $\texttt{YC-Bench}$, a benchmark that evaluates these capabilities by tasking an agent with running a simulated startup over a one-year horizon spanning hundreds of turns. The agent must manage employees, select task contracts, and maintain profitability in a partially observable environment where adversarial clients and growing payroll create compounding consequences for poor decisions. We evaluate 12 models, both proprietary and open source, across 3 seeds each. Only three models consistently surpass the starting capital of \$200K, with Claude Opus 4.6 achieving the highest average final funds at \$1.27 M, followed by GLM-5 at \$1.21 M at 11$\times$ lower inference cost. Scratchpad usage, the sole mechanism for persisting information across context truncation, is the strongest predictor of success, and adversarial client detection is the primary failure mode, accounting for $47\%$ of bankruptcies. Our analysis reveals that frontier models still fail through distinct failure modes such as over-parallelization, demonstrating the capability gaps for long-horizon performance. $\texttt{YC-Bench}$ is open-source, reproducible, and configurable.