Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
This paper analyzes the reliability of autonomous language-model agents managing real capital in a 21-day onchain trading deployment called DX Terminal Pro, involving 3,505 agents trading real ETH. The study reveals that reliability stems not just from the base model, but critically from the surrounding "operating layer" including prompt engineering, typed controls, policy validation, and execution guards. Through pre-launch testing and targeted interventions on the operating layer, the authors significantly improved agent behavior, demonstrating the importance of holistic evaluation across the entire decision-making pipeline.
Language model reliability in real-world financial agents hinges more on the surrounding engineering than the underlying model itself.
We study reliability in autonomous language-model agents that translate user mandates into validated tool actions under real capital. The setting is DX Terminal Pro, a 21-day deployment in which 3,505 user-funded agents traded real ETH in a bounded onchain market. Users configured vaults through structured controls and natural-language strategies, but only agents could choose normal buy/sell trades. The system produced 7.5M agent invocations, roughly 300K onchain actions, about $20M in volume, more than 5,000 ETH deployed, roughly 70B inference tokens, and 99.9% settlement success for policy-valid submitted transactions. Long-running agents accumulated thousands of sequential decisions, including 6,000+ prompt-state-action cycles for continuously active agents, yielding a large-scale trace from user mandate to rendered prompt, reasoning, validation, portfolio state, and settlement. Reliability did not come from the base model alone; it emerged from the operating layer around the model: prompt compilation, typed controls, policy validation, execution guards, memory design, and trace-level observability. Pre-launch testing exposed failures that text-only benchmarks rarely measure, including fabricated trading rules, fee paralysis, numeric anchoring, cadence trading, and misread tokenomics. Targeted harness changes reduced fabricated sell rules from 57% to 3%, reduced fee-led observations from 32.5% to below 10%, and increased capital deployment from 42.9% to 78.0% in an affected test population. We show that capital-managing agents should be evaluated across the full path from user mandate to prompt, validated action, and settlement.