Search papers, labs, and topics across Lattice.
This paper introduces the Think Short, Defer Smart (TSDS) framework, which optimizes edge LLM agents by integrating a lightweight convergence probe with a perplexity-based deferral rule to manage reasoning budgets effectively. By calibrating these mechanisms through a multi-objective Learn-Then-Test (LTT) procedure, TSDS ensures that agents can defer to a cloud model only when local uncertainty is too high, thereby enhancing reliability. The evaluation across multiple benchmarks shows that TSDS significantly reduces computational load by 43%-73% while maintaining guarantees on episode reward and cloud-call rate compared to existing baselines.
TSDS cuts edge reasoning compute by up to 73% while ensuring reliable decision-making through smart deferral to cloud models.
LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.
