Search papers, labs, and topics across Lattice.
This paper introduces FCGraft, a Functional Cache Grafting framework that enhances code-policy synthesis for embodied agents by addressing the inefficiencies of delayed decoding and robustness issues in open-domain environments. By maintaining a library of validated code skeletons and their associated Transformer key-value caches, FCGraft synthesizes new policies through a novel method of cache grafting, which combines function segments and adapts them to specific tasks with minimal decoding. The results show that FCGraft achieves a 18.31% higher task success rate and 2.3x faster policy synthesis compared to existing methods like RAGCache.
FCGraft reduces policy synthesis time by over 2x while significantly improving task success rates by leveraging validated code structures and efficient cache management.
Code-writing large language models (CodeLLMs) generate executable code policies for embodied agents by translating natural language goals and environmental constraints into structured control programs. However, policy generation in open-domain embodied environments suffers from two fundamental limitations: (i) delayed decoding caused by repetitive prefill computation over long prompts, and (ii) limited robustness due to fully generative decoding, which often produces API mismatches, missing safety guards, and unstable control logic. To address these limitations, we present FCGraft, a Functional Cache Grafting framework. FCGraft maintains a library of function-level validated code skeletons and their associated prompt-level Transformer key-value (KV) caches, and synthesizes new policies by retrieving relevant functions and grafting their KV caches when a new task is provided. Given retrieved function caches, FCGraft performs cache grafting via stitching, which composes cached function segments into a composite policy, and patching, which locally adapts only the necessary code regions to satisfy task-specific parameters and constraints with minimal additional decoding. By eliminating redundant prefill computation, this approach reduces generation latency, while reusing validated control structures improves robustness over prompt-level caching methods RAGCache, achieving 18.31% higher task success rate and 2.3x faster policy synthesis.
