Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
LLM agents struggle to leverage demonstration videos for in-context learning due to context length limitations and the loss of fine-grained temporal information. Recursive Video In-Context Learning (RV-ICL) addresses this by transforming demonstrations into a hierarchical structure of sub-events, allowing agents to navigate and access relevant video details at different levels of granularity. This training-free approach significantly improves success rates on complex robotic manipulation tasks, demonstrating the efficacy of structured video recall.
Robots can now learn complex manipulation tasks from a single demonstration video by recursively accessing fine-grained sub-event details, boosting success rates by up to 9%.
LLM agents that orchestrate frozen vision-language-action (VLA) policies improve across episodes through text memory, which records what the agent did but not how the task is done. A demonstration video shows it, but fits poorly into an agent's context. The full video slows every turn, fixed keyframes lose the contact detail that decides whether a grasp holds, and what the agent needs shifts from the task's structure while planning to the frames around each contact. We introduce Recursive Video In-Context Learning (RV-ICL), a training-free method that turns a demonstration into a hierarchy the agent navigates rather than a prompt it receives. The hierarchy is built from the sub-events of the demonstration, such as grasps and releases. Its levels grow finer, from keyframes of the whole task to phases, moments and short clips, and are exposed through read-only tools. The agent reads the coarse levels before planning. During execution it re-enters the hierarchy whenever a step needs more detail and loads only the clip of its current sub-goal. One demonstration per task is enough. Built on RPent, RV-ICL raises success from 92.6% to 96.5% on LIBERO-PRO and from 86.7% to 95.8% on LIBERO-Plus.