Search papers, labs, and topics across Lattice.
This paper introduces an analytic concept-centric memory framework designed to enhance long-horizon embodied manipulation in agents by structuring memory around semantic concepts, including object parts, physical states, and manipulation skills. By organizing memories into interconnected categories鈥攐bject, scene, transition, and skill memories鈥攖he framework enables agents to perform structured retrieval of relevant information, leading to improved reasoning and skill reuse. Experimental results demonstrate significant advancements in task completion rates, retrieval accuracy, and generalization capabilities compared to traditional unstructured memory systems.
Agents using a structured memory framework can achieve significantly better manipulation performance, outperforming traditional methods in task completion and skill generalization.
Long-horizon embodied manipulation requires agents to remember persistent objects, track changing scene states, and reuse prior interaction knowledge. However, existing agent memories are often stored as unstructured histories or embedding-based records, making it difficult to retrieve manipulation-relevant object parts, physical states, action effects, and executable skills. We propose an analytic concept-centric memory framework for agentic embodied manipulation. Our memory organizes experience around structured analytic concepts, where objects are represented by semantic parts, parametric templates, grounded poses, affordances, and manipulation states. It further connects object and scene memories with transition memory for action-induced state changes and skill memory for template-grounded and policy-grounded execution. At runtime, the agent performs structured coarse-to-fine retrieval to identify relevant objects, states, transitions, and skills, supporting state-consistent reasoning and skill reuse. Experiments on memory-dependent manipulation, articulated-object generalization, real-world memory evaluation, and ablations show that our approach improves task completion, retrieval accuracy, object re-identification, and cross-object skill generalization over unstructured and embedding-based memory baselines.
