Search papers, labs, and topics across Lattice.
PEAM is introduced as a novel parametric memory framework for embodied agents in Minecraft, replacing inference-time retrieval with parameter-resident skills. It uses a multimodal Mixture-of-Experts LoRA architecture with physically isolated adapters for continual learning, and trains on failure-correction trajectory pairs using a contrastive objective. Results show PEAM improves long-horizon task performance, mitigates forgetting, and improves efficiency compared to retrieval-based and other parametric memory agents.
Forget retrieval, PEAM internalizes experience directly into agent parameters, turning memories into skills that improve performance and efficiency in Minecraft.
We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for open-ended reasoning with a fast parametric module for reflexive execution of consolidated skills. The fast module is a multimodal Mixture-of-Experts LoRA architecture with per-category physically isolated adapters, enabling parameter-level continual learning without catastrophic forgetting. We treat failure as a first-class training signal: failure--correction trajectory pairs are internalized through a joint behavioral-cloning and contrastive objective, so the agent learns not only what succeeds but also how corrected actions differ from failed ones. To govern consolidation, PEAM introduces a parameterization-worthiness score for deciding which experience should be internalized, and a scale-free self-triggered consolidation mechanism for deciding when to internalize without task-specific hand-tuned thresholds, making the agent self-evolving as the trigger transfers across task distributions without re-tuning. Experiments in Minecraft show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency over retrieval-based embodied agents and parametric memory variants.
