Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the role of a prediction-error signal derived from a small predictor over a frozen encoder's latent space, demonstrating its dual function as a gate for plasticity and a substrate for metacognition. In a continual learning framework, the proposed non-parametric episodic memory system shows significant retention improvements on ImageNet classes, achieving up to 51.3 points of retention with the I-JEPA backbone, while a vision-language model effectively modulates its responses based on the novelty of concepts, achieving an AUROC of 0.966 in distinguishing known from novel inputs. The findings highlight the potential of surprise as a critical signal for enhancing memory consolidation and adaptive learning in AI systems.
Surprise signals can dramatically enhance memory retention and adaptive responses in AI, with a vision-language model achieving 99.2% recall of taught facts after consolidation.
We study a single idea across two settings: that a prediction-error signal, computed by a small predictor over the latent space of a frozen encoder, can serve both as a gate on plasticity and as a substrate for metacognition. In the first system, a non-parametric episodic memory writes a new concept only when this surprise is high, and a periodic offline replay phase consolidates recent traces into a slow linear readout. On a continual stream of 1000 ImageNet classes with a frozen DINOv2 or I-JEPA backbone, the consolidation phase recovers 17.7 points of retention on the oldest classes for DINOv2 and 51.3 points for I-JEPA (single-seed runs), and an ablation shows that replaying only a recent window is worse than no replay at all. In few-shot evaluation the same memory reaches 91.6% on 5-way 1-shot mini-ImageNet, above a task-specific baseline, while a harder 500-way regime exposes the true difficulty. In the second system, the same surprise signal, computed in a shared text-image space, modulates the behaviour of a vision-language model: it answers assertively when a concept is known, hedges when it is partially familiar, and refuses to identify the object and asks for an explanation when it is novel, learning the concept from a single user utterance. The external detector separates known from novel concepts at an AUROC of 0.966 (95% CI +/-0.024), far above the model's own verbalised confidence (0.618), while its token-level confidence sits below chance under greedy decoding; after a sleep phase that empties the fast store, the system recalls 99.2% of fifty taught facts from the consolidated store while a base model recovers none. We report both systems as proof-of-concept, with explicit limitations, and position the second against recent episodic-memory and personalised-VLM work.