Search papers, labs, and topics across Lattice.
ImplicitMemBench is introduced as a novel benchmark to evaluate implicit memory in LLMs, focusing on unconscious behavioral adaptation through procedural memory, priming, and classical conditioning. The benchmark uses a Learning/Priming-Interfere-Test protocol to assess first-attempt performance across these constructs. Results on 17 models, including DeepSeek-R1, Qwen3-32B, and GPT-4, reveal significant limitations in implicit memory capabilities, highlighting the need for architectural innovations beyond parameter scaling.
LLMs struggle to automatically apply learned procedures or avoid failed actions without explicit reminders, achieving only up to 66% on a new implicit memory benchmark.
Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval. This gap is critical: effective assistants must automatically apply learned procedures or avoid failed actions without explicit reminders. We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three cognitively grounded constructs drawn from standard cognitive-science accounts of non-declarative memory: Procedural Memory (one-shot skill acquisition after interference), Priming (theme-driven bias via paired experimental/control instances), and Classical Conditioning (Conditioned Stimulus--Unconditioned Stimulus (CS--US) associations shaping first decisions). Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring. Evaluation of 17 models reveals severe limitations: no model exceeds 66% overall, with top performers DeepSeek-R1 (65.3%), Qwen3-32B (64.1%), and GPT-5 (63.0%) far below human baselines. Analysis uncovers dramatic asymmetries (inhibition 17.6% vs. preference 75.0%) and universal bottlenecks requiring architectural innovations beyond parameter scaling. ImplicitMemBench reframes evaluation from"what agents recall"to"what they automatically enact".
