Search papers, labs, and topics across Lattice.
This paper introduces Marginal Advantage Accumulation (MAA), a novel method that addresses the challenge of contradictory feedback in batch-style trace distillation by enabling cross-batch evidence accumulation. By formalizing the concepts of alignability and comparability, MAA constructs differential signals for operation-level comparison and utilizes exponential moving averages (EMA) to accumulate evidence effectively. The results demonstrate that MAA significantly outperforms existing batch-level distillation methods in 14 out of 16 settings across multiple benchmarks, while also reducing token consumption during optimization by about 75%.
MAA not only outperforms traditional batch-level distillation methods but also slashes optimization costs by 75%, redefining efficiency in memory-driven agent evolution.
In batch-style trace distillation, the same memory operation may receive contradictory feedback across different batches. Existing methods lack a cross-batch, operation-level evidence accumulation mechanism, making it impossible to distinguish stably effective operations from accidental hits. This paper formalizes the requirement as two structural conditions, alignability and comparability, and proposes Marginal Advantage Accumulation (MAA). MAA constructs differential signals to make them comparable across batches, accumulates signed evidence per operation via EMA, and ensures cross-batch traceability through semantic identity merging. As a post-processing architecture, MAA achieves the best results in 14 out of 16 settings across 4 benchmarks and 4 target models, consistently outperforming existing batch-level distillation baselines and matching or surpassing online alternatives in most settings, while reducing optimization-phase token consumption by approximately 75%.
DAMO