Search papers, labs, and topics across Lattice.
This study investigates the effects of masking stale observations in long-horizon search agents across various model sizes and retrievers, revealing a complex relationship between context management and model performance. The authors find that the benefits of masking follow an asymmetric inverted-U shape, peaking when a strong retriever is paired with a mid-capacity model, while performance collapses in saturated models. This research highlights the importance of understanding the interplay between retriever recall and the model's filtering capacity, offering a nuanced perspective on context management in agentic search tasks.
Masking stale observations can boost search agent accuracy, but only under specific conditions鈥攖oo much masking can backfire dramatically.
Long-horizon search agents accumulate large amounts of retrieved content across many tool calls, making context-budget efficiency increasingly important. A minimal intervention is to mask stale observations from the context as the trajectory progresses, but it remains unclear when this form of context management helps and why. We study observation masking through a systematic sweep over various agent backbones (4B to 284B parameters) and three retrievers on offline and live-web agentic search benchmarks. We find that the accuracy gain from masking follows an asymmetric inverted-U shape when plotted against the model's accuracy without context management: a plateau under weak retrievers, a peak when a strong retriever meets a mid-capacity model, and a sharp collapse when the model is saturated. This pattern reflects the interaction between retriever recall and the model's implicit filtering capacity, rather than either factor in isolation. Mechanistically, masking implements a token-for-turn trade-off: it removes observations the model has largely stopped attending to and pages the agent rarely re-opens. The added turns help when they convert failures into successes, but they fail when masking removes evidence the model would otherwise have used. We therefore reframe context management as a regime-dependent intervention and provide a holistic perspective for analyzing context use in agentic deep search. We release our scaffold and trajectories here (https://github.com/i-DeepSearch/observation-masking) to support future research.
BAIR