Search papers, labs, and topics across Lattice.
This paper investigates source preference in LLM agents across three domains, finding that agents consistently favor items from certain sources, even when those items are objectively worse. This preference significantly impacts user outcomes, as agents will select an item with one fewer satisfied requirement from a preferred source over a fully satisfying item from a dispreferred source two-thirds of the time. The study identifies training biases and missing information as key drivers of this preference and demonstrates that supplying information or counter-prompting can mitigate it.
LLM agents will pick a worse item from a "preferred" source over a better item from a "dispreferred" source two-thirds of the time, revealing a critical bias that can be reduced by providing more information.
As LLM agents decide on users'behalf which product to buy, which hotel to book, or which paper to cite, a preference for items from certain sources (the sites or services they come from) shapes what users receive and which sources are selected. We study source preference in end-to-end search with 12 agent models across three domains. Comparing items from different sources that satisfy the same requirements at the same position, we find that each model prefers some sources and avoids others in every domain, largely agreeing on which. This preference can outweigh how well items satisfy the request: an item satisfying one requirement fewer is selected about two-thirds of the time when it comes from a preferred source and the better one from a dispreferred source, but almost never in the reverse case. The information identifying an item's source affects selection by itself: hiding it weakens the preference, and relabeling an item with a preferred source raises its selection rate. We test two routes to this preference: training that rewards better items can make a source a shortcut for requirement satisfaction, and missing information can trigger preconceptions about the source. Supplying missing information or a prompt countering these preconceptions reduces source preference.
