Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces BASIS, a training-free Bayesian method for inferring task-aligned similarity criteria from weak interactive feedback during test-time decision-making in language models. By maintaining a joint posterior over candidate targets and criteria and updating it with scalar scores, pairwise preferences, or verifier outputs, BASIS adaptively selects the most relevant similarity notion for each instance. This approach significantly improves performance on tasks like concept recovery, reasoning exemplar selection, and response refinement, particularly under criterion shifts and with larger candidate pools, outperforming static mixture baselines and compute-matched selection methods.
Weak interactive feedback can dynamically steer LLMs to the correct similarity criterion for test-time decisions, unlocking significant performance gains beyond static mixtures.
Language model systems repeatedly make test-time decisions by comparing similarity: they retrieve exemplars, rank candidates, and revise responses. Yet similarity is not unique: lexical, semantic, structural, and stylistic resemblance can point to different candidates, and the task-aligned notion can change across instances. We introduce BASIS (Bayesian Axis Selection via Interactive Signals), a training-free Bayesian method that uses weak test-time feedback to infer which criterion in a supplied, task-specific library governs the current decision. It maintains a joint posterior over candidate targets and criteria, updates it from scalar scores, pairwise preferences, or verifier outputs, and selects probes by expected information gain. In a finite noisy model, we characterize when the target-criterion pair is recoverable and show why fixed-criterion and fixed-mixture rules can fail under criterion shift. Across concept recovery, reasoning exemplar selection, and response refinement, BASIS improves six-probe concept-recovery Success@1 from 56.0% to 68.1%, reasoning accuracy from 72.3% to 73.1%, and refinement win rate from 61.4% to 63.8% over tuned static mixtures. It also exceeds compute-matched best-of-N selection on reasoning (72.1%) and refinement (61.2%). Gains are largest under criterion mismatch and with larger candidate pools, and matched controls indicate that adaptive criterion weighting explains much of the benefit, with joint target-criterion inference adding further value. BASIS selects only among supplied criteria and does not discover new ones.