Search papers, labs, and topics across Lattice.
This paper introduces Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a novel framework designed to enhance sample-efficient online feedback-driven search in scenarios where preferences are revealed sequentially. By employing a collective posterior sample sharing mechanism, IMPFM effectively transports multiple interactive particles towards a target distribution while maintaining broad coverage for heterogeneous preference alignment. Empirical evaluations demonstrate that IMPFM significantly outperforms existing methods, enabling global exploration and mitigating issues of reward over-optimization and mode collapse.
IMPFM achieves unprecedented global exploration in feedback-driven search by leveraging multi-particle interactions to prevent mode collapse and reward over-optimization.
While generative models have enabled training-free reward alignment, current methods typically excel in local exploration within narrow regions of the underlying distribution. These approaches struggle when preferences are unknown a priori and only revealed through sequential feedback-a scenario demanding broad exploration to uncover high-utility regions. To address this, we propose Sequentially-Controlled Interactive Multi-Particle Flow-Maps (IMPFM), a framework for sample-efficient online feedback-driven search. IMPFM progressively transports a group of interactive particles toward the target distribution, maintaining the broad coverage essential for heterogeneous preference alignment. IMPFM introduces a principled and efficient posterior sample sharing mechanism across particles powered by flow maps. By correcting individual particle drift with the collective posterior samples of the entire ensemble at each resampling step, the framework maximizes sample utility to enable global exploration while actively mitigating reward over-optimization, typical of standard control frameworks. Paired with a principled exploration-exploitation reweighting mechanism involving multi-particle interaction, this sequentially corrected multi-particle dynamics explicitly preserves structural diversity and overcomes the weight degeneracy inherent to standard SMC samplers. Crucially, we prove that the resulting sampling framework yields a multi-particle interaction-aware Feynman-Kac corrector that progressively steers the multi-particle system toward a KL-tilted target distribution, facilitating global exploration and preventing mode collapse. Extensive empirical evaluations and rigorous ablations across diverse search and alignment tasks confirm the efficacy of IMPFM over existing baselines.
