Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors develop BFM-QD, a framework that executes Quality-Diversity (QD) optimization directly within the compact latent space of pretrained Behavioral Foundation Models rather than high-dimensional policy parameter spaces. Crucially, the framework derives a closed-form, gradient-free policy improvement operator that analytically approximates policy gradient updates without requiring critic training or backpropagation. Evaluated across continuous-control benchmarks spanning locomotion, navigation, and manipulation, BFM-QD substantially outperforms parameter-space baselines, which completely collapse to near-zero performance under sparse or deceptive reward structures.
Moving quality-diversity optimization from raw policy parameters into the latent space of a behavioral foundation model rescues evolutionary search from total collapse in sparse, deceptive environments while eliminating critic training entirely.
Behavioral Foundation Models (BFMs) are an emerging paradigm in reinforcement learning, playing a role analogous to large language models in natural language processing: they have shown remarkable versatility, enabling zero-shot performance, fast imitation, and online adaptation, all by exploiting the structure of a latent space. In this work, we investigate whether the latent behavioral space induced by BFMs can serve as an effective search space to discover large repertoires of behaviorally diverse and high-performing policies through Quality-Diversity (QD) methods. While QD methods generally search directly in high-dimensional policy parameter space, in this paper, we present BFM-QD, a framework that performs QD search in the compact latent space of a BFM. We further show that the BFM-QD framework provides a closed-form, gradient-free policy improvement operator that approximates a policy gradient update, but requires no critic training and no backpropagation. Across continuous-control benchmarks spanning dense locomotion, sparse navigation, and contact-rich manipulation, BFM-QD consistently outperforms parameter-space baselines, with particularly stark gains in sparse and deceptive settings, where all tested parameter-space QD methods collapse to near-zero performance. These results show the effectiveness of the BFM-QD framework, benefiting from the synergy between dimensionality reduction of the search space and offline pretraining from diverse behavioral data. This positions BFMs as a general-purpose backbone for QD optimization, extending their utility beyond zero-shot task solving to the discovery of diverse behavioral repertoires.