Search papers, labs, and topics across Lattice.
This paper introduces the Optimal Transport Flow Concept Bottleneck Model (OTF-CBM), which redefines concept alignment in vision-language models as a dynamic cross-modal transport process. By employing Inverse Optimal Transport to learn a semantic cost for measuring cross-modal distances, OTF-CBM enhances the localization of concepts and captures semantic transitions more effectively than traditional methods. The results demonstrate that OTF-CBM outperforms existing models in classification accuracy and concept faithfulness, providing a novel geometric framework for interpretable reasoning across modalities.
OTF-CBM reveals that dynamic transport processes can significantly enhance the interpretability and accuracy of vision-language models, outperforming static alignment methods.
Concept Bottleneck Models (CBMs) promise transparent reasoning by predicting through human-interpretable concepts, yet their effectiveness fundamentally depends on how well visual and textual representations are aligned or matched. Existing vision-language CBMs often rely on pre-aligned encoders or global cosine similarity, which obscures fine-grained concept localization and fails to reflect true semantic geometry. In this work, we rethink concept alignment as a dynamic cross-modal transport process instead of static projection and propose the Optimal Transport Flow Concept Bottleneck Model (OTF-CBM). It first learns a data-driven semantic cost via Inverse Optimal Transport to measure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions between visual patches and textual concepts. With velocity-based concept activation, OTF-CBM captures interpretable geometric relations without ODE integration. Experiments further show that OTF-CBM achieves superior classification accuracy and concept faithfulness, offering a new geometric and dynamical perspective for interpretable cross-modal reasoning.
