Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of backward-compatible multimodal representations in contrastive vision-language models by proposing spherical interpolation as a post-hoc alignment technique. By interpolating along the geodesic between old-model query embeddings and new-model query embeddings (orthogonally aligned to the old space), the method aims to find an improved retrieval direction. Experiments demonstrate that spherical interpolation significantly enhances retrieval performance over orthogonal alignment alone, recovering backward-compatibility without gallery re-indexing.
Spherical interpolation between old and new model embeddings recovers backward-compatibility for vision-language retrieval, outperforming orthogonal alignment by finding optimal query directions in the interstitial space.
Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natural metric for cross-modal retrieval. A practical challenge arises during model upgrades: independently trained models generally produce incompatible representation spaces, so replacing a deployed model typically requires recomputing embeddings for the entire gallery, which is prohibitively expensive at scale. Orthogonal post-hoc alignment can partially mitigate this problem by mapping new-model queries into the old-model gallery space. However, because independently trained models can differ in fine-grained representation structure, the orthogonal alignment remains approximate, leaving a residual angular discrepancy between the old-model query and the aligned new-model query. We study whether interpolation along the spherical geodesic between these two normalized query representations can improve retrieval without re-indexing the gallery. We characterize when this path contains an interior query direction closer to an idealized retrieval-optimal direction than either endpoint, and connect this characterization to Recall@$K$ through a local margin-based certification result. Experiments across multiple benchmarks and model families show that post-alignment spherical interpolation improves over orthogonal alignment alone, recovering backward-compatibility in most evaluated settings. Consistent with our geometric characterization, per-query oracle analysis shows that retrieval-favorable interior points occur frequently in practice. Code is available at https://github.com/miccunifi/SLERP_backward_compatibility .