Search papers, labs, and topics across Lattice.
The paper introduces MICE (Minimal Interaction Cross-Encoders), a novel architecture derived from cross-encoders for efficient document re-ranking. By selectively removing unnecessary interactions within the cross-encoder, MICE achieves a late-interaction-like architecture. Experiments demonstrate that MICE reduces inference latency by fourfold compared to standard cross-encoders, matching the speed of ColBERT while maintaining competitive in-domain effectiveness and exhibiting improved out-of-domain generalization.
Cross-encoders can be made 4x faster with minimal performance loss by surgically removing interactions, creating a "minimal interaction" architecture (MICE) that rivals late-interaction models in speed and surpasses them in generalization.
Cross-encoders deliver state-of-the-art ranking effectiveness in information retrieval, but have a high inference cost. This prevents them from being used as first-stage rankers, but also incurs a cost when re-ranking documents. Prior work has addressed this bottleneck from two largely separate directions: accelerating cross-encoder inference by sparsifying the attention process or improving first-stage retrieval effectiveness using more complex models, e.g. late-interaction ones. In this work, we propose to bridge these two approaches, based on an in-depth understanding of the internal mechanisms of cross-encoders. Starting from cross-encoders, we show that it is possible to derive a new late-interaction-like architecture by carefully removing detrimental or unnecessary interactions. We name this architecture MICE (Minimal Interaction Cross-Encoders). We extensively evaluate MICE across both in-domain (ID) and out-of-domain (OOD) datasets. MICE decreases fourfold the inference latency compared to standard cross-encoders, matching late-interaction models like ColBERT while retaining most of cross-encoder ID effectiveness and demonstrating superior generalization abilities in OOD.
