Search papers, labs, and topics across Lattice.
This paper introduces PolySim, a deterministic framework designed to enable cross-modal retrieval on compute-in-memory (CiM) hardware by reformulating probabilistic retrieval methods. By approximating Gaussian embeddings with low-order polynomial bases and employing a learnable order-bilinear kernel, PolySim eliminates the need for Monte Carlo sampling while retaining essential distributional information. Experimental results demonstrate that PolySim outperforms deterministic baselines by up to 10.3% in retrieval accuracy and matches or exceeds the performance of existing probabilistic methods, all while ensuring compatibility with crossbar architectures.
PolySim achieves up to 10.3% better retrieval accuracy than traditional deterministic methods by transforming probabilistic embeddings into a fully deterministic framework suitable for edge devices.
Cross-modal retrieval on edge devices benefits from probabilistic embeddings that capture semantic uncertainty, but deploying them on compute-in-memory (CiM) hardware remains an open problem. The core difficulty is a sampling gap: probabilistic methods such as PCME rely on Monte Carlo sampling and nonlinear distance evaluation at inference, which are fundamentally incompatible with CiM crossbar arrays that support only deterministic, single-step matrix-vector multiplication. Few existing probabilistic retrieval methods can be executed on a conventional crossbar. To bridge this gap, we propose PolySim, a framework that reformulates probabilistic retrieval into a fully deterministic pipeline. PolySim approximates each Gaussian embedding dimension using low-order polynomial bases and computes similarity via a learnable order-bilinear kernel, eliminating stochastic sampling while preserving distributional information. In experiments on six benchmarks spanning video, image, and audio retrieval, PolySim improves R@1 over deterministic baselines by up to 10.3\% and matches or exceeds PCME, while reducing inference to a single crossbar-compatible matrix-vector multiplication. CrossSim evaluation under realistic device non-idealities confirms robust deployment on conventional crossbar arrays. To the best of our knowledge, PolySim is the first method to enable probabilistic cross-modal retrieval on CiM hardware.
