Search papers, labs, and topics across Lattice.
This paper introduces INTERPOL, a novel framework for de-anonymizing language models in voting-based leaderboards like LM Arena by learning to distinguish models based on stylistic patterns. INTERPOL synthesizes hard negative samples through model interpolation and uses adaptive curriculum learning to capture deep stylistic features missed by traditional statistical methods. Experiments show INTERPOL significantly outperforms existing baselines in identification accuracy, and simulations demonstrate its potential to manipulate rankings in real-world scenarios.
LM Arena's model anonymity is more vulnerable than previously thought: a new attack, INTERPOL, leverages interpolated preference learning to expose deep stylistic patterns and manipulate rankings.
Strict anonymity of model responses is a key for the reliability of voting-based leaderboards, such as LM Arena. While prior studies have attempted to compromise this assumption using simple statistical features like TF-IDF or bag-ofwords, these methods often lack the discriminative power to distinguish between stylistically similar or within-family models. To overcome these limitations and expose the severity of vulnerability, we introduce INTERPOL, a model-driven identification framework that learns to distinguish target models from others using interpolated preference data. Specifically, INTERPOL captures deep stylistic patterns that superficial statistical features miss by synthesizing hard negative samples through model interpolation and employing an adaptive curriculum learning strategy. Extensive experiments demonstrate that INTERPOL significantly outperforms existing baselines in identification accuracy. Furthermore, we quantify the real-world threat of our findings through ranking manipulation simulations on Arena battle data.
