Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces a multilingual benchmark for solvability detection in Large Language Models (LLMs), extending the ReliableMath dataset to French and Greek. By training multilingual probes to predict Solvability Belief, the authors analyze the performance of state-of-the-art LLMs across different languages. The key finding reveals that while Solvability Belief is a largely universal feature, higher-resource languages like English demonstrate stronger reasoning capabilities but lower faithfulness in solvability detection.
Higher-resource languages may boost mathematical reasoning in LLMs but paradoxically reduce the faithfulness of solvability detection.
Solvability detection is one of the most challenging aspects of mathematical reasoning for Large Language Models (LLMs). While prior work has studied this capability extensively, these analyses have been limited to English. Consequently, it remains unclear whether multilingual failures arise from differences in internal Solvability Belief or from language-dependent failures to express it. To address this gap, we introduce the first multilingual benchmark of paired solvable and unsolvable mathematical problems, extending ReliableMath to French and Greek. Using this, we train multilingual probes predicting Solvability Belief and analyze the solvability detection capabilities of state-of-the-art LLMs behaviorally, representationally, and in terms of faithfulness. We find that Solvability Belief is encoded as a largely universal, language-agnostic feature, and that higher-resource languages such as English, despite achieving stronger mathematical reasoning performance, exhibit lower solvability-detection faithfulness.