Search papers, labs, and topics across Lattice.
This paper investigates the limitations of large language models (LLMs) in performing elementary numerical tasks, despite their success in high-level mathematical reasoning. The authors introduce the Numerical Grounding Framework (NGF), which distinguishes between Representational Grounding and Procedural Grounding to analyze and improve numeracy in LLMs. Through a coordinated evaluation of various model families, they identify architectural interventions and practical strategies that can enhance numerical reliability, particularly for users of pretrained systems.
LLMs struggle with basic numerical tasks, but targeted architectural tweaks and supervised fine-tuning can significantly boost their performance.
Large language models (LLMs) achieve strong results on mathematical reasoning benchmarks yet remain unreliable on elementary numerical tasks, including magnitude comparison, large-integer arithmetic, fractions, and scientific notation. This survey examines basic numerical understanding as a capability distinct from high-level mathematical reasoning. We propose the Numerical Grounding Framework (NGF), which decomposes numeracy into Representational Grounding (RG), mapping numeral forms to value, magnitude, and equivalent representations, and Procedural Grounding (PG), executing arithmetic operations in accordance with their mathematical definitions. Using NGF, we organize recent diagnostic benchmarks, failure modes, structural explanations, and mitigation strategies. We review evidence concerning tokenization, positional encoding, embedding geometry, and pretraining-data distribution. We also apply NGF in a coordinated evaluation of three frontier model families across Number Cookbook, NumericBench, and GSM-Symbolic, comparing atomic, contextual, and reasoning-assisted numeracy. Architectural interventions such as digit-aware tokenization and Abacus Embeddings can improve models trained from scratch but are generally unavailable to users of pretrained systems, for whom supervised fine-tuning, reasoning scaffolds, and external tools are more practical. We conclude with deployment recommendations and research directions for more reliable numerical behavior in foundation models.
