Search papers, labs, and topics across Lattice.
This paper advocates for Geometric Algebra (GA), specifically Clifford algebras, as a more suitable foundation for natural language semantics compared to conventional linear algebra. It introduces a Functional Geometric Algebra (FGA) framework that extends GA to support typed, compositional semantics with inference and interpretability, while maintaining compatibility with distributional learning. The paper demonstrates GA's advantages through formal development, identification of core capabilities absent in linear algebra, a detailed semantic contrast example, and by showing how GA-based operations can be made explicit in transformer architectures.
Geometric Algebra offers a principled algebraic framework that captures higher-order semantic interactions, potentially resolving persistent limitations in compositional semantics and interpretability that plague current linear algebra-based NLP models.
Distributional and neural approaches to natural language semantics have been built almost exclusively on conventional linear algebra: vectors, matrices, tensors, and the operations that accompany them. These methods have achieved remarkable empirical success, yet they face persistent structural limitations in compositional semantics, type sensitivity, and interpretability. I argue in this paper that geometric algebra (GA) -- specifically, Clifford algebras -- provides a mathematically superior foundation for semantic representation, and that a Functional Geometric Algebra (FGA) framework extends GA toward a typed, compositional semantics capable of supporting inference, transformation, and interpretability while retaining full compatibility with distributional learning and modern neural architectures. I develop the formal foundations, identify three core capabilities that GA provides and linear algebra does not, present a detailed worked example illustrating operator-level semantic contrasts, and show how GA-based operations already implicit in current transformer architectures can be made explicit and extended. The central claim is not merely increased dimensionality but increased structural organization: GA expands an $n$-dimensional embedding space into a $2^n$ multivector algebra where base semantic concepts and their higher-order interactions are represented within a single, principled algebraic framework.
