Search papers, labs, and topics across Lattice.
This paper establishes a theoretical framework for inverse-distance attention by transitioning from a Euclidean model (Resolver) to a non-Euclidean model (Riemann GeoResolver). Key findings include the demonstration that inverse-distance attention achieves exact retrieval with significantly lower resource requirements compared to softmax, and that the non-Euclidean extension enhances performance through hyperbolic and spherical geometries, improving convergence and limiting noise memorization. The Riemann GeoResolver framework integrates multiple modules that optimize attention mechanisms and memory retrieval, providing a robust alternative to traditional methods.
Inverse-distance attention can achieve exact retrieval with constant resources, outperforming softmax's logarithmic scaling in complexity.
We present a theoretical foundation for inverse-distance attention, from its Euclidean prototype (Resolver) to its non-Euclidean realization (Riemann GeoResolver). The Euclidean part establishes three core theorems: (1) circuit separation---IDA achieves exact retrieval with $\mathcal{O}(1)$ resources while softmax requires $\Omega((\log n)^2)$ width; (2) a Polyak--Lojasiewicz inequality with $\Omega(e^{\Delta^2/\sqrt{d}}/\Delta^2)$ stronger constant than softmax, implying linear convergence, $\mathcal{O}(\log n)$ Lipschitz scaling under a low-rank/clustering assumption, $\Theta(1)$ Hessian spread, and absence of spurious local minima; (3) a width-independent effective rank bound that limits noise memorization---softmax memorizes arbitrary labels when $d_h\ge n$, while IDA limits test error to $\mathcal{O}(\eta^2)$. The non-Euclidean extension then builds upon this prototype, replacing Euclidean distance with hyperbolic geodesic distance for storage and spherical geodesic distance for routing. The Riemann GeoResolver framework comprises ten integrated modules: four HIDA operators spanning $\Theta(n^2)$ to $\Theta(1)$ per token; Hyperbolic Curvature Compression (HCC) with provable error bounds; HyperGate with gradient lower-bound theorem; Spherical Inverse Distance Attention (SIDA) with sphere-analog PL inequalities; Dynamic Memory Genesis (DMG) with $\mathcal{O}(\log T)$ regret bounds; and Geodesic Sparse Routing (GSR) with quality and communication bounds. The Euclidean theorems are proved in full; the non-Euclidean extension theorems are proved with analogous arguments. This work establishes a theoretical arc: from Euclidean attention as a special case, to hyperbolic memory, to spherical retrieval.
