Search papers, labs, and topics across Lattice.
This paper establishes a resource-aware theory of language generation under space efficiency constraints, focusing on memory-bounded learners that must output a hallucination-free hypothesis language from an adversarial positive stream. The authors prove that in the exponential-space regime, exact identification of the target language is achievable, while under stricter memory constraints, they derive a streaming algorithm that converges to a hypothesis with a generation gap of \(螖 = O(k^{2s-2})\). Additionally, they provide a near-matching lower bound, demonstrating a critical transition between polynomial and exponential space requirements for language generation tasks.
Achieving hallucination-free language generation with significantly reduced memory requirements reveals a sharp transition in capabilities that could redefine efficiency in language models.
We initiate a resource-aware theory of \textit{language generation in the limit} under the minimal constraint of space efficiency. In our framework, a learner observes an adversarial positive stream from a target language $K$ and must eventually output a hallucination-free hypothesis language $L \subseteq K$ while omitting at most $螖$ strings of $K$. We focus on $\mathcal{C}_{s,k}$, the collection of languages recognized by DFAs with at most $s$ states over an alphabet of size $k$, as the natural hypothesis class for memory-bounded learners. In the exponential-space regime, we prove that a learner can exactly identify the target $K$. Under a stricter memory budget, we characterize the strongest possible generation guarantees. In particular, we present a streaming algorithm using $\mathrm{poly}(s,k)$ space that converges to a hypothesis with generation gap $螖= O(k^{2s-2})$. Moreover, the learned hypothesis captures every string in $K$ of length at least $2s-1$. We complement this result with a near-matching lower bound through a reduction from a standard communication complexity problem. Specifically, achieving generation gap $螖\le k^{(1-\varepsilon)s}$ requires $k^{惟(\varepsilon s)}$ memory. Together, these results reveal a sharp transition between polynomial-space generation and exponential-space exact identification.