Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Neural Language Interpreter (NLI), a Latent Adaptation Network that learns its own discrete, symbolic-like programming language end-to-end. NLI uses a differentiable neural executor and Gumbel-Softmax relaxation to enable gradient-based optimization of discrete, compositional program structures. The key result is that NLI outperforms in-context learning, test-time training, and continuous latent program networks on tasks requiring combinatorial generalization and rapid adaptation due to its ability to refine initial program guesses via gradient descent through the neural executor at inference time.
Neural nets can now learn their own programming languages and refine programs via gradient descent at test time, outperforming in-context learning on combinatorial generalization tasks.
A central challenge in program induction has long been the trade-off between symbolic and neural approaches. Symbolic methods offer compositional generalisation and data efficiency, yet their scalability is constrained by formalisms such as domain-specific languages (DSLs), which are labour-intensive to create and may not transfer to new domains. In contrast, neural networks flexibly learn from data but tend to generalise poorly in compositional and out-of-distribution settings. We bridge this divide with an instance of a Latent Adaptation Network architecture named Neural Language Interpreter (NLI), which learns its own discrete, symbolic-like programming language end-to-end. NLI autonomously discovers a vocabulary of primitive operations and uses a novel differentiable neural executor to interpret variable-length sequences of these primitives. This allows NLI to represent programs that are not bound to a constant number of computation steps, enabling it to solve more complex problems than those seen during training. To make these discrete, compositional program structures amenable to gradient-based optimisation, we employ the Gumbel-Softmax relaxation, enabling the entire model to be trained end-to-end. Crucially, this same differentiability enables powerful test-time adaptation. At inference, NLI's program inductor provides an initial program guess. This guess is then refined via gradient descent through the neural executor, enabling efficient search for the neural program that best explains the given data. We demonstrate that NLI outperforms in-context learning, test-time training, and continuous latent program networks on tasks that require combinatorial generalisation and rapid adaptation to unseen tasks. Our results establish a new path toward models that combine the compositionality of discrete languages with the gradient-based search and end-to-end learning of neural networks.