Search papers, labs, and topics across Lattice.
The paper introduces Neural Computers (NCs), a new machine form unifying computation, memory, and I/O within a learned runtime state, aiming to make the model itself the running computer. As a first step, the authors explore learning NC primitives from I/O traces in video models that generate screen frames from instructions, pixels, and user actions in CLI and GUI settings. The results show that learned runtimes can acquire early interface primitives like I/O alignment and short-horizon control, but challenges remain in routine reuse, controlled updates, and symbolic stability.
Forget agents and world models – the future of computing could be learned directly from I/O traces, turning the model itself into the computer.
We propose a new frontier: Neural Computers (NCs) -- an emerging machine form that unifies computation, memory, and I/O in a learned runtime state. Unlike conventional computers, which execute explicit programs, agents, which act over external execution environments, and world models, which learn environment dynamics, NCs aim to make the model itself the running computer. Our long-term goal is the Completely Neural Computer (CNC): the mature, general-purpose realization of this emerging machine form, with stable execution, explicit reprogramming, and durable capability reuse. As an initial step, we study whether early NC primitives can be learned solely from collected I/O traces, without instrumented program state. Concretely, we instantiate NCs as video models that roll out screen frames from instructions, pixels, and user actions (when available) in CLI and GUI settings. These implementations show that learned runtimes can acquire early interface primitives, especially I/O alignment and short-horizon control, while routine reuse, controlled updates, and symbolic stability remain open. We outline a roadmap toward CNCs around these challenges. If overcome, CNCs could establish a new computing paradigm beyond today's agents, world models, and conventional computers.
