Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Latent-Steered Autoregressive (LS-AR) is a novel dual-channel architecture that decouples macro-objective steering from discrete token decoding using FiLM conditioning, addressing the single-channel limitation of standard autoregressive models. LS-AR employs a Static Goal Encoder for persistent goal retention and a Dynamic State Tracker for recurrent latent updates, achieving 100% target recall on long-horizon retrieval tasks and sustaining 89.0% completion in Blocksworld planning under perturbations. This architecture significantly improves performance on long-context tasks, increases throughput, and reduces VRAM usage, while also offering structural defense against text prompt injection.
A dual-channel architecture enables LLMs to achieve perfect long-horizon retrieval and robust planning by isolating macro-objectives from noisy context, outperforming baselines by a significant margin.
Standard autoregressive (AR) models process high-level task instructions, state history, and transient tokens within a single shared sequence of tokens. Consequently, they lack the architectural mechanisms needed to isolate macro-objectives from context noise. To overcome this single-channel limitation, we introduce Latent-Steered Autoregressive (LS-AR), a dual-channel architecture that decouples continuous goal steering from discrete token decoding via FiLM conditioning. We evaluate a Static Goal Encoder (P_0) for persistent macro-objective retention across long rollouts and a Dynamic State Tracker (P_t) for recurrent latent updates during generation. On long-horizon retrieval past context limits (H=1024, W=500), LS-AR (Static) achieves 100% target recall where parameter-matched baselines collapse (0%), while increasing throughput by ~35% and cutting peak VRAM by 52.8%. In Blocksworld planning under forced perturbations (k=1), LS-AR (Dynamic) sustains an 89.0% completion rate vs. 71.0% for the baseline, though zero-shot entity scaling (N ->N+1) exposes single-vector capacity limits (0%). Finally, dual-channel authority analysis shows that text goal dropout establishes latent-dominant control, offering structural defence against text prompt injection while introducing a latent vector attack surface.