Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores a novel approach to inference in large language models (LLMs) by introducing a dynamic program-of-layers (PoLar) mechanism that allows for flexible execution of pretrained layers, enabling them to be skipped or looped based on input requirements. The authors demonstrate that this method can lead to significantly shorter execution times while maintaining or improving accuracy, particularly correcting errors made by the original model. Their findings reveal that traditional fixed-depth execution limits the potential reasoning capabilities of LLMs, suggesting a broader space of latent computations that can be harnessed for improved performance.
Dynamic execution of layers in LLMs can enhance accuracy and efficiency, revealing a wealth of untapped reasoning potential beyond fixed-depth models.
Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic-depth methods, often while executing fewer layers, and that these gains persist under out-of-distribution evaluation. Our results suggest that fixed-depth execution captures only a narrow subset of an LLM's latent reasoning capacity.