Search papers, labs, and topics across Lattice.
The paper introduces s-Trace, a method for efficiently estimating the minimal subgraph of an LLM required to approximate the full model output. Using s-Trace, the authors find that LLM computation is organized into two phases: an initial sparse subgraph in early layers that predicts the head of the output distribution, followed by denser computation in later layers (especially attention heads) for refinement. They also show that the amount of computation correlates with model uncertainty and that sparser subgraphs capture shallow statistics.
LLMs use a surprisingly small, early-layer core for initial prediction, suggesting they may be more computationally efficient than their size implies.
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their full capacity for all inputs. We introduce the s-Trace method to efficiently estimate the subgraph of size s that best approximates a full model output. With this method, we find the computation in a variety of LLMs to be organized in two distinct phases. A small subgraph mostly composed of early-layer nodes can reconstruct the head of the full model output distribution. Adding further nodes, mostly located in later layers and increasingly consisting of attention heads, leads to incremental refinements in approximating the full output distribution. We find moreover that the amount of necessary computation per input correlates with model uncertainty, and that sparser subgraphs encode shallow statistics, such as unigram frequency. Overall, our results suggest a consistent modular organization in effective LLM computation, with a sparse early-layer core providing a rough prediction that is further refined through denser computations in later layers.
Apple ML