Search papers, labs, and topics across Lattice.
This paper investigates whether cognitive complexity, as defined by Bloom's Taxonomy, is linearly encoded within the residual streams of LLMs. The authors train linear probes to classify different cognitive levels (Remember, Understand, Apply, Analyze, Evaluate, Create) based on activation vectors from various layers of LLMs. They find that linear classifiers achieve high accuracy (approximately 95%) in distinguishing these levels, suggesting that cognitive difficulty is represented in a linearly accessible subspace.
LLMs encode cognitive complexity from basic recall to abstract synthesis in linearly separable subspaces, suggesting a surprisingly structured internal representation of thought.
The black-box nature of Large Language Models necessitates novel evaluation frameworks that transcend surface-level performance metrics. This study investigates the internal neural representations of cognitive complexity using Bloom's Taxonomy as a hierarchical lens. By analyzing high-dimensional activation vectors from different LLMs, we probe whether different cognitive levels, ranging from basic recall (Remember) to abstract synthesis (Create), are linearly separable within the model's residual streams. Our results demonstrate that linear classifiers achieve approximately 95% mean accuracy across all Bloom levels, providing strong evidence that cognitive level is encoded in a linearly accessible subspace of the model's representations. These findings provide evidence that the model resolves the cognitive difficulty of a prompt early in the forward pass, with representations becoming increasingly separable across layers.
