Search papers, labs, and topics across Lattice.
The paper investigates whether Function Vectors (FVs), which drive in-context learning (ICL) in LLMs, represent concepts abstractly across different input formats. It demonstrates that FVs are not invariant to input format, as they are nearly orthogonal when extracted from different formats targeting the same concept. The authors then identify Concept Vectors (CVs), derived using Representational Similarity Analysis (RSA) to select attention heads encoding concepts consistently across formats, and show that CVs generalize better out-of-distribution compared to FVs.
LLMs don't represent concepts abstractly in the way we thought: the "function vectors" that drive in-context learning are surprisingly format-dependent, while more invariant "concept vectors" exist but don't drive ICL as effectively.
Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.
