Search papers, labs, and topics across Lattice.
This study analyzes how decoder-only transformers, specifically GPT-2, Llama, and Qwen, handle lexical ambiguity through a detailed layer-wise examination. The researchers found that while representations of ambiguous words become distinct in the middle layers, they partially reconverge in the late layers, leading to maximum KL divergence in next-token predictions at the final layers. The results suggest that despite increased similarity in embedding space, late-layer representational differences significantly influence model outputs, challenging the reliance on embedding similarity for tasks like semantic search and clustering.
Late-layer representational differences in LLMs can dramatically affect outputs, even when embeddings appear similar, revealing a hidden layer of complexity in model behavior.
In this work we investigate how decoder-only transformers resolve lexical ambiguity through layer-by-layer analysis of three models spanning three parameter sizes (GPT-2-Small-117M, Llama-3.2-3B, Qwen2.5-32B). For both homonyms and polysemes, we find that representations become maximally distinct in middle layers, then partially reconverge in late layers, while the KL divergence between their next-token predictions reaches its maximum in the final layers. The activation patching experiment provides causal evidence that late-layer representational differences directly determine outputs despite apparent increased similarity in embedding space. Our single-layer ablation experiment indicates that models achieve equivalent disambiguation despite qualitatively different layer-wise vulnerabilities. These findings offer a mechanism for recent observations where models' internal embedding similarities show low correlation with their behavioural outputs despite strong performance. The semantic distinctions therefore remain present but become increasingly invisible to similarity measures over the embeddings, with implications for embedding-based methods such as semantic search, retrieval, and clustering that rely on late-layer cosine similarity.
