Kentaro Inui

LLMs use a surprisingly structured "Cell-based Binding Representation" to track entities and relations in discourse, opening the door to targeted interventions and improved relational reasoning.

Uncertainty estimates from LLMs can crumble under distribution shift, but the right probe design – think middle layers and token aggregation – can make them surprisingly resilient.

Despite domain-specific encoding subspaces, language models represent hierarchical relationships with surprisingly consistent linear transformations across different semantic domains.

LVLMs encode nodes early but edges late, suggesting a fundamental bottleneck in how these models process relational information in diagrams.