Najoung Kim

Grammaticality is not just a matter of probability; it’s a distinct and robust dimension encoded in the internal representations of neural language models.

LMs don't incrementally track entities through state changes like you'd expect, but instead use a surprisingly fragile "global suppression tag" to handle removals, leading to predictable failure modes.