Sebastian Schuster

LMs don't incrementally track entities through state changes like you'd expect, but instead use a surprisingly fragile "global suppression tag" to handle removals, leading to predictable failure modes.

LLMs can become proactive collaborators that independently recognize when to ask questions to elicit missing information in real-world, underspecified tasks.

Diffusion LLMs secretly use end-of-sequence tokens as a hidden scratchpad, boosting their reasoning abilities in complex tasks.

LLMs may ace logic puzzles, but they still struggle to reason like humans when it comes to the messy, probabilistic inferences we make every day.