Siddharth Boppana

This work finds that simple difference of means vectors coherently represent reward hacking in Kimi K3, GLM 5.2, and Qwen 3.8 Max, and provides evidence that simple, white-box methods can be used to scalably study and monitor reward hacking behaviors in frontier open source models.

Transforming post-training from opaque reward optimization into a transparent process of auditing and sculpting the learning signal could revolutionize how we guide model behavior.

Sparse autoencoders, despite their popularity for extracting interpretable features, often fail to capture the underlying manifold structure of concepts, instead fragmenting them across multiple, diluted features.

LLMs often know the answer long before their "reasoning" suggests, wasting tokens on performative chain-of-thought.