Andrew Draganov

There is not enough public evidence for us to confidently state that midtraining can address the core difficulties inherent in aligning powerful AI systems, and it is believed that demonstrations must be present either in midtraining or post-training datasets for these rules to be robustly learned.

A novel detection approach reveals that while traditional methods fail, a simple token look-up can effectively recover backdoor triggers in LLMs.