Zhongkai Sun

Models struggle with handwritten text, showing a 63-91% reliance on language priors for errors, while humans exhibit a more balanced error profile.

Achieving a Relational Delivery Score of 66.6% reveals that even advanced models struggle with the complexities of merging interacting pull requests safely.

Forecasting future coding tasks can yield a dataset that is 58.1% relevant to real-world software engineering needs, sidestepping the pitfalls of historical data replay.

Turns out, LLMs rely far more on raw code access than documentation when answering repository-level questions, challenging the assumption that documentation is the primary driver of code understanding.