David Lo

Coding agent benchmarks can now dynamically scale difficulty and avoid data leakage by having agents evaluate each other, with the latest models generating tasks that defeat older solvers 10x more often.

In-code comments only help LLMs when they smuggle in correct downstream solution content that the model could not otherwise elicit, debunking the idea that self-commentary acts as general chain-of-thought scaffolding.

Vulnerabilities spanning multiple functions can be detected more accurately with VulAgentRL, which verifies evidence through a novel Code Property Graph approach.

Achieving up to 3.97x speedup in human-written code, Optimo redefines how we optimize dynamic languages by employing a multi-level approach that outperforms traditional methods.

LLM agents are not just chatbots: they can find hundreds of real-world software vulnerabilities that traditional tools miss.

Security patch detectors trained on standard vulnerability databases are practically useless in the real world, losing up to 90% F1-score when deployed on in-the-wild data.