Code Generation & Program Synthesis - Weekly Roundup

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

State-of-the-art coding agents exhibit critical boundary violations, with nearly all misinterpreting dependencies and over half compromising environment integrity or user inputs.

Aesthetic code generation is unlocked by using a VLM judge to translate visual design principles from a reference image into actionable rewards for code models.

Fine-tune robot simulation physics with unprecedented interactive control, enabling AI agents to learn from real-world interactions and achieve accurate real-to-sim transfer.

Coding agents routinely fail to build agents that truly meet user requirements, with the gap between demonstration and delivery being the norm.

Terminal agents can learn to overcome their own mistakes, achieving a 14.6-point Pass@1 improvement on complex coding benchmarks by recovering from failures.

State-of-the-art coding agents exhibit critical boundary violations, with nearly all misinterpreting dependencies and over half compromising environment integrity or user inputs.

Aesthetic code generation is unlocked by using a VLM judge to translate visual design principles from a reference image into actionable rewards for code models.

Fine-tune robot simulation physics with unprecedented interactive control, enabling AI agents to learn from real-world interactions and achieve accurate real-to-sim transfer.

Coding agents routinely fail to build agents that truly meet user requirements, with the gap between demonstration and delivery being the norm.

Human GUI interactions can be precisely captured and translated into reusable agent skills, dramatically boosting performance on unseen software and workflows.

Spatial coding agents can be distilled into standalone MLLMs, achieving superior spatial reasoning and generalization without external tools.

LEGO-building AI agents can assemble physically valid structures from text prompts, but their designs lag significantly behind human creativity and complexity.

Robots can achieve 70% success on complex bimanual tasks using only code for decision-making, bypassing runtime vision-language models entirely.

Weisfeiler-Lehman graph kernels unlock superior structural features for algorithm selection, outperforming traditional methods on challenging constraint satisfaction problems.

LLMs can now generate executable feature programs from long text, enabling efficient structured data prediction without real-time LLM inference.

LLM test generation quality is overestimated by 30% when evaluated against single references, with a new metric revealing widespread missed behaviors and faulty assertions.

Most AI agent skills copied on GitHub originate from a few hidden hubs, rendering security patches ineffective for the vast majority of users.

Coding agent benchmarks can now dynamically scale difficulty and avoid data leakage by having agents evaluate each other, with the latest models generating tasks that defeat older solvers 10x more often.

Synthesized software environments unlock a 50% boost in CUA performance on unseen benchmarks, proving scalable supervision is key.

Automated kernel discovery now yields reusable inductive biases that generalize beyond training data, outperforming meta-learned deep kernels and even human-refined versions on unseen tasks.

Small Language Models can parse ML pipeline structures, but their insights into data science practices diverge significantly from other classifiers, highlighting the critical impact of taxonomy definition.

Current LLMs struggle to translate visual scenes into executable geometric functions, failing even basic trajectory generation tasks.

Key contribution not extracted.

Real-world coding agent usage reveals that task flow, not just task completion, dramatically impacts agent cost and effectiveness, suggesting current benchmarks are misaligned with user behavior.