Wei-Jung Huang

Relying on partial evaluations can lead to misleading conclusions, but ParEvalLayer ensures accurate decision-making with as little as 25% of the data.

Partial evaluations can mislead if not carefully calibrated, with required task fractions varying dramatically across benchmarks—15% for AppWorld but 95% for SWE-bench Lite.

LLM-generated skills fail to outperform basic task prompts in data science workflows, challenging the assumption that automated skill generation enhances AI performance.

CI/CD workflows are riddled with reliability issues, with over 434,000 anti-patterns identified across thousands of projects, highlighting the urgent need for improved observability and recommendations.