Norihiro Yoshida

Judging AI coding agents solely by merged/rejected pull requests is misleading: over 60% of rejections aren't the agent's fault, and even "successful" merges often hide significant human intervention.

Closed-world testing assumptions crumble when faced with the infinite possibilities of open-world games, demanding a new paradigm focused on characterizing behavior under uncertainty.