Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces pAI-Econ-claude, a gated, human-in-the-loop multi-agent architecture designed to enhance the reliability of AI-assisted economic theory development in scenarios lacking machine-readable correctness signals. By organizing agent coordination through inspectable intermediate records and implementing specialized gates for diagnosing failure modes, the architecture significantly reduces mean failure severity from 1.58 to 1.16 while increasing overall usefulness from 2.60 to 3.10 in five matched economic-theory tasks. The findings suggest that gated oversight enhances auditability without replacing formal verification, highlighting the importance of human judgment in decision-making processes.
Gated oversight in AI-assisted economic theory development can improve auditability and reduce failure severity without formal verification.
In many social-science research tasks, such as economics, LLM-based agents must produce outputs for which no cheap, task-complete, machine-readable correctness signal exists. This creates a distinctive reliability problem for multi-agent systems: how should generation, critique, coordination, and human judgment be organized when no component can certify the final result? We address this problem through pAI-Econ-claude, a gated, human-in-the-loop multi-agent architecture for AI-assisted economic theory development. Agents coordinate through a shared workspace of inspectable intermediate records; specialized gates diagnose targeted failure modes and recommend loopbacks without certifying correctness; and human checkpoints retain authority over decisions that are costly to reverse. We evaluate the architecture on five matched economic-theory tasks against an ungated baseline. Two evaluators blinded to configuration agreed on all five pairwise rankings, preferring the gated architecture in four tasks and the baseline in one. Mean failure severity fell from 1.58 to 1.16, while overall usefulness rose from 2.60 to 3.10. The largest observed gain occurred when a reality check rejected a false market-structure premise and a proof review prompted revision of a false welfare claim. The negative case shows that scaffolding can also compress an economically important mechanism too aggressively. The results support a bounded claim: gated oversight improves the auditability of AI-assisted economic theory without substituting for formal verification, and the allocation of irreversible human judgment is a more informative design variable than pure agent autonomy. The workflow is publicly available at https://github.com/maxwell2732/pAI-Econ-claude.