Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an agentic evaluation architecture for detecting historical biases in textbooks, comprising a multimodal screening agent, a jury of evaluative agents, and a meta-agent. A key innovation is the Source Attribution Protocol, which differentiates textbook narrative from quoted historical sources to avoid misattributions. Experiments on Romanian history textbooks show the architecture significantly reduces false positives compared to zero-shot baselines, achieving 83.3% classification of excerpts as pedagogically acceptable versus 5.4% for the baseline, and is preferred by human evaluators in 64.8% of cases.
Agentic evaluation slashes false positives in historical bias detection, making textbook auditing scalable and economically viable.
History textbooks often contain implicit biases, nationalist framing, and selective omissions that are difficult to audit at scale. We propose an agentic evaluation architecture comprising a multimodal screening agent, a heterogeneous jury of five evaluative agents, and a meta-agent for verdict synthesis and human escalation. A central contribution is a Source Attribution Protocol that distinguishes textbook narrative from quoted historical sources, preventing the misattribution that causes systematic false positives in single-model evaluators. In an empirical study on Romanian upper-secondary history textbooks, 83.3\% of 270 screened excerpts were classified as pedagogically acceptable (mean severity 2.9/7), versus 5.4/7 under a zero-shot baseline, demonstrating that agentic deliberation mitigates over-penalization. In a blind human evaluation (18 evaluators, 54 comparisons), the Independent Deliberation configuration was preferred in 64.8\% of cases over both a heuristic variant and the zero-shot baseline. At approximately \$2 per textbook, these results position agentic evaluation architectures as economically viable decision-support tools for educational governance.