Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces the AI Textbook Auditor, a modular multi-agent system designed to automate the quality assurance of educational materials by auditing for factual accuracy, technical correctness, and linguistic quality. Utilizing a dual-track approach, the system employs specialized LLM agents to identify errors in content and grammar, while a Judge Agent filters out false positives based on domain-specific rules. Tested on Romanian upper-secondary textbooks, the system achieved a precision of 62.5% in identifying technical findings, significantly reducing the manual effort required for quality assurance in educational content.
Automated auditing of textbooks reveals a staggering 56 technical errors in a single computer science textbook, showcasing the potential for AI to enhance educational material quality.
Ensuring the quality of educational materials requires more than standard proofreading: textbooks must be audited for factual accuracy, domain-specific technical correctness, and linguistic quality simultaneously -- a task that general-purpose grammar checkers cannot address. We present \textbf{AI Textbook Auditor}, a modular multi-agent pipeline for automated quality assurance of educational materials across subject domains. The system accepts a textbook PDF and produces a structured, human-reviewable report via two analysis tracks: a \textbf{Factual and Technical Track} in which an ensemble of specialized LLM agents detects factual inaccuracies, code errors, incorrect definitions, and conceptual inconsistencies, augmented with web search for humanities domains; and a \textbf{Grammar Track} operating PDF-natively to preserve diacritical encoding. A \textbf{Judge Agent} filters false positives using domain-specific rules before presenting findings to a human reviewer. The pipeline supports two ingestion modes -- vision-native page rendering and PyMuPDF text extraction -- and is domain-adaptable via custom prompts encoding subject-specific error taxonomies. We demonstrate the system on two Romanian upper-secondary textbooks: a CS textbook (56 technical findings across seven categories, with an expert-validated precision of 62.5\%) and a history and social sciences textbook (72 findings spanning factual errors, ideological bias, and grammar). The system is designed as a triage tool that reduces the manual effort of locating candidate issues, with human expert validation required before any editorial action.