Search papers, labs, and topics across Lattice.
AuthTrace is introduced as a diagnostic benchmark for evidence construction systems, leveraging single-author corpora to provide a unified evaluation framework across different paradigms like chunk retrieval, agent memory, and knowledge-graph traversal. The benchmark consists of 2,099 instances with exhaustive gold evidence and a fan-in gradient, enabling detailed analysis of system performance. Experiments on eight systems reveal that evidence recall is the dominant factor for answer quality, fan-in exposes paradigm-specific failure modes, and full-context prompting alone is insufficient.
Evidence recall, not precision, is the unsung hero of QA systems, explaining 96% of answer quality variance.
Evidence construction systems--chunk retrieval, agent memory, knowledge-graph traversal, and thematic indexing--are evaluated on separate benchmarks with incompatible corpora and metrics, making cross-paradigm diagnosis impossible. We introduce AuthTrace, the first diagnostic benchmark that places all major paradigms on a single corpus and query set by exploiting the dual nature of single-author collections. Built on thematically dense corpora where all texts share style, topic, and vocabulary, AuthTrace provides 2,099 instances with exhaustive gold evidence and a fan-in gradient as the primary diagnostic axis. Comparing eight systems across two QA models, we find that (1) evidence recall--not precision--is the dominant predictor of answer quality (r = 0.96); (2) fan-in exposes paradigm-specific collapse patterns, with flat retrieval degrading 3x faster than structured-evidence systems; and (3) full-context prompting fails uniformly, establishing evidence construction as a necessary capacity beyond raw corpus exposure.
