Search papers, labs, and topics across Lattice.
This study demonstrates that small language models can achieve grading accuracy comparable to larger models when using a well-defined rubric, highlighting the importance of rubric design in assessment. By evaluating six cost-efficient model configurations against 24 open-ended examination questions, the authors found that the identity of the answer accounted for 95.6% of score variance, while the judge's identity had minimal impact. The findings indicate that the rubric effectively decouples grading from the intelligence of the judge, with the official answer playing a crucial role in maintaining grading reliability.
Rubric-based grading allows small language models to outperform larger models, revealing that grading quality hinges more on structured criteria than on model size or judge intelligence.
Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit rubric. We test this claim as the design principle behind any-to-bench: a frontier model reads source documents once, at ingestion, to extract each question and its rubric; lower-cost models then perform all repeated grading work. We evaluate six cost-efficient model configurations from two model families at three reasoning-effort levels. Each configuration answers 24 open-ended examination questions, and each also grades every answer sheet three times, yielding 3,456 per-question grades. Scores depend overwhelmingly on the answer being graded: answer identity explains 95.6% of score variance, whereas judge identity explains only 0.2%. Raising a writer's reasoning effort moves earned scores by as much as 0.143 of full marks, while raising a judge's reasoning effort moves assigned scores by at most 0.006. Six frontier-tier judges, added as a check, reproduce these scores and are no more reliable as a panel. Two ablations then decompose the rubric on the same questions and answers. Removing its criteria and levels while keeping the official answer changes nothing measurable. Removing the official answer as well collapses reliability (ICC 0.888 to 0.628), inflates scores, and makes judge reasoning effort matter again. The rubric is what decouples grading from judge intelligence, and within the rubric the official answer does nearly all the work. We find no evidence of length preference or same-family preference under rubric-anchored grading.
