Search papers, labs, and topics across Lattice.
This paper introduces judgment-grounded expansion, a collaborative approach to automatic review generation that integrates human evaluative claims with AI-generated review comments. By modeling the process as a structured generate-check-refine cycle, the authors address key challenges in scalable evaluation and candidate set curation, demonstrating that conformal prediction effectively balances candidate size and coverage. The findings establish a new framework for enhancing accountability in automated peer review processes, paving the way for more reliable human-AI collaboration in scientific evaluation.
Human evaluative claims can significantly enhance the quality of AI-generated peer reviews, striking a crucial balance between automation and accountability.
Automatic review generation is a promising direction for accelerating scientific progress. While most work adopts an end-to-end setup, its fully automated nature makes it less suitable for settings that demand accountability. To better balance automation and accountability, we formalize judgment-grounded expansion, a human-AI collaboration mode where a reviewer provides an evaluative claim and the system expands it into review comment candidate(s). We model it as a structured generate-check-refine process and conduct a user study to collect human-model interaction data. We study two practical challenges for judgment-grounded expansion: scalable evaluation and candidate set curation. We develop methods to simulate the process for large-scale evaluation, and show that conformal prediction is well suited to balancing candidate set size and target coverage. Our work establishes judgment-grounded expansion as a concrete task and provides empirical and methodological foundations for the design of future collaborative review generation systems.
