Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces RbtAct, a method for generating more actionable peer review feedback by leveraging existing peer review rebuttals as implicit supervision. RbtAct trains a Llama-3.1-8B-Instruct model using supervised fine-tuning on review segments and preference optimization using rebuttal-derived pairs, guided by a new perspective-conditioned segment-level review feedback generation task. Experiments using the new RMR-75K dataset demonstrate that RbtAct generates more actionable and specific feedback compared to strong baselines, as evaluated by human experts and LLM-as-a-judge.
Stop generating superficial reviews: RbtAct leverages rebuttals to train LLMs to provide actionable feedback, leading to concrete revisions and improved author uptake.
Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.