Search papers, labs, and topics across Lattice.
This study analyzes sentiment evolution in multi-round peer reviews by employing deep learning techniques to classify aspect-level sentiments from comments on 11,063 accepted papers. By constructing a manually annotated corpus of 5,000 review sentences, the researchers developed the LCF-BERT-CDM model, which achieved a Macro-F1 score of 82.65%. The findings reveal a significant trend where positive sentiments increase and negative sentiments decrease with more review rounds, highlighting key aspects like "experiments" and "research significance" as critical indicators of sentiment shifts.
As peer reviews progress, positive sentiment rises significantly, challenging the notion that critique intensifies with scrutiny. WHY_IT MATTERS: This insight could transform how we understand the peer review process, potentially leading to improved evaluation strategies in scientific publishing.
Mining sentiment information from the textual content of peer review comments offers valuable insights into the scientific evaluation process. However, previous studies are often constrained by coarse-grained analysis and the lack of differentiation across review rounds. Notably, the dynamic shifts in reviewers' focus and sentiment tendencies throughout multiple review stages remain underexplored. To address this gap, the present study investigates the distribution and evolution of aspect-level sentiments and examines their correlation with the number of review rounds. We begin by segmenting the multi-round review comments of 11,063 accepted papers from Nature Communications and identifying fine-grained review aspect clusters. A manually annotated corpus of approximately 5,000 review sentences is then constructed. Using this dataset, we train a series of deep learning-based aspect sentiment classification models. Among them, the LCF-BERT-CDM model achieves the best performance, with a Macro-F1 score of 82.65%. Subsequent statistical analysis reveals a consistent trend: as the number of review rounds increases, the proportion of positive sentiments rises, while negative sentiments decline. Correlation analysis further indicates that aspect sentiment scores are negatively associated with the total number of review rounds. Key aspects exhibiting stronger correlations include "experiments", "research significance" and "result analysis".
