Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a reinforcement learning approach to train LLMs to edit argumentative text in a more human-like manner, focusing on generating self-contained, meaning-preserving sentence-level edits. The method uses group relative policy optimization with a multi-component reward function that balances edit-level semantic similarity, fluency, and pattern conformity with argument-level appropriateness. Experiments show the approach outperforms baselines in human-like editing, achieving appropriateness levels comparable to full rewriting through multi-round editing.
LLMs can learn to edit text like humans, making targeted, meaning-preserving changes to improve appropriateness, rather than scattered, disruptive edits.
Editing human-written text has become a standard use case of large language models (LLMs), for example, to make one's arguments more appropriate for a discussion. Comparing human to LLM-generated edits, however, we observe a mismatch in editing strategies: While LLMs often perform multiple scattered edits and tend to change meaning notably, humans rather encapsulate dependent changes in self-contained, meaning-preserving edits. In this paper, we present a reinforcement learning approach that teaches LLMs human-like editing to improve the appropriateness of arguments. Our approach produces self-contained sentence-level edit suggestions that can be accepted or rejected independently. We train the approach using group relative policy optimization with a multi-component reward function that jointly optimizes edit-level semantic similarity, fluency, and pattern conformity as well as argument-level appropriateness. In automatic and human evaluation, it outperforms competitive baselines and the state of the art in human-like editing, with multi-round editing achieving appropriateness close to full rewriting.