Search papers, labs, and topics across Lattice.
This study investigates the ability of language models to simulate student writing by introducing SWIM, a task that focuses on proficiency-conditioned essay generation. The research evaluates various methods, including prompting, supervised fine-tuning (SFT), and reinforcement learning (RL), to assess their effectiveness in achieving alignment with different writing proficiency levels. Results indicate that while SFT significantly enhances alignment, RL with a proficiency-alignment reward further improves performance across writing traits, highlighting the challenges in replicating low-proficiency writing styles.
Language models can struggle to mimic the nuanced variations in student writing proficiency, but reinforcement learning with a tailored reward can bridge this gap.
Writing proficiency manifests in how students develop content, organize ideas, choose words, and use language. Despite growing interest in LLM-based student simulation, whether LLMs can reproduce such multidimensional variation in extended writing remains largely unexplored. In this work, we explore if language models can realistically simulate student writing, and introduce SWIM, a task that formulates Student Writing sIMulation as proficiency-conditioned essay generation. We evaluate prompting, supervised fine-tuning (SFT), and reinforcement learning (RL) methods for writing simulation using automated essay scoring as a measure of profile alignment. Extensive experiments reveal that prompting provides limited proficiency control, even for strong proprietary LLMs with rubric-grounded strategies. In particular, while models can adjust content-oriented traits, they struggle to reproduce the lexical, grammatical, and organizational variation in different proficiency levels. SFT substantially improves alignment, while RL with the proposed proficiency-alignment reward yields further gains across all writing traits and essay prompts. Our findings suggest that explicit supervision enables substantially stronger profile alignment than prompting alone, while authentic low-proficiency writing remains challenging to reproduce.
