Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the behavior of large language models (LLMs) when editing code originally generated by different models, revealing that they tend to make excessive edits on foreign code due to stylistic differences. To address this issue, the authors introduce CROCODIL, a post-training framework that employs a similarity reward to penalize large changes and an execution reward to ensure functional correctness. The key finding shows that CROCODIL effectively reduces excessive edits while maintaining high success rates in code editing tasks.
LLMs can significantly over-edit code generated by other models, but CROCODIL curbs this tendency without sacrificing performance.
Large language models (LLMs) have become ubiquitous tools for code generation and editing. However, development teams often use multiple LLM assistants. Different developers may prefer different models, and individual developers may switch between models across different coding sessions. Because of this, the edits any one model makes are frequently applied to foreign code originally generated by another model. These LLMs are often trained on different datasets, and as a result have different stylistic preferences. Do LLMs behave differently when they edit foreign code originally written by a different LLM with a different coding style? We find that models tend to make more, and often excessive, edits on foreign code. We introduce CROCODIL (Cross-model Code Editing with LLMs), a post-training framework for reducing excessive edits while preserving functional correctness. CROCODIL's similarity reward penalizes large changes, while its execution reward scores build and test success. We use the product of these two rewards to encourage the policy to decrease the edit size without decreasing the edit task success rate. CROCODIL is available at https://github.com/EngineeringSoftware/Crocodil.