Search papers, labs, and topics across Lattice.
This paper introduces SWEzze, a framework for compressing code context to improve the performance of LLMs on GitHub issue resolution. It uses Oracle-guided Code Distillation (OCD) to identify minimal code subsequences required for successful fixes, then fine-tunes a lightweight model to compress code context at inference time. Experiments on SWE-bench Verified demonstrate that SWEzze achieves a 6x compression rate, reduces token budget by 51.8%-71.3%, and improves issue resolution rates by 5.0%-9.2% compared to uncompressed settings and outperforms existing compression baselines.
LLMs fix more bugs when you feed them *less* code, thanks to a new compression technique that distills context to the minimal, crucial snippets.
Large Language Models (LLMs) are now capable of resolving real-world GitHub issues. However, current approaches overapproximate the code context and suffer from two compounding problems: the prohibitive cost of processing massive inputs, and low effectiveness as noise floods the context window and distracts the model from the bug-fixing signal. Existing compression techniques fail to resolve this tension: generic compressors compromise the semantic integrity of code, while code-specific tools lack awareness of code structure and task context to preserve essential patch ingredients. To address this, we propose a novel framework consisting of two components. First, Oracle-guided Code Distillation (OCD), a context distillation algorithm that combines genetic search and delta debugging to systematically reduce code contexts to their minimal sufficient subsequence - retaining only the ingredients required for a successful fix. We use this distilled data to fine-tune SWEzze, a lightweight model that learns to compress code context at inference time, filtering noise and combating distraction while preserving fix ingredients. Evaluated on SWE-bench Verified across three frontier LLMs, SWEzze maintains a stable compression rate of about 6 times across models, reduces the total token budget by 51.8%-71.3% relative to the uncompressed setting, improves issue resolution rates by 5.0%-9.2%, and delivers the best overall balance among effectiveness, compression ratio, and latency compared with state-of-the-art context compression baselines.
