Search papers, labs, and topics across Lattice.
This paper introduces Decoupling CLI Agent Scaffolding (DCAS), a novel backend-substitution interception layer that facilitates cross-scaffold evaluation and planning-aware trajectory collection for CLI-based software-engineering agents. By addressing the fine-tuning-induced performance degradation observed when models trained on OpenHands are deployed in different environments, the authors demonstrate that planning quality significantly influences model performance across scaffolds. The results show that models fine-tuned on DCAS-collected trajectories exhibit consistent performance improvements in non-training scaffolds, highlighting the importance of learned planning capabilities over fixed scaffold artifacts.
Fine-tuning on planning-aware trajectories can enhance model performance across diverse CLI environments, mitigating the pitfalls of scaffold-specific training.
CLI-based software-engineering agents have matured rapidly, yet the open ecosystem has converged on a single training environment: trajectory datasets used to fine-tune open models are collected almost exclusively under OpenHands. Models fine-tuned on this data score well under OpenHands but degrade substantially when deployed under any non-training scaffold. Untrained base models do not show this divergence, indicating the gap is fine-tuning-induced and tied to the conventions of the training scaffold. We argue that a load-bearing scaffold-specific behavior is planning structure, in two senses this paper distinguishes: explicit planning, a pre-execution plan produced as a first-class artifact, and implicit planning, the structural conventions that shape execution throughout the agent loop. Under this hypothesis, closing the gap requires moving planning from a fixed scaffold artifact to a learned model capability. We introduce Decoupling CLI Agent Scaffolding (DCAS), a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model without modifying the scaffold, enabling cross-scaffold evaluation and planning-aware trajectory collection. Using DCAS, a controlled plan-source intervention confirms planning quality is a high-leverage component, with gains exceeding the cross-scaffold drops we observe. A model fine-tuned on a small set of DCAS-collected planning-aware trajectories under a single scaffold gains consistently across non-training scaffolds, and the two senses of planning are empirically separable in training data.
