Search papers, labs, and topics across Lattice.
This paper introduces SoftSkill, a method that utilizes a compact continuous context object initialized from natural-language skills, allowing for the refinement of behavior through a trainable soft delta while keeping the base model frozen. The approach significantly enhances performance on various tasks, achieving notable improvements in accuracy on SearchQA, LiveMath, and DocVQA compared to traditional prompting methods. By replacing extensive Markdown skill tokens with a minimal number of virtual tokens, SoftSkill demonstrates a more efficient way to leverage agent skills in language models, suggesting a shift in how task skills are integrated into model inference.
SoftSkill achieves up to 42.1 points improvement on LiveMath by transforming lengthy Markdown skills into a few powerful virtual tokens.
Agent skills are commonly deployed as natural-language Markdown files that encode answer policies, evidence-use habits, and task procedures. These files are readable and portable, but they are consumed indirectly: for each task instance, a frozen language model must translate a long textual artifact into generation-time behavior. This paper asks whether a natural-language skill can instead initialize a compact continuous context object, refined by a trainable soft delta while the base model remains frozen. We propose SoftSkill, a frozen-backbone method that tunes such soft skills with next-token prediction and deploys them as latent behavioral priors at inference time. In our main single-round setting, a length-32 SoftSkill prefix on Qwen3.5-4B improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA. Relative to SkillOpt, SoftSkill improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath, while replacing hundreds to thousands of Markdown skill tokens with a few virtual tokens. We further study agentic execution as a harder boundary case, where sparse trajectory imitation provides useful signal but does not yet robustly compress long-horizon procedural behavior. More broadly, the results suggest that some task skills are better treated not as additional Markdown to be reinterpreted at inference time, but as compact latent controls over how a frozen model enters the task.
