Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper explores specializing small language models (SLMs) for generating Kubernetes manifests using a novel context-instrumental data distillation method. The method involves synthetic data generation and reverse instruction generation from real YAML files, filtering training pairs based on external validation and domain context matching. Experiments fine-tuning Qwen2.5-Coder-1.5B-Instruct via LoRA on a distilled corpus achieved 91.5% full-pass@1 accuracy, demonstrating the importance of strict output formatting over sheer data volume.
Forget scaling up: nailing the output format is the real secret to getting small language models to generate perfect Kubernetes YAML.
This paper examines the specialization of Small Language Models (SLMs) with up to 4 billion parameters for generating artifacts in domain-specific languages (DSL). Kubernetes manifests are chosen as the target domain. We propose the context-instrumental data distillation method: the source corpus is formed through synthetic generation and, in an extended scheme, through reverse instruction generation from real Kubernetes YAML files, with pairs included in training only upon passing external validators and matching the domain context model. Unlike classical KL-divergence knowledge distillation, the baseline implementation reduces to supervised fine-tuning on instrumentally verified examples. The experimental section presents a pilot implementation under resource-constrained conditions: the DeepSeek-V4 Flash API serves as the teacher for synthetic generation, while Qwen2.5-Coder-1.5B-Instruct is fine-tuned via LoRA on CPU. On the K8s-Distill-Pilot corpus (train_1200, validation_100, test_200), we achieved full-pass@1 = 91.5% (183/200) with a stricter prompt formulation and max_new_tokens=768. The key empirical finding is that for Kubernetes YAML, result quality in the pilot depended more on strict output format requirements than on simply increasing the number of training examples.