Search papers, labs, and topics across Lattice.
This paper introduces a MinHash-based similarity scheduling framework for fine-tuning large language models (LLMs) in the context of neural architecture search (NAS), focusing on constructing a progressive curriculum over neural architecture code. By utilizing 128-permutation MinHash signatures to create similarity bands, the framework presents architectures with increasing heterogeneity, achieving a peak success rate of 60% at high similarity levels without the need for post-processing repair. The study highlights the distinct failure modes targeted by curriculum scheduling and interface repair, revealing that while the base model outperforms the curriculum model without repair, both approaches converge in performance when partial repair is applied.
Curriculum fine-tuning can significantly enhance the success rate of LLMs in neural architecture synthesis, but distinct failure modes require different repair strategies.
Introduce a MinHash-based similarity scheduling framework that constructs a progressive curriculum over neural architecture code for LLM-based neural architecture search (NAS). Using 128-permutation MinHash signatures over normalised 7-gram source code shingles, we partition the reference pool into similarity bands and present them in increasing architectural heterogeneity, with the best LoRA adapter from each stage merged cumulatively into the backbone. We evaluate the framework on OlympicCoder-7B within the LEMUR benchmark on CIFAR-10 image classification, generating N =15 candidate architectures per epoch across six progressive fine-tuning steps. The curriculum achieves 60% peak success rate at the high-similarity level without post-processing repair. A 2*2 ablation at the most diverse level curriculum versus base model, with versus without partial interface repair reveals that without repair the base model (47% peak SR) substantially outperforms the curriculum model (7% SR), while adding partial repair brings both to 53% SR. This pattern is consistent with merge-level weight drift progressively erasing evaluator-interface priors, and suggests that interface repair and curriculum scheduling target distinct failure modes. We further report a cross-dataset transfer observation on SVHN, where direct base-model generation without curriculum warmup yields 27% peak SR at substantially lower accuracy (60.5%) than the CIFAR-10 equivalent, consistent with the increased synthesis difficulty of the unq-family anchor architecture.
