Search papers, labs, and topics across Lattice.
This study investigates the cognitive plasticity of open-weight large language models (LLMs) through behavioral reprogramming to foster a proactive, Socratic dialogue style. By conducting a hyperparameter sweep across 405 high-performance computing jobs, the authors establish mathematical bounds for parameter-efficient fine-tuning (PEFT), identifying an optimal LoRA rank of 16 and a training window of 2 to 3 epochs for maximum generalization capacity. The results reveal that scaling to 14B parameters leads to improved performance metrics, including lower perplexity and successful cross-lingual persona transfer, highlighting the potential for robust alignment in diverse linguistic contexts.
Achieving a proactive Socratic dialogue style in LLMs reveals that fine-tuning can significantly enhance cognitive flexibility and alignment across languages.
Large language models (LLMs) are predominantly aligned to function as passive, sycophantic assistants. We challenge this default paradigm by empirically evaluating the cognitive plasticity of open-weight architectures when subjected to rigorous behavioral reprogramming. Our objective is to induce a proactive, Socratic conversational framework, characterized by high-frequency question generation under strictly constrained high-performance computing (HPC) conditions. Through a massively parallelized hyperparameter sweep comprising 405 HPC jobs, we define precise mathematical bounds for parameter-efficient fine-tuning (PEFT). We identify an architectural threshold at LoRA rank $r=16$ and demonstrate via extensive epoch ablation that generalization capacity strictly reaches its optimal convergence within an optimized training window of $e \in [2, 3]$ depending on dataset density (minimum validation loss of 0.919). Furthermore, scaling model capacity to 14B parameters yielded a lower localized evaluation perplexity (1.414). Subsequent Direct Preference Optimization (DPO) successfully decoupled the underlying assertive behavior from localized syntax, while rigorous cross-lingual stress testing reveals both the capabilities and the structural boundaries of zero-shot persona transfer, demonstrating robust alignment in closely related linguistic families alongside identifiable degradation pathways in morphologically distant targets. These findings establish a rigorous empirical framework for compute-efficient, cross-lingual behavioral modification.
