Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
This paper introduces HARGO, a novel approach for reinforcement learning post-training of large language models (LLMs) that addresses the challenges posed by heterogeneous high-performance computing (HPC) tasks. By employing per-response importance weighting through confidence-modulated advantage, HARGO optimizes for task-specific rewards, significantly improving performance across diverse tasks like data race detection and factual question answering. The results demonstrate that HARGO outperforms traditional uniform-weight RL methods, achieving superior alignment quality and task-specific metrics, including a WinRate of 54.62% and an F1 score of 91.30% for data races.
HARGO achieves unprecedented alignment quality for LLMs on heterogeneous HPC tasks, outperforming traditional methods by leveraging confidence-modulated advantage weighting.
Supervised fine-tuning (SFT) can equip large language models (LLMs) with domain knowledge for high-performance computing (HPC) tasks such as data race detection and benchmark question answering. However, knowledge alone does not guarantee task-appropriate behavior: the same SFT model that correctly classifies 88.65\% of C/C++ data race samples produces verbose, imprecise answers to factual queries, with 65.9\% of MLPerf responses exceeding 40 characters. Reinforcement learning (RL) post-training addresses this gap by optimizing for task-specific rewards rather than token-level imitation. Yet HPC tasks exhibit extreme heterogeneity, with binary classification, factual QA, and semantic generation differing by 58x in answer length, spanning three distinct reward distributions, and showing widely varying SFT accuracy. This makes uniform-weight RL methods such as GRPO suboptimal. We propose HARGO, Heterogeneity-Aware Reward-Guided Optimization, which introduces per-response importance weighting via confidence-modulated advantage: computing a discrimination signal from group-level reward contrast and a confidence signal from reference model log-probabilities, then modulating the advantage before computing per-response weights, without requiring task-type labels. Across four HPC tasks and nine methods, HARGO achieves the best performance on all three primary metrics: WinRate 54.62\%, Data Race F1 91.30\%, and PLP Similarity 0.8558. Ablation confirms complementary contributions from both signals. HARGO establishes the best overall alignment quality among compared methods for heterogeneous HPC tasks.