Search papers, labs, and topics across Lattice.
This paper tackles the challenge of weak-to-strong generalization by introducing trust functions that assign scalar trust scores to weak labels, effectively filtering unreliable supervision. By applying this method across various domains, the authors demonstrate that students trained with trust-filtered weak supervision can match or even exceed the performance of those trained with ground-truth labels. The iterative process of reusing trained students as teachers further amplifies these gains, showcasing a novel approach to leveraging weak supervision in scenarios with scarce reliable labels.
Trust functions can transform weak supervision into a powerful training signal, enabling models to achieve near-lossless generalization even with unreliable labels.
Weak-to-strong generalization studies how to improve a strong student using supervision from a weaker teacher when reliable labels are scarce. We view this primarily as a data selection problem, where the key challenge is to identify which weak labels are reliable enough to serve as a training signal. To address this, we introduce trust functions that assign each weak label a scalar trust score and use these scores to filter weak supervision. Across several domains, including world knowledge, quantitative reasoning, and strategy games, trust filtering yields students that match and sometimes surpass ground-truth supervision, achieving near-lossless weak-to-strong generalization. Moreover, trust functions enable an iterative weak-to-strong chain that compounds gains by training a student and reusing it as the next teacher, amplifying the gains. There are several mechanisms to which advantage of trust functions can be attributed.
