Search papers, labs, and topics across Lattice.
This paper introduces BridgeAlign, a novel preference-alignment pipeline specifically designed for the nuanced requirements of humanities and social sciences (HSS) tasks. By employing a three-phase approach that includes seed curation, preference data synthesis, and preference optimization, the authors generate over 210,000 synthetic preference samples that enhance the performance of the Qwen3-8B model across 17 benchmarks. The key finding reveals that BridgeAlign achieves superior results in both human-preference and knowledge-based capabilities without compromising either, marking a significant advancement in aligning LLMs with HSS quality judgments.
BridgeAlign achieves unprecedented alignment in humanities and social sciences, enabling LLMs to excel in both human-preference and knowledge-based tasks simultaneously.
While data synthesis for large language models (LLMs) is prevalent, it primarily targets domains with verifiable answers, overlooking open-ended humanities and social sciences (HSS), where nuanced quality judgments matter more than objective correctness. This makes preference alignment a natural paradigm for broad HSS tasks. Yet existing methods are either costly or not tailored to broad HSS disciplines. We thus propose BridgeAlign, among the first preference-alignment pipelines for broad HSS disciplines, with three phases: i) Seed Curation: curating HSS seed documents from web corpora via heuristic/LLM-based filtering and text refinement; ii) Preference Data Synthesis: generating preference triplets via persona-based instruction inversion with Q&A consistency checks; iii) Preference Optimization: moving beyond naive human-vs-model heuristics by first grounding preferences in HSS quality rubric, then generating transitional responses via controlled quality degradation to form near-boundary preference pairs for finer-grained quality discrimination. Aligning over 210k synthetic preference samples, BridgeAlign enables Qwen3-8B to achieve the best average across 17 benchmarks against 11 strong baselines; importantly, leading on both human-preference and knowledge-based capabilities at once, with no trade-off between them, as supported by extensive experiments and contextualized by existing theories.
