Search papers, labs, and topics across Lattice.
This paper introduces HSS-Synth, a novel data synthesis pipeline specifically designed for the humanities and social sciences (HSS), addressing the scarcity of high-quality data for large language models (LLMs). By employing a subject-centric approach, the authors construct seed documents through a multi-step filtering process and generate diverse instructions that maintain semantic integrity via teacher-forced answering. The resulting dataset of 237k instruction-tuning samples significantly outperforms 14 leading baselines across 16 benchmarks, setting a new state-of-the-art for the fine-tuned Qwen3-8B-Base model.
HSS-Synth generates 237k high-quality instruction samples that not only outperform existing baselines but also redefine data synthesis for the humanities and social sciences.
High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. Data synthesis is a viable alternative and succeeds on closed tasks, yet the humanities and social sciences (HSS) are overlooked, and their open-ended nature makes synthesis challenging. Moving beyond prior capability-centric, fragmented attempts, we adopt a subject-centric paradigm, define the first HSS domain system covering 14 mainstream fields, and introduce HSS-Synth, the first data synthesis pipeline for HSS. HSS-Synth comprises: (1) constructing seed documents from web corpora via multi-step filtering and text refinement evaluated by a judge; (2) specifying"requirements + persona"to backtranslate seed documents into diverse yet faithful instructions with a strict Q&A alignment check; and (3) breaking LLM response limits via teacher-forced Answering that feeds seed documents during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. HSS-Synth yields 237k high-quality, diverse instruction-tuning samples that outperform 14 leading baselines on 16 benchmarks. The fine-tuned Qwen3-8B-Base sets a new SOTA and approaches the official Qwen3-8B, improving both human preference and knowledge capabilities without performance seesaws. Extensive experiments demonstrate HSS-Synth's robustness and transferability. Our code is publicly available at https://github.com/pengr/HSS-Synth.
