Search papers, labs, and topics across Lattice.
This paper investigates the fragility of learning from self-generated question-answer (QA) pairs, revealing that the generation process is not a neutral preprocessing step but an implicit policy that biases both the selection of evidence and the manner of answering. The authors demonstrate that question generation often focuses on salient spans of text, leading to coverage saturation and vulnerability to artifacts, while answering is influenced by instruction-like passages, with larger models showing increased compliance under task conflict. By implementing strategies such as tying questions to fixed targets and filtering instruction-like spans, they significantly reduce biased selection and improve the robustness of the training signal.
Self-generated QA pairs can introduce significant biases, leading to fragile learning dynamics that may undermine model performance.
Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the resulting pairs are used to fine-tune, distill, or compress knowledge into another model. We show that this generation step is not neutral preprocessing. It is an implicit policy that both selects which evidence becomes training signal and decides how that evidence is answered, and it is fragile at both stages. When choosing what to ask, generators do not scan a document uniformly. Coverage saturates early and concentrates on salient spans, diverse prompts converge on the same regions, and what looks question-worthy is driven by local presentation. As a result, salient artifacts such as poorly cleaned markup can hijack question generation across model families and scales. When answering, the model that produces the supervision tends to obey instruction-like passages embedded in the text. This compliance depends on the intent and surface form of the passage rather than its strictness, and is worst under task conflict, where larger models comply more often. These failure modes arise from choices made during QA generation, so they can be reduced without changing the training loop. Tying each question to a fixed target reduces biased selection, and filtering instruction-like spans before answering lowers mean injection compliance from $88\%$ to $13\%$ in our evaluation while retaining nearly all clean text.
