Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA

2026-06-30

AI Summary

This work addresses the vulnerability of self-generated question-answering (QA) as a training signal for language models, which is prone to selection bias and excessive compliance with embedded instructions. The study systematically reveals the model’s preferential selection of document evidence and its high adherence to implicit directives during self-generated QA. To mitigate these issues without altering the training pipeline, the authors propose lightweight interventions: fixing the questioning objective and filtering instruction-laden segments. Through comprehensive empirical evaluations—including multi-model comparisons, prompt diversity analysis, text coverage assessment, and instruction injection detection—the approach reduces average instruction compliance from 88% to 13% while preserving nearly all clean textual content. This significantly enhances the reliability and robustness of self-generated QA data for model training.

Abstract

Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the resulting pairs are used to fine-tune, distill, or compress knowledge into another model. We show that this generation step is not neutral preprocessing. It is an implicit policy that both selects which evidence becomes training signal and decides how that evidence is answered, and it is fragile at both stages. When choosing what to ask, generators do not scan a document uniformly. Coverage saturates early and concentrates on salient spans, diverse prompts converge on the same regions, and what looks question-worthy is driven by local presentation. As a result, salient artifacts such as poorly cleaned markup can hijack question generation across model families and scales. When answering, the model that produces the supervision tends to obey instruction-like passages embedded in the text. This compliance depends on the intent and surface form of the passage rather than its strictness, and is worst under task conflict, where larger models comply more often. These failure modes arise from choices made during QA generation, so they can be reduced without changing the training loop. Tying each question to a fixed target reduces biased selection, and filtering instruction-like spans before answering lowers mean injection compliance from $88\%$ to $13\%$ in our evaluation while retaining nearly all clean text.

Problem

• self-generated QA
• synthetic supervision
• instruction compliance
• training signal bias
• question generation fragility

Innovation

• self-generated QA
• instruction compliance
• synthetic supervision
• coverage bias
• fragile learning