Where Does Social Reasoning Come From? Capability Provenance in Language Models

2026-06-17

AI Summary

This study investigates the distinct data origins underlying social and STEM reasoning capabilities in language models. Leveraging the OLMo3-7B model, it pioneers the application of training data attribution methods to trace the provenance of reasoning abilities. By integrating TrackStar—a gradient-based attribution technique built on Bergson—with WebOrganizer’s categorization of the Dolma3 corpus into 576 format-theme bins, the work identifies critical data regions associated with each reasoning type. Causal validation via targeted machine unlearning demonstrates that social and STEM reasoning rely significantly on different corpus segments—e.g., literary content disproportionately influences social reasoning—and that selectively forgetting high-attribution bins markedly degrades performance on corresponding tasks. The project releases all code and data, offering interpretable and actionable causal evidence for how training data shapes model reasoning.

Abstract

We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-reasoning versus STEM-reasoning in OLMo3-7B. Training-data attribution measures how strongly each training document influences a model's predictions on a benchmark, but document-level scores are too noisy to identify which corpus regions support which capabilities, and prior work has emphasized factual knowledge rather than reasoning. We compute gradient-based attribution (TrackStar via Bergson) over a working set drawn from the de-duplicated Dolma3 mix, aggregate influence across WebOrganizer's 24-format x 24-topic taxonomy (576 bins), and contrast benchmark pairs in a 2x2 design that varies domain (social vs. STEM) and capability type (reasoning vs. knowledge): SocialIQA and MMLU Social Sciences against ARC-Challenge and MMLU STEM. Social and STEM reasoning draw on qualitatively distinct corpus regions, and the contrast is sharper at the reasoning level than at the knowledge level. Targeted machine unlearning provides partial causal validation: forgetting high-attribution topic bins (e.g., Literature for SocialIQA) degrades the aligned benchmark more than within-bin random baselines, and we open-source all code, sampling manifests, the bin-level influence matrix, and unlearning checkpoints.

Problem

• social reasoning
• capability provenance
• training-data attribution
• language models
• reasoning vs. knowledge

Innovation

• training-data attribution
• social reasoning
• capability provenance
• machine unlearning
• gradient-based influence