Less is More: Quality-Aware Training Data Selection for Scientific Summarization

2026-06-23

AI Summary

Existing scientific long-document summarization datasets suffer from inconsistent summary quality, limited scale, and structural incompatibility with modern long-context models. To address these limitations, this work constructs a large-scale biomedical and life sciences summarization dataset comprising 1.88 million PMC articles and introduces a summary quality assessment framework based on source alignment and model-driven metrics to curate a high-quality subset. Experimental results demonstrate that, under identical training scales, models trained on this high-quality subset significantly outperform those trained on randomly sampled data; notably, they even surpass larger models trained on random subsets in factual consistency metrics. These findings validate the efficacy of a quality-first data selection strategy for scientific summarization.

Abstract

Scientific long-document summarization datasets commonly treat author-written abstracts as gold reference summaries, although their quality and alignment with the source article vary. At the same time, publicly available scientific summarization datasets remain limited in scale and structure for modern long-context models. In this work, we address both challenges by a) constructing and releasing one of the largest biomedical and life science datasets for long-document summarization, containing 1.88 million PMC articles, and b) analyzing the reference quality of author-written abstracts with source-grounded and model-based metrics. We show that author-written abstracts vary in their alignment with the full article and that these quality signals can guide training-data selection. Training on selected high-quality subsets outperforms random sampling at matched training sizes and can match or exceed larger random subsets on factuality-oriented metrics. Our findings suggest that reference quality is an important factor in scientific summarization and that quality-aware data selection can improve training efficiency.

Problem

• scientific summarization
• training data selection
• reference quality
• long-document summarization
• author-written abstracts

Innovation

• quality-aware data selection
• scientific summarization
• long-document summarization
• reference quality evaluation
• training efficiency