Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs

2026-06-19

AI Summary

This study addresses the underexplored issue that fine-tuning large language models (LLMs) on scientific domains often leads to degraded factual accuracy and heightened hallucination risks, with existing evaluations largely confined to biomedicine and lacking cross-disciplinary granularity. To this end, the authors introduce SciFactCheck—a benchmark and modular evaluation framework—and conduct controlled experiments across five scientific domains on 18 models. Their analysis reveals, for the first time, that fine-tuning systematically exacerbates three hallucination types: unverifiable claims, overconfident assertions, and attribution errors. Notably, models exhibit reduced internal confidence yet produce more assertive language. The study employs minimal-pair prompts, fine-grained hallucination annotations, expert validation, and highlights only moderate agreement between current fact-checking tools and expert judgments, alongside divergent interpretations of scientific verifiability.

Abstract

Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses significant risks in this high stakes use-case. Prior hallucination evaluation work remains largely restricted to the biomedical domain, treats hallucination as a binary task, and has not examined the growing family of scientifically fine-tuned LLMs. We address these gaps with SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, paired with a modular evaluation framework targeting three factuality hallucination types: unverifiability, overclaim, and attribution. Using a controlled minimal-pairing design, we evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base. Our results indicate that 1. Scientifically fine-tuned models exhibit degraded factual reliability across all hallucination types and scientific domains, and 2. Fine-tuned models are internally less confident yet linguistically more assertive. A human pilot study further reveals that current fact-checking tools show only modest agreement with expert judgments on scientific content, and that defining scientifically check-worthy claims remains contested even among human annotators. Our findings fundamentally challenge current methods of domain-specific fine-tuning for factuality and call for developing improved verification infrastructure for scientific content.

Problem

• hallucination
• scientific fine-tuning
• factuality
• large language models
• evaluation benchmark

Innovation

• scientific fine-tuning
• hallucination evaluation
• factuality benchmark
• SciFactCheck
• minimal-pairing design