How Post-Training Shapes Biological Reasoning Models

2026-06-15

AI Summary

This study investigates the mechanisms by which successive post-training stages—continued pretraining (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL)—affect the generalization capabilities of biomedical reasoning models. Through systematic training and evaluation of over 100 models across genomic, transcriptomic, and protein-related tasks, the authors employ a controlled ablation approach to dissect the heterogeneous and stage-specific impacts of each phase on in-distribution (ID) and out-of-distribution (OOD) performance. The findings reveal that CPT enhances alignment with biological language structures, SFT improves ID performance at the cost of OOD generalization, and RL can recover OOD robustness when applied following strong SFT. The optimal strategy combines brief SFT, extended RL, and asymmetric stage capacities, effectively balancing ID accuracy and OOD generalization.

Abstract

Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are built through post-training, yet how each stage shapes reasoning and generalization remains poorly understood. We study when post-training improves performance and when it induces over-specialization. Across genomics, transcriptomics, and proteins, we train and evaluate more than 100 biological reasoning models under controlled variation in backbone, continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL), measuring both in-domain (ID) and out-of-domain (OOD) performance. We find that each post-training stage reshapes generalization in a distinct way rather than contributing uniform gains. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline as models fit the training distribution. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recovers generalization. These results show that biological reasoning does not improve monotonically with additional supervision or compute. Instead, performance depends on how training stages are composed. Under fixed post-training budgets, the strongest ID-OOD trade-off comes from brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.

Problem

• biological reasoning
• post-training
• generalization
• in-domain performance
• out-of-domain performance

Innovation

• post-training
• biological reasoning
• out-of-domain generalization
• reinforcement learning
• supervised fine-tuning