Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

2026-08-16

AI Summary

This study addresses the challenges of expensive objective optimization within open-ended hypothesis spaces and unreliable generative model evaluation by proposing the "Large Discovery Model." This approach establishes an experience-driven loop architecture that couples generative models with Bayesian nonparametric surrogates, enabling candidate generation and online adaptive updates through uncertainty-aware value guidance to facilitate efficient open-ended scientific search. Experimental results demonstrate the model’s superiority in complex scientific discovery tasks, achieving a 2.4-fold reduction in bits-per-byte (bpb), an 18.2% decrease in antibody binding energy, and over a 60% improvement in multi-objective molecular performance. These findings validate the framework's effectiveness in navigating vast search spaces while mitigating evaluation uncertainties, marking a significant advancement in automated scientific exploration.

Abstract

Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a $2.4\times$ greater reduction in validation BPB, an $18.2\%$ relative decrease in binding energy, and more than $60\%$ relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.

Problem

• Scientific Discovery
• Open-Ended Search
• Generative Models
• Expensive-to-evaluate Objectives
• Hypothesis Spaces

Innovation

• Large Discovery Model
• Bayesian non-parametric surrogate
• Uncertainty-aware search
• Open-ended hypothesis space
• Continual learning