Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

2026-06-15

AI Summary

Natural scientific research faces challenges stemming from task heterogeneity and model specialization, lacking a unified modeling framework. This work proposes LOGOS, an autoregressive language model grounded in a shared scientific grammar that encodes diverse scientific entities and their spatial interactions into discrete token sequences. By modeling complex structural relationships purely through sequential representations, LOGOS reformulates various downstream tasks as next-token prediction problems within a common syntactic space. Notably, it achieves the first unified cross-domain modeling across natural sciences without relying on coordinate systems or geometric networks, effectively capturing spatial constraints and contact patterns. Trained at multiple scales (1B, 3B, and 8B parameters), LOGOS matches or surpasses domain-specific baselines across numerous tasks, demonstrating the feasibility of a single model for general-purpose scientific reasoning and advancing AI for Science (AI4S) through open-sourced model weights.

Abstract

In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.

Problem

• generative foundation model
• natural sciences
• unified representation
• scientific tasks
• large language models

Innovation

• scientific foundation model
• unified scientific grammar
• discrete spatial tokenization
• autoregressive science modeling
• AI for Science