2 OLMo 2 Furious

2024-12-31

AI Summary

To address bottlenecks in training stability, computational efficiency, and generalization of open-source large language models (LLMs), this work introduces OLMo 2—a new family of fully open-stack LLMs (7B/13B). Methodologically, we propose Dolmino Mix 1124, a novel late-stage curriculum pretraining strategy that improves per-token computational efficiency; Verifiable Reward Reinforcement Learning (RLVR), a framework enabling high-fidelity instruction tuning with rigorous reward verification; and an enhanced dense autoregressive architecture coupled with a relaxed data-instruction tuning paradigm. Our contributions are threefold: (1) full-stack openness—including datasets, code, training logs, and checkpoints; (2) base models achieving Pareto-optimal performance under equivalent compute budgets, significantly outperforming Llama 3.1 and Qwen 2.5; and (3) instruction-tuned models matching or exceeding the performance of leading open models such as Gemma 2. Collectively, OLMo 2 advances reproducible, verifiable, and transparent LLM research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present OLMo 2, the next generation of our fully open language models. OLMo 2 includes dense autoregressive models with improved architecture and training recipe, pretraining data mixtures, and instruction tuning recipes. Our modified model architecture and training recipe achieve both better training stability and improved per-token efficiency. Our updated pretraining data mixture introduces a new, specialized data mix called Dolmino Mix 1124, which significantly improves model capabilities across many downstream task benchmarks when introduced via late-stage curriculum training (i.e. specialized data during the annealing phase of pretraining). Finally, we incorporate best practices from T""ulu 3 to develop OLMo 2-Instruct, focusing on permissive data and extending our final-stage reinforcement learning with verifiable rewards (RLVR). Our OLMo 2 base models sit at the Pareto frontier of performance to compute, often matching or outperforming open-weight only models like Llama 3.1 and Qwen 2.5 while using fewer FLOPs and with fully transparent training data, code, and recipe. Our fully open OLMo 2-Instruct models are competitive with or surpassing open-weight only models of comparable size, including Qwen 2.5, Llama 3.1 and Gemma 2. We release all OLMo 2 artifacts openly -- models at 7B and 13B scales, both pretrained and post-trained, including their full training data, training code and recipes, training logs and thousands of intermediate checkpoints. The final instruction model is available on the Ai2 Playground as a free research demo.

Problem

• Efficient Language Model
• Natural Language Processing
• Transparency and Reproducibility

Innovation

• OLMo 2
• Advanced Model Design
• Efficient Training