Beyond Adam: SOAP and Muon for Faster, Label-Efficient Training of Machine Learning Interatomic Potentials

2026-07-02

AI Summary

This work addresses the long-standing reliance on Adam-family optimizers in training machine learning interatomic potentials (MLIPs), which has constrained improvements in both convergence speed and accuracy. For the first time, we introduce matrix-structured optimizers—SOAP, Muon, and their hybrid SOAP-Muon—into the training of NequIP and Allegro models, systematically investigating their impact on MLIP performance, particularly under partial force supervision and limited labeled data. Our experiments demonstrate that SOAP and SOAP-Muon significantly outperform Adam, achieving notable gains in both convergence rate and final prediction accuracy. These findings reveal optimizer selection as a critical yet overlooked design dimension for enhancing the efficiency and precision of MLIPs.

Abstract

Machine learning interatomic potentials (MLIPs) have become a hallmark of AI for scientific simulation. While efforts on new architectures and datasets have led to increasingly accurate and general models, the choice of optimizer for training has largely remained unexplored, defaulting to Adam and its variants in the community. Here, we implement and systematically compare a class of recently proposed matrix-structured optimizers, including Muon, SOAP, and the hybrid SOAP-Muon, for training NequIP and Allegro MLIP models. We find that these optimizers can substantially outperform Adam in both convergence speed and final accuracy. SOAP and SOAP-Muon emerge as robust and consistently strong methods, while Muon only provides partial gains relative to Adam. The improvements are particularly pronounced under partial force supervision. Our results indicate that optimizer choice is an overlooked yet impactful design axis for MLIPs.

Problem

• machine learning interatomic potentials
• optimizer
• training efficiency
• label efficiency
• convergence speed

Innovation

• matrix-structured optimizers
• SOAP
• Muon
• label-efficient training
• machine learning interatomic potentials