Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit

2025-07-22

AI Summary

Addressing the challenge of efficiently and accurately modeling multiscale features in high-resolution turbulence simulations—from large-scale eddies down to viscous dissipation scales—this work introduces a multiscale hierarchical Turbulence Transformer architecture coupled with RingX sequence parallelism. Our method enables pixel-level long-context learning, employs hierarchical attention mechanisms, and compresses sequence length to achieve, for the first time, full-resolution reconstruction of dissipative-scale eddies by an AI model—overcoming computational bottlenecks inherent in conventional long-context modeling. We trained the model at trillion-grid scale on the Frontier supercomputer using 32,768 AMD GPUs, attaining a peak performance of 1.1 EFLOPS and 94% strong scaling efficiency. This establishes a scalable, high-fidelity AI modeling paradigm for extreme-scale, multiphysics-coupled predictions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Diffusion Models for Vision
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range.

Problem

• Resolving small-scale turbulence eddies at exascale
• Reducing computational costs for extreme-resolution turbulence data
• Enabling scalable long-context learning for turbulence modeling

Innovation

• Multiscale hierarchical Turbulence Transformer reduces sequence length
• RingX sequence parallelism enables scalable long-context learning
• Achieves 1.1 EFLOPS on Frontier supercomputer with 94% efficiency