A Benchmarking Framework for AI models in Automotive Aerodynamics

2025-07-14

AI Summary

The lack of systematic evaluation standards for AI-driven aerodynamic prediction models in automotive applications hinders fair comparison and industrial adoption. Method: This paper introduces the first open-source benchmarking framework for standardized, end-to-end evaluation of surface and volumetric flow field predictions. It features an open, extensible multi-dimensional assessment suite integrating physics-based metrics (e.g., mass/momentum conservation errors), computational efficiency measures, and cross-dataset generalization quantification, complemented by CAE-specific data–model integration guidelines. Leveraging the NVIDIA PhysicsNeMo-CFD platform and the DrivAerML dataset, we conduct a unified benchmark of three graph neural networks—DoMINO, X-MeshGraphNet, and FIGConvNet. Contribution/Results: Our evaluation reveals critical trade-offs among accuracy, robustness, and scalability across models, establishing a transparent, reproducible assessment paradigm and practical benchmark to guide model selection, optimization, and industrial deployment of AI-powered aerodynamic simulation.

Technology Category

• Machine Learning: Evaluation and Analysis
• Application Domains: Mobility, Driving & Flight
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In this paper, we introduce a benchmarking framework within the open-source NVIDIA PhysicsNeMo-CFD framework designed to systematically assess the accuracy, performance, scalability, and generalization capabilities of AI models for automotive aerodynamics predictions. The open extensible framework enables incorporation of a diverse set of metrics relevant to the Computer-Aided Engineering (CAE) community. By providing a standardized methodology for comparing AI models, the framework enhances transparency and consistency in performance assessment, with the overarching goal of improving the understanding and development of these models to accelerate research and innovation in the field. To demonstrate its utility, the framework includes evaluation of both surface and volumetric flow field predictions on three AI models: DoMINO, X-MeshGraphNet, and FIGConvNet using the DrivAerML dataset. It also includes guidelines for integrating additional models and datasets, making it extensible for physically consistent metrics. This benchmarking study aims to enable researchers and industry professionals in selecting, refining, and advancing AI-driven aerodynamic modeling approaches, ultimately fostering the development of more efficient, accurate, and interpretable solutions in automotive aerodynamics

Problem

• Assess AI models for automotive aerodynamics predictions
• Standardize methodology for comparing AI model performance
• Enable selection and advancement of AI-driven aerodynamic modeling

Innovation

• Benchmarking framework for AI aerodynamic models
• Integrates diverse CAE metrics for evaluation
• Supports extensible model and dataset integration