SceneFactory: GPU-Accelerated Multi-Agent Driving Simulation with Physics-Based Vehicle Dynamics

2026-05-08

AI Summary

This work addresses the longstanding challenge in autonomous driving simulation of simultaneously achieving high physical fidelity and scalable parallel training efficiency. Building upon NVIDIA Isaac Sim and Isaac Lab, the authors introduce the first multi-agent driving simulation platform that leverages GPU vectorization by unifying scenes, vehicles, and interactions into tensor operations. The framework preserves high-fidelity vehicle dynamics—including tire–road contact, suspension systems, and surface-dependent friction—while enabling highly efficient concurrent simulation. Evaluated on a single GPU, the system achieves 19,250 agent-steps per second, yielding a 127× throughput improvement over a non-vectorized PhysX baseline. Policies trained with this approach demonstrate a 52.7% reduction in peak DRAC on slippery surfaces and achieve a 99.5% success rate when transferred across simulators.

Abstract

Autonomous-driving simulators typically trade physical fidelity for scalable parallelism. Physics-based platforms such as CARLA and MetaDrive provide articulated vehicle dynamics and contact, but their non-vectorized interfaces make batched training difficult. GPU-batched systems such as Waymax and GPUDrive scale to hundreds of scenarios by replacing rigid-body physics with simplified kinematic models, omitting tire--road interaction, suspension, contact dynamics, and road-condition-dependent friction. We introduce SceneFactory, a GPU-vectorized platform for procedural scene construction, physics-based multi-agent simulation, and RL in autonomous-driving environments. Built on NVIDIA Isaac Sim + Isaac Lab, SceneFactory represents worlds and agents as batched tensors: control, observations, rewards, resets, and policy inference run as GPU tensor operations over the Isaac Lab tensor API. SceneFactory converts Waymo Open Motion Dataset road topologies into simulation-ready USD worlds, runs many worlds concurrently on one GPU, populates each with multiple articulated PhysX vehicles, and maps precipitation and road-surface type to PhysX material friction coefficients. With GPU vectorization, SceneFactory achieves up to 127$\times$ higher throughput than a non-vectorized PhysX baseline on the same GPU and physics solver, reaching 19,250 controlled-agent simulation steps per second at 256 worlds $\times$ 16 agents. Cross-simulator transfer reveals an asymmetric dynamics gap: physics-grounded RL policies transfer to a simplified kinematic bicycle model with 99.5% success, whereas reverse transfer drops to 47.3%. Under wet-road friction, friction-aware policies reduce mean peak DRAC from 58.7 to 27.8,m/s$^2$ without sacrificing goal reach. SceneFactory shows that scalable autonomous-driving training need not discard articulated rigid-body dynamics or physically grounded road-condition variation.

Problem

• autonomous driving simulation
• physics-based vehicle dynamics
• GPU vectorization
• multi-agent simulation
• road-condition-dependent friction

Innovation

• GPU vectorization
• physics-based simulation
• multi-agent driving
• autonomous driving RL
• articulated vehicle dynamics