Vision-Augmented On-Track System Identification for Autonomous Racing via Attention-Based Priors and Iterative Neural Correction

2026-03-10

AI Summary

This work addresses the challenge of accurately modeling highly nonlinear tire dynamics in real time for autonomous race cars under extreme operating conditions, where conventional online optimization methods often suffer from cold-start failures and inadequate representation of high-frequency transient dynamics. To overcome these limitations, the authors propose a vision-augmented iterative system identification framework that leverages a lightweight MobileNetV3 to generate friction priors from road texture for warm-starting the optimization, integrates an S4 sequence model to capture dynamic residuals, and employs derivative-free Nelder-Mead optimization to iteratively extract physically interpretable Pacejka parameters from CarSim-in-the-loop simulations. Experimental results demonstrate that the proposed approach reduces friction estimation error by 76.1%, decreases FLOPs by 85%, accelerates cold-start convergence by 71.4%, and lowers lateral force RMSE by over 60%, significantly outperforming existing neural architectures.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Operating autonomous vehicles at the absolute limits of handling requires precise, real-time identification of highly non-linear tire dynamics. However, traditional online optimization methods suffer from "cold-start" initialization failures and struggle to model high-frequency transient dynamics. To address these bottlenecks, this paper proposes a novel vision-augmented, iterative system identification framework. First, a lightweight CNN (MobileNetV3) translates visual road textures into a continuous heuristic friction prior, providing a robust "warm-start" for parameter optimization. Next, a S4 model captures complex temporal dynamic residuals, circumventing the memory and latency limitations of traditional MLPs and RNNs. Finally, a derivative-free Nelder-Mead algorithm iteratively extracts physically interpretable Pacejka tire parameters via a hybrid virtual simulation. Co-simulation in CarSim demonstrates that the lightweight vision backbone reduces friction estimation error by 76.1 using 85 fewer FLOPs, accelerating cold-start convergence by 71.4. Furthermore, the S4-augmented framework improves parameter extraction accuracy and decreases lateral force RMSE by over 60 by effectively capturing complex vehicle dynamics, demonstrating superior performance compared to conventional neural architectures.

Problem

• system identification
• autonomous racing
• tire dynamics
• cold-start
• real-time estimation

Innovation

• vision-augmented system identification
• attention-based friction prior
• S4 temporal modeling
• iterative neural correction
• Pacejka tire parameter estimation