LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Driving

2026-07-12

AI Summary

This work addresses the mismatch between multi-source observational redundancy and dynamic decision modeling in autonomous driving by proposing a decoder-free latent interaction Dreamer architecture. Instead of reconstructing observations, the method employs latent alignment and introduces a latent-tanh action residual chain to model continuous control. It further integrates residual action sequence contrastive learning with multi-step rollout alignment to achieve risk-aware state abstraction and long-horizon dynamics prediction. Evaluated across diverse simulated driving scenarios, the approach substantially outperforms existing world models in terms of both cumulative reward and task success rate, while also demonstrating strong transferability to real-world traffic environments as validated on the nuPlan benchmark.

Abstract

Autonomous driving requires long-horizon closedloop decision making in dynamic traffic environments. Latent world models offer an effective framework for this problem by enabling imagination-based decision making in compact latent spaces. However, multi-source observations contain controlirrelevant redundancy, whereas reliable driving decisions rely on risk-relevant relations, future dynamics, and continuous action adjustments. This mismatch makes observation reconstruction and absolute action modeling suboptimal for learning decisionrelevant latent dynamics. We propose LIDAR-AD, a decoderfree Latent-Interaction Dreamer with Action-Residual Chains for autonomous driving. LIDAR-AD replaces observation reconstruction with redundancy-reduced latent alignment, encouraging compact representations of risk-relevant relations in multi-source driving inputs. It further models vehicle control as residual action updates and uses residual-action sequence contrastive learning to align multi-step residual-driven rollouts with future latent states. A deterministic analysis shows that the latent-tanh residual parameterization preserves interior action reachability while representing smooth long-horizon control as compact local updates. Together, these designs improve risk-aware state abstraction, continuous-control modeling, and long-horizon dynamics prediction. Extensive experiments across diverse simulated driving scenarios demonstrate that LIDAR-AD consistently outperforms world-model baselines, achieving the highest reward and the best success rate among learning-based methods. Evaluations on nuPlan-derived log-reconstructed scenarios further demonstrate the transferability of LIDAR-AD under real-world traffic layouts.

Problem

• autonomous driving
• latent world models
• multi-source observations
• risk-relevant relations
• long-horizon decision making

Innovation

• latent world models
• residual action chains
• decoder-free
• contrastive learning
• autonomous driving