A Stitch in Time Saves Nine: Preserving Policy Compatibility Under Perception Updates in End-to-End Autonomous Driving

2026-06-19

AI Summary

This work addresses the performance degradation of frozen driving policies in end-to-end autonomous driving systems caused by perceptual model updates, which perturb latent representations. To mitigate this issue, the authors propose a lightweight latent-space alignment method that employs linear and convolutional “stitchers” to maintain policy compatibility across perceptual updates—without requiring policy retraining or architectural decoupling. The approach accommodates diverse perceptual evolutions, including variations in initialization, sensor configurations, and training domains. Evaluated in a cross-domain transfer from nuScenes to CARLA, the method recovers over 91% of the original driving performance with only 0.91 hours of adaptation time, compared to the original 22.18 hours, substantially reducing system maintenance costs.

Abstract

End-to-end autonomous driving systems tightly couple perception and decision-making through latent representations. Consequently, updates to perception models can alter these representations and degrade the performance of downstream policies that remain fixed. Existing solutions typically rely on policy retraining or architectural decoupling, both of which incur substantial computation and validation costs. In this paper, we formulate the model stitching problem for end-to-end autonomous driving and test the hypothesis that policy compatibility can be preserved through lightweight latent-space alignment. We study low-complexity model stitching methods, including linear and convolutional stitchers, for restoring compatibility between updated perception modules and frozen downstream policy modules. Experiments demonstrate that stitching effectively preserves downstream driving behavior under diverse perception updates, including changes in random initialization, sensor configuration, and training domain. In the most challenging cross-domain setting from nuScenes to CARLA, convolutional stitching retains over 91\% of the no-shift driving score while reducing adaptation time from \SI{22.18}{h} to \SI{0.91}{h}. These results suggest that model stitching provides an effective and computationally efficient alternative to retraining or fine-tuning for maintaining end-to-end autonomous driving systems. The model will be open-sourced upon paper acceptance at https://github.com/SCP-CN-001/model-stitching to support further research and development in autonomous driving.

Problem

• policy compatibility
• perception updates
• end-to-end autonomous driving
• latent representation
• model stitching

Innovation

• model stitching
• latent-space alignment
• end-to-end autonomous driving
• policy compatibility
• perception update