Local Conformal Calibration of Dynamics Uncertainty from Semantic Images

2026-05-13

AI Summary

This work addresses the limitations of existing conformal prediction methods, which struggle to distinguish regions in the state-action space with varying degrees of model mismatch and rely heavily on environment-specific data, hindering generalization to unseen test scenarios. The paper proposes OCULAR, the first algorithm to integrate semantic visual perception with conformal prediction for locally calibrating linear Gaussian dynamics models at arbitrary precision. Without requiring strong distributional assumptions, OCULAR provides non-asymptotic, distribution-free probabilistic coverage guarantees for future states. By leveraging visually similar environmental data, it adaptively quantifies uncertainty levels across different inputs, enabling probabilistic safety-aware planning. Experiments on a double integrator system demonstrate that OCULAR accurately captures uncertainty both in-distribution and out-of-distribution, achieving significantly higher volumetric efficiency in prediction regions compared to baseline methods that depend on environment-specific data.

Abstract

We introduce Observation-aware Conformal Uncertainty Local-Calibration (OCULAR), a conformal prediction-based algorithm that uses perception information to provide uncertainty quantification guarantees for unseen test-time environments. While previous conformal approaches lack the ability to discriminate between state-action space regions leading to higher or lower model mismatch, and require environment-specific data, our method uses data collected from visually similar environments to provably calibrate a given linear Gaussian dynamics model of arbitrary fidelity. The prediction regions generated from OCULAR are guaranteed to contain the future system states with, at least, a user-set likelihood, despite both aleatoric and epistemic uncertainty -- i.e., uncertainty arising from both stochastic disturbances and lack of data. Our guarantees are non-asymptotic and distribution-free, not requiring strong assumptions about the unknown real system dynamics. Our calibration procedure enables distinguishing between observation-velocity-action inputs leading to higher and lower next-state-uncertainty, which is helpful for probabilistically-safe planning. We numerically validate our algorithm on a double-integrator system subject to random perturbations and significant model mismatch, using both a simplified sensor and a more realistic simulated camera. Our approach appropriately quantifies uncertainty both when in-distribution and out-of-distribution, being comparatively volume-efficient to baselines requiring environment-specific data.

Problem

• conformal prediction
• dynamics uncertainty
• semantic images
• uncertainty quantification
• out-of-distribution

Innovation

• Conformal Prediction
• Uncertainty Quantification
• Perception-aware Calibration
• Distribution-free Guarantee
• Dynamics Mismatch