🤖 AI Summary
This work addresses a critical limitation in existing trustworthy multimodal fusion methods, which rely on model prediction confidence to assess input quality and consequently fail when models are confidently incorrect. To overcome this circular dependency between confidence and correctness, the authors propose a geometry-inspired reliability criterion. Specifically, they model the transport path from an input to a reliable region in latent space using a Rectified Flow–based diffusion Schrödinger bridge, and define a calibration score as the squared norm of the initial transport velocity. This score provides a model-confidence–agnostic measure of input reliability. Empirical results demonstrate that the proposed approach significantly outperforms current baselines under challenging conditions involving strong sensor noise and semantic conflicts, thereby substantially enhancing the robustness of multimodal fusion.
📝 Abstract
Real-world multimodal systems must be robust against low-quality data, such as sensor noise, incomplete multimodal data and conflicting inputs. However, existing trustworthy fusion methods rely on the model's own prediction confidence to judge data quality. This creates a circular dependency: when a model is confident but wrong, these methods fail to detect the error. To break this loop, we propose Geometry-based Multimodal Fusion (GMF). Instead of relying on predictions, we evaluate reliability by measuring how much transport correction the input needs in latent space. We implement Diffusion Schrödinger Bridge transport with Rectified Flow, where the squared initial velocity gives an efficient learned correction score. Valid data has low squared velocity magnitude, while noisy, incomplete data or conflicting data requires stronger transport correction. This geometry-based reliability signal acts as an independent judge, effectively flagging unreliable inputs even when the classifier is fooled. Extensive experiments demonstrate that GMF significantly improves robustness against severe sensor noise and semantic conflicts compared to confidence-based baselines.