OTCHA: Optimal Transport-driven Confidence-aware Latent Hub Alignment for Multi-View Medical Image Classification

2026-06-18

AI Summary

This work addresses the challenge that direct fusion of multi-view medical images often introduces noise due to misalignment, view-specific artifacts, and irrelevant background regions, thereby degrading classification robustness. To mitigate this, the authors propose the OTCHA module, which leverages a learnable, cross-view shared implicit center token to formulate a conditional optimal transport scheme integrating both feature and geometric information. This enables confidence-aware partial matching and filtering of irrelevant tokens. Crucially, OTCHA implicitly aligns image patch tokens prior to fusion and utilizes transport confidence to guide message passing and alignment loss. Evaluated on three multi-view medical imaging datasets encompassing diverse anatomical structures and view configurations, OTCHA consistently outperforms existing methods.

Abstract

Multi-view imaging, such as mammography and chest radiography, is a standard component of clinical practice. However, medical images are often unregistered and contain view-specific artifacts or irrelevant background cues that can obscure diagnostically relevant findings. Many existing methods directly fuse per-view representations, allowing such irrelevant content to contaminate the fused embedding and reducing robustness under varying view configurations. We propose OTCHA, a confidence-aware latent hub token alignment module based on optimal transport (OT) that refines patch tokens before fusion for multi-view classification. OTCHA introduces a set of learnable latent hub tokens shared across views. For each view, we compute an OT plan between patch tokens and hub tokens that jointly considers feature similarity and geometry, and augment the OT formulation with token-conditional dustbins to enable partial matching and discard irrelevant tokens. The resulting transport plan provides token-wise matching confidence, which gates hub-mediated message passing and weights a novel optimal-transport-based representation alignment loss to stabilize refinement. Experiments on three multi-view medical image datasets demonstrate consistent improvements over competing baselines across diverse anatomies and view configurations. Our code is available at https://github.com/labhai/OTCHA.

Problem

• multi-view medical image classification
• unregistered images
• view-specific artifacts
• irrelevant background cues
• representation fusion

Innovation

• Optimal Transport
• Latent Hub Alignment
• Confidence-aware
• Multi-view Medical Image Classification
• Partial Matching