BTI-Net: Bidirectional Decoder-Level Task Interaction via Uncertainty-Aware Gating for Multi-Task Medical Image Analysis

2026-06-27

AI Summary

This work addresses the limitation in multi-task medical image analysis where shared encoders often cause decoders to overlook complementary boundary and semantic information across tasks. To overcome this, the authors propose a Task Interaction Module (TIM) coupled with Uncertainty Proxy Attention (UPA), enabling input- and level-adaptive bidirectional task interaction across four resolution levels of the decoder. This mechanism dynamically fuses spatial boundary context with global semantic priors without requiring additional annotations or increasing inference cost. Evaluated on ultrasound, dermoscopic, and brain MRI datasets, the method consistently outperforms current baselines, achieving an improvement of +2.36 IoU and up to +2.26% in classification accuracy through adaptive gating.

Abstract

Jointly learning to segment and classify medical images demands cross-task synergy, yet encoder-sharing architectures limit decoder reconstruction to task-private representations, permanently discarding the boundary cues and semantic priors each branch could supply to the other. This work introduces BTI-Net, which establishes bidirectional communication at every decoder level through two parallel pathways via Task Interaction Modules (TIM). Spatial boundary context is gated into the classification branch, while global semantic priors multiplicatively modulate the decoder, with refined features propagating progressively from coarse semantics to fine boundary detail across all four decoder resolutions. Since cross-task interaction is not equally reliable for every input, Uncertainty Proxy Attention (UPA) gates each TIM output per instance and per level using three signals that capture cross-task alignment, scene complexity, and prediction confidence, without external annotations or additional inference passes. Experiments on three medical benchmarks spanning ultrasound, dermoscopy, and brain MRI demonstrate consistent improvements in segmentation IoU and classification accuracy over both encoder-sharing and decoder-interaction baselines. Ablation confirms adaptive gating contributes +2.36 IoU over fixed bidirectional interaction, and classification accuracy improves by up to +2.26 points over the strongest multi-task baseline. UPA's uncertainty proxies serve as reliable single-pass task-failure signals without the overhead of stochastic sampling. Code: https://github.com/C-loud-Nine/BTI-Net_MTL

Problem

• multi-task learning
• medical image analysis
• segmentation
• classification
• task interaction

Innovation

• Bidirectional Task Interaction
• Uncertainty-Aware Gating
• Decoder-Level Communication
• Multi-Task Medical Image Analysis
• Task Interaction Module