FuseUNet: A Multi-Scale Feature Fusion Method for U-like Networks

2025-06-06

AI Summary

In medical image segmentation, U-Net’s skip connections suffer from insufficient cross-scale feature interaction and simplistic fusion strategies (e.g., concatenation or addition). To address these limitations, this work pioneers modeling skip connections as discrete nodes of an ordinary differential equation (ODE), and introduces an adaptive ODE-based fusion mechanism grounded in linear multistep methods. This enables continuous, learnable, and differentiable multi-scale feature integration along the decoding path. The proposed method is architecture-agnostic—decoupled from encoder-decoder design—and thus universally pluggable. Evaluated on five benchmark datasets (ACDC, KiTS2023, MSD Brain Tumor, ISIC), it consistently improves Dice scores by 1.2–2.8%, reduces parameter count by 12–18%, and significantly enhances feature utilization. These results empirically validate the effectiveness and generalizability of ODE modeling for optimizing skip connections in U-shaped architectures.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Feature Construction/Reformulation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Medical image segmentation is a critical task in computer vision, with UNet serving as a milestone architecture. The typical component of UNet family is the skip connection, however, their skip connections face two significant limitations: (1) they lack effective interaction between features at different scales, and (2) they rely on simple concatenation or addition operations, which constrain efficient information integration. While recent improvements to UNet have focused on enhancing encoder and decoder capabilities, these limitations remain overlooked. To overcome these challenges, we propose a novel multi-scale feature fusion method that reimagines the UNet decoding process as solving an initial value problem (IVP), treating skip connections as discrete nodes. By leveraging principles from the linear multistep method, we propose an adaptive ordinary differential equation method to enable effective multi-scale feature fusion. Our approach is independent of the encoder and decoder architectures, making it adaptable to various U-Net-like networks. Experiments on ACDC, KiTS2023, MSD brain tumor, and ISIC2017/2018 skin lesion segmentation datasets demonstrate improved feature utilization, reduced network parameters, and maintained high performance. The code is available at https://github.com/nayutayuki/FuseUNet.

Problem

• Improves multi-scale feature fusion in U-Net skip connections
• Replaces simple concatenation with adaptive ODE-based fusion
• Enhances medical image segmentation across diverse datasets

Innovation

• Reimagines UNet decoding as initial value problem
• Uses adaptive ODE for multi-scale feature fusion
• Independent of encoder and decoder architectures