E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation

2023-12-07

Neural Information Processing Systems

AI Summary

For 3D medical image segmentation under resource-constrained settings, this paper proposes E2ENet—a lightweight and efficient network. To reconcile the trade-off between model capacity and computational overhead, we introduce two core innovations: (1) a dynamic sparse feature fusion mechanism that adaptively selects and fuses multi-scale features to minimize redundant information propagation; and (2) constrained-depth displaced 3D convolutions, which preserve volumetric spatial modeling capability while reducing computational complexity to near-2D levels. Integrated with parameter sharing, dynamic gating, and computation compression strategies, E2ENet achieves state-of-the-art accuracy on benchmarks including AMOS-CT, with 68% fewer parameters and 29% lower inference FLOPs compared to prior lightweight methods—demonstrating significant improvements in both efficiency and performance.

Abstract

Deep neural networks have evolved as the leading approach in 3D medical image segmentation due to their outstanding performance. However, the ever-increasing model size and computation cost of deep neural networks have become the primary barrier to deploying them on real-world resource-limited hardware. In pursuit of improving performance and efficiency, we propose a 3D medical image segmentation model, named Efficient to Efficient Network (E2ENet), incorporating two parametrically and computationally efficient designs. i. Dynamic sparse feature fusion (DSFF) mechanism: it adaptively learns to fuse informative multi-scale features while reducing redundancy. ii. Restricted depth-shift in 3D convolution: it leverages the 3D spatial information while keeping the model and computational complexity as 2D-based methods. We conduct extensive experiments on BTCV, AMOS-CT and Brain Tumor Segmentation Challenge, demonstrating that E2ENet consistently achieves a superior trade-off between accuracy and efficiency than prior arts across various resource constraints. E2ENet achieves comparable accuracy on the large-scale challenge AMOS-CT, while saving over 68% parameter count and 29% FLOPs in the inference phase, compared with the previous best-performing method. Our code has been made available at: https://github.com/boqian333/E2ENet-Medical.

Problem

• Improves 3D medical image segmentation
• Reduces model size and computation cost
• Enhances efficiency on resource-limited hardware

Innovation

• Dynamic sparse feature fusion
• Restricted depth-shift 3D convolution
• Efficient and accurate medical segmentation