HNOSeg-XS: Extremely Small Hartley Neural Operator for Efficient and Resolution-Robust 3D Image Segmentation

2025-07-10

AI Summary

Medical 3D image segmentation faces dual bottlenecks: limited receptive fields in CNNs and high computational complexity (O(N²)) in Transformers; resolution reduction during training further degrades high-resolution performance. To address this, we propose HNOSeg-XS—the first frequency-domain segmentation architecture built upon learnable partial differential equations and the Hartley neural operator. By replacing Fourier with Hartley transforms, it enables global modeling with ultra-low parameter count (<34.7k). The framework integrates frequency-domain reconstruction and low-dimensional embedding, enabling zero-shot super-resolution and cross-resolution robust inference. Evaluated on BraTS’23, KiTS’23, and MVSeg’23, HNOSeg-XS achieves inference times <0.24 seconds and GPU memory usage <1.8 GiB—significantly outperforming state-of-the-art CNN- and Transformer-based methods while simultaneously ensuring accuracy, efficiency, and resolution robustness.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

In medical image segmentation, convolutional neural networks (CNNs) and transformers are dominant. For CNNs, given the local receptive fields of convolutional layers, long-range spatial correlations are captured through consecutive convolutions and pooling. However, as the computational cost and memory footprint can be prohibitively large, 3D models can only afford fewer layers than 2D models with reduced receptive fields and abstract levels. For transformers, although long-range correlations can be captured by multi-head attention, its quadratic complexity with respect to input size is computationally demanding. Therefore, either model may require input size reduction to allow more filters and layers for better segmentation. Nevertheless, given their discrete nature, models trained with patch-wise training or image downsampling may produce suboptimal results when applied on higher resolutions. To address this issue, here we propose the resolution-robust HNOSeg-XS architecture. We model image segmentation by learnable partial differential equations through the Fourier neural operator which has the zero-shot super-resolution property. By replacing the Fourier transform by the Hartley transform and reformulating the problem in the frequency domain, we created the HNOSeg-XS model, which is resolution robust, fast, memory efficient, and extremely parameter efficient. When tested on the BraTS'23, KiTS'23, and MVSeg'23 datasets with a Tesla V100 GPU, HNOSeg-XS showed its superior resolution robustness with fewer than 34.7k model parameters. It also achieved the overall best inference time (< 0.24 s) and memory efficiency (< 1.8 GiB) compared to the tested CNN and transformer models.

Problem

• Efficient 3D medical image segmentation with limited computational resources
• Resolution robustness in segmentation without input size reduction
• Reducing model parameters while maintaining performance in segmentation

Innovation

• Uses Hartley transform for resolution robustness
• Models segmentation via learnable PDEs
• Extremely parameter-efficient with <34.7k parameters