Anisotropic Fourier Features for Positional Encoding in Medical Imaging

2025-09-02

AI Summary

Medical imaging exhibits complex structural patterns and pronounced spatial anisotropy, rendering conventional sinusoidal positional encodings (SPEs) inadequate for preserving high-dimensional Euclidean distances, while isotropic Fourier feature positional encodings (IFPEs) fail to capture direction-dependent spatial relationships. To address these limitations, we propose Anisotropic Fourier Feature Positional Encoding (AFPE), the first method to jointly embed anisotropic kernels, category-specific priors, and domain-specific spatial constraints into positional encoding. AFPE is designed for multimodal medical imaging—including X-ray, CT, and echocardiography—and demonstrates consistent superiority over SPEs and IFPEs across diverse tasks: multi-label chest X-ray classification, CT organ segmentation, and ejection fraction regression. Notably, performance gains are most substantial on highly anisotropic data. By explicitly modeling directional spatial structure, AFPE enhances the robustness and fidelity of spatial representation learning in medical Transformers, establishing a new foundation for geometry-aware vision-language modeling in clinical imaging.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The adoption of Transformer-based architectures in the medical domain is growing rapidly. In medical imaging, the analysis of complex shapes - such as organs, tissues, or other anatomical structures - combined with the often anisotropic nature of high-dimensional images complicates these adaptations. In this study, we critically examine the role of Positional Encodings (PEs), arguing that commonly used approaches may be suboptimal for the specific challenges of medical imaging. Sinusoidal Positional Encodings (SPEs) have proven effective in vision tasks, but they struggle to preserve Euclidean distances in higher-dimensional spaces. Isotropic Fourier Feature Positional Encodings (IFPEs) have been proposed to better preserve Euclidean distances, but they lack the ability to account for anisotropy in images. To address these limitations, we propose Anisotropic Fourier Feature Positional Encoding (AFPE), a generalization of IFPE that incorporates anisotropic, class-specific, and domain-specific spatial dependencies. We systematically benchmark AFPE against commonly used PEs on multi-label classification in chest X-rays, organ classification in CT images, and ejection fraction regression in echocardiography. Our results demonstrate that choosing the correct PE can significantly improve model performance. We show that the optimal PE depends on the shape of the structure of interest and the anisotropy of the data. Finally, our proposed AFPE significantly outperforms state-of-the-art PEs in all tested anisotropic settings. We conclude that, in anisotropic medical images and videos, it is of paramount importance to choose an anisotropic PE that fits the data and the shape of interest.

Problem

• Addressing suboptimal positional encodings for medical imaging challenges
• Preserving Euclidean distances in high-dimensional anisotropic medical images
• Incorporating anisotropic spatial dependencies in positional encoding methods

Innovation

• Anisotropic Fourier Feature Positional Encoding (AFPE)
• Incorporates anisotropic class-specific spatial dependencies
• Outperforms state-of-the-art PEs in anisotropic settings