Displacement Preserving Relational Distillation for Robust Medical Segmentation

2026-07-05

AI Summary

This work addresses the challenges of large anatomical variability and high computational cost in 3D medical image segmentation, where conventional knowledge distillation struggles to preserve complex structures and is susceptible to noise. The authors propose Displacement-Preserving Relational Distillation (DPRD), which, for the first time, incorporates displacement relationships along anatomical trajectories into the distillation process. By leveraging a vector-based relational alignment mechanism within the nnU-Net framework and integrating task-aware anchors to focus on critical structures, DPRD mitigates signal dilution and effectively retains the directional and relative scale properties of the teacher model’s manifold. Evaluated on AMOS 2022, DPRD achieves a Dice score of 85.46% with only approximately 5% of the parameters and 3% of the FLOPs of the larger MedNeXt teacher model, significantly improving boundary accuracy, structural consistency, and deployment efficiency.

Abstract

Accurate 3D medical segmentation is limited by anatomical variability and high computational costs. While knowledge distillation (KD) offers a route for model compression, conventional methods often fail to preserve complex structures and are overwhelmed by background noise. We propose Displacement-Preserving Relational Distillation (DPRD), which distills latent anatomical trajectories via vector based alignment to preserve the orientation and relative scale of the teacher's manifold, and prevents signal dilution by anchoring distillation in task-relevant structures. Integrated into nnU-Net, DPRD outperforms established baselines on ISLES 2022 and AMOS 2022 benchmarks. Notably, on the AMOS dataset, DPRD achieves a Dice score of 85.46%, edging out the high-capacity MedNeXt teacher while significantly reducing boundary errors. Despite utilizing only ~5% of the teacher's parameters and ~3% of its FLOPs, our approach maintains high structural consistency. This provides a robust, efficient solution for deploying high performance segmenters in resource-constrained clinical environments. Code: https://github.com/ClinicaAlpha/DPRD-3D-MedSeg

Problem

• medical segmentation
• anatomical variability
• knowledge distillation
• model compression
• background noise

Innovation

• Displacement-Preserving Relational Distillation
• Knowledge Distillation
• Medical Image Segmentation
• Manifold Alignment
• Model Compression