GET: Generative Embedding Translation for Medical Image Segmentation

2026-08-23

AI Summary

为解决医学图像分割问题，提出GET方法，通过在Stable Diffusion VAE的冻结潜在空间内逐步转换图像嵌入到掩码嵌入，实现了高效准确的分割。

Abstract

Generative segmentation provides an alternative to direct pixel-wise prediction by operating on learned latent representations, but effective image-to-mask translation must preserve target structure while remaining computationally efficient. We propose Generative Embedding Translation (GET), a structured embedding-translation framework that progressively transforms image embeddings into mask embeddings within the frozen latent space of a Stable Diffusion VAE. GET uses a U-Net-style Embedding Translation Network with 1.07M trainable parameters, combining Mobile Bottleneck Convolutions, Subsampled Self-Attention, and Multi-scale Feature Enrichment for local modeling, global context, and multi-scale refinement. Across five medical segmentation datasets, GET outperforms generative, CNN, and Transformer baselines. Compared with the strongest generative baseline, GMS, GET improves average Dice and IoU by 0.93% and 1.26%, reduces HD95 by 0.81 pixels, and uses 31.41% fewer trainable parameters. Under bidirectional BUS-BUSI domain shift, GET further improves Dice and IoU by 3.51% and 3.39%, while reducing HD95 by 27.37 pixels. Our code is available at: https://github.com/maklachur/GET.

Problem

• Generative Segmentation
• Medical Image Segmentation
• Latent Representations
• Efficiency
• Target Structure Preservation

Innovation

• Generative Embedding Translation
• Medical Image Segmentation
• Stable Diffusion VAE
• Mobile Bottleneck Convolutions
• Subsampled Self-Attention