Interactive Tumor Progression Modeling via Sketch-Based Image Editing

2025-03-10

AI Summary

To address the subjectivity and low accuracy in visualizing tumor progression in medical imaging, this paper proposes a sketch-guided conditional diffusion model for interactive and interpretable modeling and editing of tumor morphological evolution. Methodologically, it innovatively incorporates hand-drawn sketches as structural priors into the diffusion process, complemented by an anatomy-consistent structural preservation loss and a cross-dataset adaptive fine-tuning strategy—jointly ensuring morphological plausibility and visual fidelity. Comprehensive evaluation is conducted on four major public benchmarks: BraTS, LiTS, KiTS, and MSD-Pancreas. Quantitative results demonstrate a 12.6% improvement in image fidelity over state-of-the-art methods and a 9.3% gain in tumor segmentation Dice score. The framework significantly enhances clinical controllability and interpretability, enabling intuitive, user-driven tumor evolution simulation while preserving anatomical coherence.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Structured Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Accurately visualizing and editing tumor progression in medical imaging is crucial for diagnosis, treatment planning, and clinical communication. To address the challenges of subjectivity and limited precision in existing methods, we propose SkEditTumor, a sketch-based diffusion model for controllable tumor progression editing. By leveraging sketches as structural priors, our method enables precise modifications of tumor regions while maintaining structural integrity and visual realism. We evaluate SkEditTumor on four public datasets - BraTS, LiTS, KiTS, and MSD-Pancreas - covering diverse organs and imaging modalities. Experimental results demonstrate that our method outperforms state-of-the-art baselines, achieving superior image fidelity and segmentation accuracy. Our contributions include a novel integration of sketches with diffusion models for medical image editing, fine-grained control over tumor progression visualization, and extensive validation across multiple datasets, setting a new benchmark in the field.

Problem

• Accurate visualization and editing of tumor progression in medical imaging.
• Overcoming subjectivity and limited precision in existing tumor progression methods.
• Enabling precise, realistic tumor region modifications using sketch-based diffusion models.

Innovation

• Sketch-based diffusion model for tumor editing
• Leverages sketches as structural priors
• Validated across multiple diverse medical datasets