MIP Candy: A Modular PyTorch Framework for Medical Image Processing

2026-02-24

AI Summary

This work proposes a modular, open-source PyTorch-based framework for medical image processing that addresses the limitations of existing tools—either too low-level and complex to integrate or overly rigid and difficult to customize. The framework enables researchers to implement only a single method while automatically gaining access to a complete training and inference pipeline, with fine-grained control over individual components. Its core innovations include the LayerT lazy configuration mechanism, which allows dynamic replacement of convolution, normalization, and activation modules without subclassing, and an extensible model bundle ecosystem that unifies the trainer-predictor paradigm. Integrated features such as k-fold cross-validation, automatic ROI detection, deep supervision, EMA, multi-platform experiment tracking (W&B/Notion/MLflow), training resumption, and commercial regression validation significantly lower development barriers and enhance experimental efficiency and reproducibility. The code is open-sourced and compatible with Python 3.12+.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Medical image processing demands specialized software that handles high-dimensional volumetric data, heterogeneous file formats, and domain-specific training procedures. Existing frameworks either provide low-level components that require substantial integration effort or impose rigid, monolithic pipelines that resist modification. We present MIP Candy (MIPCandy), a freely available, PyTorch-based framework designed specifically for medical image processing. MIPCandy provides a complete, modular pipeline spanning data loading, training, inference, and evaluation, allowing researchers to obtain a fully functional process workflow by implementing a single method, $\texttt{build_network}$, while retaining fine-grained control over every component. Central to the design is $\texttt{LayerT}$, a deferred configuration mechanism that enables runtime substitution of convolution, normalization, and activation modules without subclassing. The framework further offers built-in $k$-fold cross-validation, dataset inspection with automatic region-of-interest detection, deep supervision, exponential moving average, multi-frontend experiment tracking (Weights&Biases, Notion, MLflow), training state recovery, and validation score prediction via quotient regression. An extensible bundle ecosystem provides pre-built model implementations that follow a consistent trainer--predictor pattern and integrate with the core framework without modification. MIPCandy is open-source under the Apache-2.0 license and requires Python~3.12 or later. Source code and documentation are available at https://github.com/ProjectNeura/MIPCandy.

Problem

• medical image processing
• modular framework
• PyTorch
• software framework
• volumetric data

Innovation

• modular framework
• LayerT
• deferred configuration
• medical image processing
• quotient regression