Data Augmentation for High-Fidelity Generation of CAR-T/NK Immunological Synapse Images

2026-02-01

AI Summary

This study addresses the scarcity of annotated immunological synapse (IS) images in CAR-T/NK cell therapy, which limits the generalization of deep learning models for IS detection and segmentation. To overcome this challenge, we propose a novel data augmentation framework that integrates Instance-Aware Automatic Augmentation (IAAA) and Semantic-Aware AI Augmentation (SAAA). IAAA preserves individual structural features, while SAAA leverages diffusion models to generate semantically consistent segmentation masks and employs Pix2Pix for conditional image synthesis. This approach uniquely combines instance-preserving and semantics-controllable generation strategies, supporting multimodal microscopy and patient-derived samples to significantly enhance the realism and diversity of synthetic images. Experimental results demonstrate that the augmented data substantially improve the accuracy and robustness of IS detection and segmentation, offering critical support for developing reliable imaging biomarkers for therapeutic response prediction.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance

Abstract

Chimeric antigen receptor (CAR)-T and NK cell immunotherapies have transformed cancer treatment, and recent studies suggest that the quality of the CAR-T/NK cell immunological synapse (IS) may serve as a functional biomarker for predicting therapeutic efficacy. Accurate detection and segmentation of CAR-T/NK IS structures using artificial neural networks (ANNs) can greatly increase the speed and reliability of IS quantification. However, a persistent challenge is the limited size of annotated microscopy datasets, which restricts the ability of ANNs to generalize. To address this challenge, we integrate two complementary data-augmentation frameworks. First, we employ Instance Aware Automatic Augmentation (IAAA), an automated, instance-preserving augmentation method that generates synthetic CAR-T/NK IS images and corresponding segmentation masks by applying optimized augmentation policies to original IS data. IAAA supports multiple imaging modalities (e.g., fluorescence and brightfield) and can be applied directly to CAR-T/NK IS images derived from patient samples. In parallel, we introduce a Semantic-Aware AI Augmentation (SAAA) pipeline that combines a diffusion-based mask generator with a Pix2Pix conditional image synthesizer. This second method enables the creation of diverse, anatomically realistic segmentation masks and produces high-fidelity CAR-T/NK IS images aligned with those masks, further expanding the training corpus beyond what IAAA alone can provide. Together, these augmentation strategies generate synthetic images whose visual and structural properties closely match real IS data, significantly improving CAR-T/NK IS detection and segmentation performance. By enhancing the robustness and accuracy of IS quantification, this work supports the development of more reliable imaging-based biomarkers for predicting patient response to CAR-T/NK immunotherapy.

Problem

• CAR-T/NK immunological synapse
• data augmentation
• limited annotated dataset
• image segmentation
• artificial neural networks

Innovation

• Data Augmentation
• Immunological Synapse
• CAR-T/NK Cell Therapy
• Diffusion Model
• Instance-Aware Synthesis