RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applications

2026-02-09

AI Summary

The scarcity of large-scale, realistically annotated data has hindered the advancement of deep learning–based 3D reconstruction in colonoscopy. To address this limitation, this work presents a high-fidelity synthetic colon dataset generated by extracting colonic geometries from 10 clinical CT scans and simulating intraoperative imaging conditions within a virtual endoscopy environment. Realistic vascular textures are rendered to produce 28,130 photorealistic image frames, each paired with corresponding depth maps, optical flow, 3D meshes, and camera trajectories. This dataset is the first to combine clinical anatomical diversity with visual realism in synthetic colonoscopy imagery, substantially improving model generalization to real clinical data. Experimental results demonstrate that models trained on this dataset significantly outperform those trained on existing synthetic datasets in both depth estimation and camera pose estimation tasks.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data generation and simulation
• Security and Privacy: Data transparency and provenance

Abstract

Deep learning has the potential to improve colonoscopy by enabling 3D reconstruction of the colon, providing a comprehensive view of mucosal surfaces and lesions, and facilitating the identification of unexplored areas. However, the development of robust methods is limited by the scarcity of large-scale ground truth data. We propose RealSynCol, a highly realistic synthetic dataset designed to replicate the endoscopic environment. Colon geometries extracted from 10 CT scans were imported into a virtual environment that closely mimics intraoperative conditions and rendered with realistic vascular textures. The resulting dataset comprises 28\,130 frames, paired with ground truth depth maps, optical flow, 3D meshes, and camera trajectories. A benchmark study was conducted to evaluate the available synthetic colon datasets for the tasks of depth and pose estimation. Results demonstrate that the high realism and variability of RealSynCol significantly enhance generalization performance on clinical images, proving it to be a powerful tool for developing deep learning algorithms to support endoscopic diagnosis.

Problem

• 3D reconstruction
• colonoscopy
• synthetic dataset
• deep learning
• ground truth data

Innovation

• synthetic dataset
• 3D reconstruction
• colonoscopy
• deep learning
• medical simulation