A Shape-Aware Total Body Photography System for In-focus Surface Coverage Optimization

2025-05-22

AI Summary

Existing full-body photography systems for automated skin lesion detection suffer from limited image resolution and focusing accuracy. This paper introduces an autonomous full-body imaging system tailored for skin cancer screening, integrating depth+RGB multimodal sensing, real-time 360° 3D human reconstruction, and a geometry-driven surface-focusing optimization algorithm. It achieves, for the first time, pose-wise optimal focal-length planning guided by 3D body shape perception. Through combined phantom and human subject calibration, the system covers 95% (empirical) and 85% (simulated) of total body surface area, with an average spatial resolution of 0.068 mm/pixel—substantially surpassing conventional autofocus approaches. The core innovation lies in the shape-aware surface-focusing coverage optimization method, which overcomes the longstanding technical bottleneck of achieving globally high-fidelity imaging across complex, non-planar human anatomy.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Total Body Photography (TBP) is becoming a useful screening tool for patients at high risk for skin cancer. While much progress has been made, existing TBP systems can be further improved for automatic detection and analysis of suspicious skin lesions, which is in part related to the resolution and sharpness of acquired images. This paper proposes a novel shape-aware TBP system automatically capturing full-body images while optimizing image quality in terms of resolution and sharpness over the body surface. The system uses depth and RGB cameras mounted on a 360-degree rotary beam, along with 3D body shape estimation and an in-focus surface optimization method to select the optimal focus distance for each camera pose. This allows for optimizing the focused coverage over the complex 3D geometry of the human body given the calibrated camera poses. We evaluate the effectiveness of the system in capturing high-fidelity body images. The proposed system achieves an average resolution of 0.068 mm/pixel and 0.0566 mm/pixel with approximately 85% and 95% of surface area in-focus, evaluated on simulation data of diverse body shapes and poses as well as a real scan of a mannequin respectively. Furthermore, the proposed shape-aware focus method outperforms existing focus protocols (e.g. auto-focus). We believe the high-fidelity imaging enabled by the proposed system will improve automated skin lesion analysis for skin cancer screening.

Problem

• Optimizes image resolution and sharpness for full-body photography
• Enhances automatic detection of suspicious skin lesions
• Improves focused coverage over complex 3D body geometry

Innovation

• Uses depth and RGB cameras on rotary beam
• Employs 3D body shape estimation technique
• Optimizes focus distance for each camera pose