🤖 AI Summary
Real-world data often fail to meet the demand for high-quality, diverse pedestrian datasets required for autonomous driving, particularly in safety-critical scenarios. This work proposes a StyleGAN2-based controllable generation method that automatically maps facial textures and identity-level appearance variations onto a unified 3D pedestrian mesh, enabling efficient synthesis of large-scale, simulation-ready assets without requiring new geometric modeling. To the best of our knowledge, this is the first application of controllable generative AI to diversify 3D pedestrian appearances. Experiments demonstrate that the synthesized data significantly enhance the robustness of 2D detection models, while also revealing the high sensitivity of 3D perception models to geometric domain discrepancies—providing critical insights for simulation-based training.
📝 Abstract
In recent years, autonomous driving has significantly in creased the demand for high-quality data to train 2D and 3D perception models for safety-critical scenarios. Real world datasets struggle to meet this demand as require ments continuously evolve and large-scale annotated data collection remains costly and time-consuming making syn thetic data a scalable, practical and controllable alterna tive. Pedestrian detection is among the most safety-critical tasks in autonomous driving. In this paper, we propose a simple yet effective method for scaling variability in 3D pedestrian assets for synthetic scene generation. Starting from a single 3D base asset, we generate multiple distinct pedestrian instances by synthesizing diverse facial textures and identity-level appearance variations using StyleGAN2 and automatically mapping them onto 3D meshes. This ap proach enables scalable appearance-level asset diversifica tion without requiring the design of new geometries for each instance. Using the assets, we construct synthetic datasets and study the impact of mixing real and synthetic data for RGB-based object detection. Through complementary ex periments, we analyze geometry-driven distribution shifts in point cloud perception for 3D object detection. Our findings demonstrate that controlled synthetic diversifica tion improves robustness in 2D detection while revealing the sensitivity of 3D perception models to geometric domain gaps. Overall, this work highlights how generative AI en ables scalable, simulation-ready pedestrian diversification through controlled facial texture synthesis, along with the benefits and limitations of cross-domain training strategies in autonomous driving pipelines.