OBJVanish: Physically Realizable Text-to-3D Adv. Generation of LiDAR-Invisible Objects

2025-10-08

AI Summary

Existing LiDAR-based 3D object detectors exhibit limited robustness against physically realizable adversarial attacks, particularly in achieving complete target disappearance. This paper proposes the first physically implementable text-to-3D adversarial generation framework. By optimizing multimodal text prompts—integrating verbs, objects, and poses—it synthesizes pedestrian 3D models that are topologically consistent with real-world objects yet rendered entirely invisible to LiDAR sensors. We systematically analyze how 3D model topology, connectivity, and structural rigidity influence detector vulnerability. Leveraging multi-agent interactions in CARLA simulation, we perform iterative prompt refinement to enhance attack transferability. The method successfully evades six state-of-the-art LiDAR detectors in both simulated and real-world settings, achieving complete pedestrian occlusion. This exposes critical safety vulnerabilities in current 3D perception systems and demonstrates the practical feasibility of deploying such adversarial generation for robustness evaluation.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents
• Machine Learning: Adversarial Learning & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

LiDAR-based 3D object detectors are fundamental to autonomous driving, where failing to detect objects poses severe safety risks. Developing effective 3D adversarial attacks is essential for thoroughly testing these detection systems and exposing their vulnerabilities before real-world deployment. However, existing adversarial attacks that add optimized perturbations to 3D points have two critical limitations: they rarely cause complete object disappearance and prove difficult to implement in physical environments. We introduce the text-to-3D adversarial generation method, a novel approach enabling physically realizable attacks that can generate 3D models of objects truly invisible to LiDAR detectors and be easily realized in the real world. Specifically, we present the first empirical study that systematically investigates the factors influencing detection vulnerability by manipulating the topology, connectivity, and intensity of individual pedestrian 3D models and combining pedestrians with multiple objects within the CARLA simulation environment. Building on the insights, we propose the physically-informed text-to-3D adversarial generation (Phy3DAdvGen) that systematically optimizes text prompts by iteratively refining verbs, objects, and poses to produce LiDAR-invisible pedestrians. To ensure physical realizability, we construct a comprehensive object pool containing 13 3D models of real objects and constrain Phy3DAdvGen to generate 3D objects based on combinations of objects in this set. Extensive experiments demonstrate that our approach can generate 3D pedestrians that evade six state-of-the-art (SOTA) LiDAR 3D detectors in both CARLA simulation and physical environments, thereby highlighting vulnerabilities in safety-critical applications.

Problem

• Generating physically realizable 3D objects invisible to LiDAR detectors
• Overcoming limitations of existing 3D adversarial attacks in physical environments
• Systematically creating adversarial pedestrians that evade multiple SOTA detectors

Innovation

• Generates 3D models invisible to LiDAR detectors
• Optimizes text prompts iteratively for adversarial generation
• Constrains generation using real-world 3D object combinations