Prof. Robot: Differentiable Robot Rendering Without Static and Self-Collisions

2025-03-14

AI Summary

Differentiable rendering–driven robot motion optimization often lacks physical awareness, leading to frequent static environment and self-collisions. Method: We propose a physics-aware differentiable robot rendering framework that jointly models implicit scene geometry and robot configuration by coupling a neural collision classifier with Eikonal regularization—ensuring consistent, differentiable gradients for collision constraints. This enables end-to-end, image-supervised, collision-free motion optimization without explicit collision detection or handcrafted penalty terms. Contribution/Results: Our method significantly reduces collision rates across diverse scenarios while improving optimization reliability. It is compatible with mainstream differentiable rendering frameworks and achieves superior qualitative and quantitative performance compared to existing baselines.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Differentiable rendering has gained significant attention in the field of robotics, with differentiable robot rendering emerging as an effective paradigm for learning robotic actions from image-space supervision. However, the lack of physical world perception in this approach may lead to potential collisions during action optimization. In this work, we introduce a novel improvement on previous efforts by incorporating physical awareness of collisions through the learning of a neural robotic collision classifier. This enables the optimization of actions that avoid collisions with static, non-interactable environments as well as the robot itself. To facilitate effective gradient optimization with the classifier, we identify the underlying issue and propose leveraging Eikonal regularization to ensure consistent gradients for optimization. Our solution can be seamlessly integrated into existing differentiable robot rendering frameworks, utilizing gradients for optimization and providing a foundation for future applications of differentiable rendering in robotics with improved reliability of interactions with the physical world. Both qualitative and quantitative experiments demonstrate the necessity and effectiveness of our method compared to previous solutions.

Problem

• Addresses collision risks in differentiable robot rendering.
• Introduces neural collision classifier for physical awareness.
• Enhances action optimization with Eikonal regularization.

Innovation

• Neural robotic collision classifier for physical awareness
• Eikonal regularization for consistent gradient optimization
• Seamless integration into differentiable robot rendering frameworks