ReMoSPLAT: Reactive Mobile Manipulation Control on a Gaussian Splat

2025-12-10

AI Summary

To address the challenge of real-time collision avoidance and coordinated control for mobile manipulators in dense environments, this paper proposes a lightweight reactive control framework based on Gaussian Splatting. The method bypasses computationally expensive global path planning by integrating geometric constraints and rasterization-accelerated collision queries within a unified quadratic programming (QP) formulation, enabling joint optimization of base and manipulator motion while preserving target pose accuracy and ensuring collision-free operation. Its key contributions are: (i) the first use of Gaussian Splatting as a geometric proxy for real-time collision detection; (ii) two efficient robot–obstacle distance estimation algorithms; and (iii) tight coupling of geometric reasoning with differentiable rasterization for low-latency collision querying. Evaluated on both synthetic and real-world scanned scenes, the approach achieves a 32% improvement in obstacle avoidance success rate and maintains single-step control latency below 15 ms—performance approaching that of an ideal ground-truth controller.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Reactive control can gracefully coordinate the motion of the base and the arm of a mobile manipulator. However, incorporating an accurate representation of the environment to avoid obstacles without involving costly planning remains a challenge. In this work, we present ReMoSPLAT, a reactive controller based on a quadratic program formulation for mobile manipulation that leverages a Gaussian Splat representation for collision avoidance. By integrating additional constraints and costs into the optimisation formulation, a mobile manipulator platform can reach its intended end effector pose while avoiding obstacles, even in cluttered scenes. We investigate the trade-offs of two methods for efficiently calculating robot-obstacle distances, comparing a purely geometric approach with a rasterisation-based approach. Our experiments in simulation on both synthetic and real-world scans demonstrate the feasibility of our method, showing that the proposed approach achieves performance comparable to controllers that rely on perfect ground-truth information.

Problem

• Develops reactive control for mobile manipulators using Gaussian Splat representation
• Enables obstacle avoidance without costly planning in cluttered environments
• Compares geometric and rasterisation methods for efficient distance calculation

Innovation

• Reactive controller using quadratic program formulation
• Gaussian Splat representation for collision avoidance
• Efficient distance calculation via geometric and rasterization methods