SplatCtrl: Perception-Action Coupling via Gaussian Scene Representations and Reactive Robot Control

2026-07-09

AI Summary

This work addresses the challenge of achieving safe and real-time obstacle avoidance and motion control for robots operating in unstructured dynamic environments. The authors propose a novel approach that integrates 3D Gaussian Splatting–based scene reconstruction with reactive control. By leveraging voxel filtering and dynamic Gaussian relocation, the method enables efficient reconstruction of dynamic scenes. Notably, it introduces—for the first time—a differentiable, continuous signed distance function derived from isotropic Gaussians, seamlessly bridging implicit representations with classical distance fields. This representation is further combined with Control Barrier Functions to establish a closed-loop perception-to-action pipeline. Extensive evaluations in simulation, on physical robots, and within human-robot shared workspaces demonstrate the system’s capability to perform high-fidelity dynamic scene reconstruction and ensure real-time, collision-free navigation in previously unknown environments.

Abstract

Robotic manipulators excel in structured environments but face substantial challenges in unstructured and dynamic settings. This paper presents SplatCtrl, a unified framework for real-time scene reconstruction and reactive robot motion generation to enable collision-free robotic arm control in previously unseen and continuously changing environments. Building on 3D Gaussian Splatting (3D-GS), we introduce a hybrid voxel-based filtering and dynamic Gaussian relocation strategy that supports efficient scene reconstruction from RGB-D streams while accommodating environmental changes. For safe and reactive control, we further propose a method for deriving continuous signed distance functions from isotropic Gaussians, providing stable and differentiable collision probability estimates that bridge classical distance fields with the modern implicit representation. These continuous distance metrics are incorporated into control barrier functions, resulting in a unified perception-action coupling framework that supports smooth and reliable real-time motion generation in response to scene changes. Experimental validation in simulation, on physical robot, and within shared human-robot workspace demonstrates the framework's effectiveness, achieving integrated scene reconstruction and reactive control in uncertain, and dynamic environments.

Problem

• robotic manipulation
• dynamic environments
• collision-free control
• real-time perception
• reactive motion planning

Innovation

• 3D Gaussian Splatting
• reactive robot control
• continuous signed distance function
• control barrier functions
• perception-action coupling