Goal-oriented Communication for Fast and Robust Robotic Fault Detection and Recovery

2026-01-26

AI Summary

This work addresses the high latency and unreliable motion generation in existing robotic fault detection and recovery methods, which stem from the lack of co-design between the communication–computation–control (3C) loop and downstream task objectives. To overcome this limitation, the authors propose a Goal-oriented Communication (GoC) framework that jointly optimizes the 3C loop for rapid and robust fault handling. The approach innovatively leverages a 3D scene graph for semantic-aware fault detection and integrates a lightweight language model—fine-tuned via LoRA and knowledge distillation—to enhance the generalization of recovery strategies. Additionally, a task-oriented, lightweight digital twin module is introduced for precise control. Experimental results demonstrate that the proposed method reduces fault recovery time by up to 82.6% and improves task success rates by as much as 76% compared to current vision–language model-based approaches.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Autonomous robotic systems are widely deployed in smart factories and operate in dynamic, uncertain, and human-involved environments that require low-latency and robust fault detection and recovery (FDR). However, existing FDR frameworks exhibit various limitations, such as significant delays in communication and computation, and unreliability in robot motion/trajectory generation, mainly because the communication-computation-control (3C) loop is designed without considering the downstream FDR goal. To address this, we propose a novel Goal-oriented Communication (GoC) framework that jointly designs the 3C loop tailored for fast and robust robotic FDR, with the goal of minimising the FDR time while maximising the robotic task (e.g., workpiece sorting) success rate. For fault detection, our GoC framework innovatively defines and extracts the 3D scene graph (3D-SG) as the semantic representation via our designed representation extractor, and detects faults by monitoring spatial relationship changes in the 3D-SG. For fault recovery, we fine-tune a small language model (SLM) via Low-Rank Adaptation (LoRA) and enhance its reasoning and generalization capabilities via knowledge distillation to generate recovery motions for robots. We also design a lightweight goal-oriented digital twin reconstruction module to refine the recovery motions generated by the SLM when fine-grained robotic control is required, using only task-relevant object contours for digital twin reconstruction. Extensive simulations demonstrate that our GoC framework reduces the FDR time by up to 82.6% and improves the task success rate by up to 76%, compared to the state-of-the-art frameworks that rely on vision language models for fault detection and large language models for fault recovery.

Problem

• fault detection and recovery
• goal-oriented communication
• robotic systems
• 3C loop
• low-latency

Innovation

• Goal-oriented Communication
• 3D Scene Graph
• Small Language Model
• Low-Rank Adaptation
• Digital Twin