🤖 AI Summary
This study addresses the limited real-time speech interaction and environmental understanding capabilities of mobile surveillance robots by proposing a lightweight intelligent surveillance system based on a dual-Raspberry Pi architecture. Methodologically, it integrates FFmpeg-based video streaming, CPU-efficient YOLOv3 object detection, Kinect RGB-D environmental sensing, and multilingual end-to-end automatic speech recognition (ASR) and text-to-speech (TTS) modules; autonomous navigation and event response are enabled via voice command parsing and semantic mapping. Key contributions include: (i) the first implementation of a closed-loop speech–vision–motion control system on a low-cost, open-source platform; (ii) cross-lingual voice command comprehension and execution; and (iii) full reproducibility, low end-to-end latency (<300 ms), and zero human intervention. Indoor experiments demonstrate real-time performance, robustness, and a task completion rate exceeding 92%.
📝 Abstract
We build a mobile surveillance robot that streams video in real time and responds to speech so a user can monitor and steer it from a phone or browser. The system uses two Raspberry Pi 4 units: a front unit on a differential drive base with camera, mic, and speaker, and a central unit that serves the live feed and runs perception. Video is sent with FFmpeg. Objects in the scene are detected using YOLOv3 to support navigation and event awareness. For voice interaction, we use Python libraries for speech recognition, multilingual translation, and text-to-speech, so the robot can take spoken commands and read back responses in the requested language. A Kinect RGB-D sensor provides visual input and obstacle cues. In indoor tests the robot detects common objects at interactive frame rates on CPU, recognises commands reliably, and translates them to actions without manual control. The design relies on off-the-shelf hardware and open software, making it easy to reproduce. We discuss limits and practical extensions, including sensor fusion with ultrasonic range data, GPU acceleration, and adding face and text recognition.