Exosense: A Vision-Based Scene Understanding System For Exoskeletons

2024-03-21

AI Summary

To address weak scene perception, significant pose drift, and poor terrain adaptability of self-balancing exoskeletons during prolonged operation in complex multi-level indoor environments, this paper proposes a vision-centric real-time scene understanding system. Methodologically, we introduce a novel multi-sensor visual-inertial mapping device that tightly fuses monocular/RGB-D vision, inertial measurement units (IMUs), and simultaneous localization and mapping (SLAM) to construct a lightweight, domain-specific navigation stack. This stack enables dense terrain reconstruction, visual relocalization, and cross-floor mapping. Our contributions include: (i) high-precision pose estimation under periodic gait constraints, achieving odometry drift of only 4 cm/m; (ii) sub-centimeter terrain reconstruction with mean error <1 cm; and (iii) robust visual localization in previously mapped environments. These advances substantially enhance the exoskeleton’s autonomous navigation capability and human–robot collaborative comfort in realistic indoor settings.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Self-balancing exoskeletons are a key enabling technology for individuals with mobility impairments. While the current challenges focus on human-compliant hardware and control, unlocking their use for daily activities requires a scene perception system. In this work, we present Exosense, a vision-centric scene understanding system for self-balancing exoskeletons. We introduce a multi-sensor visual-inertial mapping device as well as a navigation stack for state estimation, terrain mapping and long-term operation. We tested Exosense attached to both a human leg and Wandercraft's Personal Exoskeleton in real-world indoor scenarios. This enabled us to test the system during typical periodic walking gaits, as well as future uses in multi-story environments. We demonstrate that Exosense can achieve an odometry drift of about 4 cm per meter traveled, and construct terrain maps under 1 cm average reconstruction error. It can also work in a visual localization mode in a previously mapped environment, providing a step towards long-term operation of exoskeletons.

Problem

• Exoskeleton
• Environmental Perception
• Complex Environment Adaptability

Innovation

• Exosense
• Terrain Perception
• Stability and Comfort