Deep Fusion of Ultra-Low-Resolution Thermal Camera and Gyroscope Data for Lighting-Robust and Compute-Efficient Rotational Odometry

2025-06-14

AI Summary

To address the low accuracy and severe gyroscope drift of rotational odometry on resource-constrained micro-robots (e.g., UAVs) under varying illumination, this paper proposes a thermal–gyro tightly coupled fusion method that jointly estimates angular velocity from ultra-low-resolution (16×12) thermal imagery and gyroscope measurements, enabling illumination-invariant, low-drift, and high-throughput heading estimation. Key contributions include: (i) the first end-to-end thermal–gyro tight-coupling paradigm; (ii) a lightweight CNN architecture with cross-modal early-feature alignment; and (iii) the first publicly available synchronized thermal–gyro–ground-truth dataset. Experiments demonstrate that, with 83% less memory footprint and inference latency under 5 ms, the proposed method reduces angular error by 67% compared to pure gyroscope integration, while exhibiting complete robustness to illumination changes.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Accurate rotational odometry is crucial for autonomous robotic systems, particularly for small, power-constrained platforms such as drones and mobile robots. This study introduces thermal-gyro fusion, a novel sensor fusion approach that integrates ultra-low-resolution thermal imaging with gyroscope readings for rotational odometry. Unlike RGB cameras, thermal imaging is invariant to lighting conditions and, when fused with gyroscopic data, mitigates drift which is a common limitation of inertial sensors. We first develop a multimodal data acquisition system to collect synchronized thermal and gyroscope data, along with rotational speed labels, across diverse environments. Subsequently, we design and train a lightweight Convolutional Neural Network (CNN) that fuses both modalities for rotational speed estimation. Our analysis demonstrates that thermal-gyro fusion enables a significant reduction in thermal camera resolution without significantly compromising accuracy, thereby improving computational efficiency and memory utilization. These advantages make our approach well-suited for real-time deployment in resource-constrained robotic systems. Finally, to facilitate further research, we publicly release our dataset as supplementary material.

Problem

• Fuses thermal and gyro data for robust rotational odometry
• Reduces thermal camera resolution without losing accuracy
• Enables efficient odometry for resource-constrained robots

Innovation

• Fuses ultra-low-resolution thermal and gyroscope data
• Uses lightweight CNN for rotational speed estimation
• Reduces thermal resolution without accuracy loss