Finding 3D Positions of Distant Objects from Noisy Camera Movement and Semantic Segmentation Sequences

2025-09-25

AI Summary

Accurate 3D localization of distant objects—e.g., fire sources in UAV-based wildland fire monitoring—remains challenging under noisy camera motion and semantic segmentation sequences, particularly when onboard computational constraints preclude dense depth estimation or full 3D reconstruction. Method: This paper proposes a lightweight, particle-filter-based framework for single- and multi-object 3D localization. It is the first to adapt particle filtering to this setting, requiring no detector-specific design and thus exhibiting strong task transferability. The method tightly fuses GNSS-aided camera pose estimates with semantic segmentation masks. Contribution/Results: Evaluated on both synthetic and real-world UAV datasets, the approach achieves robust 3D localization under extreme conditions—long range and low signal-to-noise ratio—where conventional methods fail. Its low computational footprint makes it suitable for resource-constrained platforms, significantly enhancing safety-critical perception capabilities in autonomous aerial systems.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: 3D Computer Vision
• Search and Optimization: Local Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D object localisation based on a sequence of camera measurements is essential for safety-critical surveillance tasks, such as drone-based wildfire monitoring. Localisation of objects detected with a camera can typically be solved with dense depth estimation or 3D scene reconstruction. However, in the context of distant objects or tasks limited by the amount of available computational resources, neither solution is feasible. In this paper, we show that the task can be solved using particle filters for both single and multiple target scenarios. The method was studied using a 3D simulation and a drone-based image segmentation sequence with global navigation satellite system (GNSS)-based camera pose estimates. The results showed that a particle filter can be used to solve practical localisation tasks based on camera poses and image segments in these situations where other solutions fail. The particle filter is independent of the detection method, making it flexible for new tasks. The study also demonstrates that drone-based wildfire monitoring can be conducted using the proposed method paired with a pre-existing image segmentation model.

Problem

• Locating distant 3D objects using noisy camera movement and semantic segmentation
• Solving 3D localization when depth estimation or reconstruction is infeasible
• Enabling drone-based wildfire monitoring with limited computational resources

Innovation

• Particle filters for single and multiple target scenarios
• Uses camera poses and image segments for localization
• Independent of detection method for flexible application