Semantic Temporal Single-photon LiDAR

2025-12-02

AI Summary

To address the weak capability of time-resolved single-photon LiDAR (TSP-LiDAR) in open-set scenarios—particularly under low signal-to-noise ratio (SNR) and ultra-short acquisition times (i.e., sparse photon counts)—to recognize previously unseen targets, this paper proposes a Semantic Time-resolved Single-Photon LiDAR framework. The core innovation is an autonomously updating semantic knowledge base, which reformulates target recognition as a semantic communication process, enabling online accumulation of novel target semantics and zero-shot adaptation without model retraining. Our method integrates single-photon detection, semantic communication theory, and a lightweight neural network to achieve robust semantic-level feature extraction and matching from sparse, noisy photon data. Experiments demonstrate 89% recognition accuracy across nine previously unknown target classes—outperforming baseline methods by 23 percentage points—and significantly enhancing robustness in open environments and continual learning capability.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Temporal single-photon (TSP-) LiDAR presents a promising solution for imaging-free target recognition over long distances with reduced size, cost, and power consumption. However, existing TSP-LiDAR approaches are ineffective in handling open-set scenarios where unknown targets emerge, and they suffer significant performance degradation under low signal-to-noise ratio (SNR) and short acquisition times (fewer photons). Here, inspired by semantic communication, we propose a semantic TSP-LiDAR based on a self-updating semantic knowledge base (SKB), in which the target recognition processing of TSP-LiDAR is formulated as a semantic communication. The results, both simulation and experiment, demonstrate that our approach surpasses conventional methods, particularly under challenging conditions of low SNR and limited acquisition time. More importantly, our self-updating SKB mechanism can dynamically update the semantic features of newly encountered targets in the SKB, enabling continuous adaptation without the need for extensive retraining of the neural network. In fact, a recognition accuracy of 89% is achieved on nine types of unknown targets in real-world experiments, compared to 66% without the updating mechanism. These findings highlight the potential of our framework for adaptive and robust target recognition in complex and dynamic environments.

Problem

• Enables open-set target recognition in TSP-LiDAR systems
• Improves performance under low SNR and short acquisition times
• Dynamically updates semantic knowledge without extensive retraining

Innovation

• Semantic communication framework for LiDAR target recognition
• Self-updating semantic knowledge base for dynamic adaptation
• Improved accuracy under low SNR and short acquisition times