KLTNet: Learning Sparse Feature Tracking for Robust and Accurate Monocular Visual-Inertial Odometry

2026-08-25

AI Summary

本文提出KLTNet，一种基于学习的稀疏特征跟踪器，旨在提高单目视觉惯性里程计在快速运动或低纹理环境下的准确性和鲁棒性。

Abstract

Many feature-based visual-inertial odometry (VIO) systems rely on sparse feature tracking, whose accuracy and robustness directly affect state estimation. Classical KLT trackers rely primarily on local image patches and can become unreliable under rapid motion or in low-texture environments. We propose KLTNet, a lightweight learning-based, plug-and-play sparse feature tracker designed to replace classical KLT trackers in KLT-based VIO front ends. KLTNet follows a coarse-to-fine, dense-to-sparse architecture that combines low-resolution dense optical flow for robust global motion initialization with triplet-patch refinement for accurate and temporally consistent tracking. A fixed reference patch provides a stable anchor throughout each feature track and helps reduce accumulated tracking drift. In addition, KLTNet predicts anisotropic confidence weights supervised through differentiable multi-view triangulation, which can be used as observation weights in compatible VIO estimators. Experiments with VINS-Mono and OpenVINS on public benchmarks and a self-collected low-texture dataset demonstrate improved tracking and odometry accuracy over classical KLT, while maintaining real-time performance on an embedded platform.

Problem

• sparse feature tracking
• visual-inertial odometry
• KLT tracker
• low-texture environments
• rapid motion

Innovation

• KLTNet
• Sparse Feature Tracking
• Dense Optical Flow
• Triplet-Patch Refinement
• Anisotropic Confidence Weights