DefVINS: Visual-Inertial Odometry for Deformable Scenes

2026-01-02

arXiv.org

AI Summary

This work addresses the challenge of severe drift in traditional visual-inertial odometry (VIO) caused by violations of the scene rigidity assumption in non-rigid environments. To this end, we propose DefVINS, a novel framework that explicitly decouples IMU-anchored rigid motion from non-rigid deformations through an embedded deformation graph. A key innovation is the introduction of a visibility-aware conditional activation mechanism that dynamically enables deformation degrees of freedom based on observability analysis. This approach effectively identifies and leverages motion patterns that are otherwise unobservable, significantly enhancing localization robustness in non-rigid scenes while preserving system consistency. Ablation studies validate the efficacy of both the IMU anchoring strategy and the observability-aware deformation activation scheme.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Deformable scenes violate the rigidity assumptions underpinning classical visual-inertial odometry (VIO), often leading to over-fitting to local non-rigid motion or severe drift when deformation dominates visual parallax. We introduce DefVINS, a visual-inertial odometry framework that explicitly separates a rigid, IMU-anchored state from a non--rigid warp represented by an embedded deformation graph. The system is initialized using a standard VIO procedure that fixes gravity, velocity, and IMU biases, after which non-rigid degrees of freedom are activated progressively as the estimation becomes well conditioned. An observability analysis is included to characterize how inertial measurements constrain the rigid motion and render otherwise unobservable modes identifiable in the presence of deformation. This analysis motivates the use of IMU anchoring and informs a conditioning-based activation strategy that prevents ill-posed updates under poor excitation. Ablation studies demonstrate the benefits of combining inertial constraints with observability-aware deformation activation, resulting in improved robustness under non-rigid environments.

Problem

• deformable scenes
• visual-inertial odometry
• non-rigid motion
• drift
• rigidity assumption

Innovation

• visual-inertial odometry
• deformable scenes
• embedded deformation graph
• observability analysis
• IMU anchoring