ROMAN: Open-Set Object Map Alignment for Robust View-Invariant Global Localization

2024-10-10

arXiv.org

AI Summary

To address global robot relocalization failure under large viewpoint changes (e.g., reverse mapping), this paper proposes a viewpoint-invariant, open-set-compatible graph-matching localization method. The approach constructs open-set object subgraphs by jointly leveraging gravity-direction priors and multimodal object representations—integrating geometric shape and semantic features. It further introduces a graph neural network–driven cross-view subgraph alignment algorithm that enforces geometric constraints for robust data association. Unlike conventional methods relying on closed-set assumptions and viewpoint consistency, our framework generalizes across diverse environments. Experiments demonstrate significantly improved pose accuracy over image- and segmentation-based registration baselines in complex indoor, outdoor, and forest scenes. When integrated as a loop-closure module in multi-robot SLAM, it reduces trajectory error by 35%.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Global localization is a fundamental capability required for long-term and drift-free robot navigation. However, current methods fail to relocalize when faced with significantly different viewpoints. We present ROMAN (Robust Object Map Alignment Anywhere), a global localization method capable of localizing in challenging and diverse environments by creating and aligning maps of open-set and view-invariant objects. ROMAN formulates and solves a registration problem between object submaps using a unified graph-theoretic global data association approach with a novel incorporation of a gravity direction prior and object shape and semantic similarity. This work's open-set object mapping and information-rich object association algorithm enables global localization, even in instances when maps are created from robots traveling in opposite directions. Through a set of challenging global localization experiments in indoor, urban, and unstructured/forested environments, we demonstrate that ROMAN achieves higher relative pose estimation accuracy than other image-based pose estimation methods or segment-based registration methods. Additionally, we evaluate ROMAN as a loop closure module in large-scale multi-robot SLAM and show a 35% improvement in trajectory estimation error compared to standard SLAM systems using visual features for loop closures. Code and videos can be found at https://acl.mit.edu/roman.

Problem

• Global localization under significantly different viewpoints
• Open-set object map alignment for diverse environments
• Robust view-invariant object association for accurate pose estimation

Innovation

• Open-set object map alignment for global localization
• Graph-theoretic global data association approach
• Gravity direction prior and object similarity integration