Lorentz Framework for Semantic Segmentation

2026-04-18

AI Summary

This work addresses the numerical instability, optimization challenges, and computational inefficiency of existing hyperbolic semantic segmentation methods operating in the Poincaré ball model by introducing the Lorentz model to semantic segmentation for the first time. The proposed approach features a stable training strategy that eliminates the need for Riemannian optimizers and establishes a general-purpose segmentation framework. It incorporates text embeddings to guide pixel-wise hierarchical representations, seamlessly integrates mainstream Euclidean backbones, and introduces novel uncertainty and confidence metrics based on Lorentz cone embeddings. Experiments demonstrate significant improvements in segmentation robustness and uncertainty awareness across ADE20K, COCO-Stuff-164k, Pascal-VOC, and Cityscapes benchmarks, yielding flatter generalization minima and enabling versatile extensions such as zero-shot transfer and hierarchical retrieval.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Semantic segmentation in hyperbolic space enables compact modeling of hierarchical structure while providing inherent uncertainty quantification. Prior approaches predominantly rely on the Poincaré ball model, which suffers from numerical instability, optimization, and computational challenges. We propose a novel, tractable, architecture-agnostic semantic segmentation framework (pixel-wise and mask classification) in the hyperbolic Lorentz model. We employ text embeddings with semantic and visual cues to guide hierarchical pixel-level representations in Lorentz space. This enables stable and efficient optimization without requiring a Riemannian optimizer, and easily integrates with existing Euclidean architectures. Beyond segmentation, our approach yields free uncertainty estimation, confidence map, boundary delineation, hierarchical and text-based retrieval, and zero-shot performance, reaching generalized flatter minima. We introduce a novel uncertainty and confidence indicator in Lorentz cone embeddings. Further, we provide analytical and empirical insights into Lorentz optimization via gradient analysis. Extensive experiments on ADE20K, COCO-Stuff-164k, Pascal-VOC, and Cityscapes, utilizing state-of-the-art per-pixel classification models (DeepLabV3 and SegFormer) and mask classification models (mask2former and maskformer), validate the effectiveness and generality of our approach. Our results demonstrate the potential of hyperbolic Lorentz embeddings for robust and uncertainty-aware semantic segmentation. Code is available at https://github.com/mxahan/Lorentz_semantic_segmentation.

Problem

• semantic segmentation
• hyperbolic space
• Poincaré ball model
• numerical instability
• optimization challenges

Innovation

• Lorentz model
• hyperbolic semantic segmentation
• uncertainty quantification
• text-guided representation
• architecture-agnostic framework