HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space

2026-06-30

AI Summary

Existing surgical vision-language models compress multi-level semantics—such as actions, steps, and phases—into a flat embedding space, neglecting their inherent hierarchical containment relationships. This oversight leads to inadequate modeling of long-range dependencies and introduces structurally misleading negative samples. To address this, this work proposes the first surgical video-language pretraining framework that explicitly incorporates hyperbolic geometry to model hierarchical semantic structures. By leveraging hyperbolic space embeddings, hierarchical contrastive learning, and cross-granularity semantic consistency constraints, the method captures the nested nature of surgical workflows. Evaluated across multiple surgical benchmarks, the approach significantly improves zero-shot and few-shot phase recognition performance and demonstrates strong generalization across diverse surgical procedures and medical institutions.

Abstract

Surgical vision-language foundation models typically adopt educational materials, such as surgical lecture videos, to transfer surgical knowledge encoded in language into visual representations. These knowledge are multi-dimensional and hierarchical: fine-grained action cues appear in narration, mid-level key steps are summarized in subsection headings, and global procedural context, such as patient history and surgical strategy, is described in abstract texts. Prior work largely collapses these heterogeneous signals into a single flat embedding space, implicitly assuming independence across hierarchy levels. However, this is suboptimal because it ignores cross-level semantic containment, e.g., actions belong to steps, steps compose phases, weakens long-range dependency modeling. To this end, we propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure by mitigating structural false negatives induced by procedural context and enforcing semantic consistency between parent phases and their constituent child steps. Extensive experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition across procedures and institutions.

Problem

• surgical video-language pre-training
• hierarchical structure
• hyperbolic space
• semantic containment
• long-range dependency

Innovation

• hyperbolic space
• hierarchical representation
• surgical video-language pre-training
• semantic consistency
• structure-aware modeling