GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models

2026-07-09

AI Summary

This work addresses the inefficiency of video large language models caused by excessive visual tokens and the semantic ambiguity introduced by existing compression methods that often overemphasize local regions. To this end, the authors propose GeoTrace, a training-free spatiotemporal token compression framework. GeoTrace preserves semantically rich skeleton tokens through Contextual Farthest-Point Anchoring (CFPA) to ensure high coverage and generates trajectory-aware, low-ambiguity residual event tokens via Trajectory-Constrained Residual Condensation (TCRC). When retaining only 10% of the original visual tokens, GeoTrace reduces computational cost by 12.99× TFLOPs on LLaVA-OneVision while preserving 99.1% of the original performance. Extensive experiments across four benchmarks and multiple models demonstrate its strong generalization capability.

Abstract

Although Video Large Language Models (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame-wise saliency or heuristic token merging, which can over-focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training-free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest-Point Anchoring (CFPA) preserves salient, context-consistent, and high-coverage skeleton tokens, while Trajectory-Constrained Residual Condensation (TCRC) compresses residual tokens through one-to-one temporal trajectories and constrained near-manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA-OneVision, with only 10\% visual tokens retained, GeoTrace achieves a \(12.99\times\) TFLOPs reduction while preserving 99.1\% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \href{https://github.com/guohuan-xie/GeoTrace.git}{\texttt{Code}}.

Problem

• Video LLMs
• token compression
• visual tokens
• spatiotemporal representation
• feature ambiguity

Innovation

• token compression
• geometry-aware
• trajectory-constrained
• training-free
• video understanding