See More, Store Less: Memory-Efficient Resolution for Video Moment Retrieval

2026-01-14

AI Summary

This work addresses the challenges in video moment retrieval posed by memory bottlenecks from dense frame processing and critical information loss due to sparse sampling. To overcome these limitations, the authors propose SMORE, a novel framework that integrates query-guided semantic encoding with an adaptive frame compression mechanism. Leveraging a multimodal large language model, SMORE enables query-aware importance modulation to efficiently preserve salient high-resolution video content under constrained memory budgets. By moving beyond conventional sparse sampling strategies, the method achieves state-of-the-art performance across three major benchmarks: QVHighlights, Charades-STA, and ActivityNet-Captions.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent advances in Multimodal Large Language Models (MLLMs) have improved image recognition and reasoning, but video-related tasks remain challenging due to memory constraints from dense frame processing. Existing Video Moment Retrieval (VMR) methodologies rely on sparse frame sampling, risking potential information loss, especially in lengthy videos. We propose SMORE (See MORE, store less), a framework that enhances memory efficiency while maintaining high information resolution. SMORE (1) uses query-guided captions to encode semantics aligned with user intent, (2) applies query-aware importance modulation to highlight relevant segments, and (3) adaptively compresses frames to preserve key content while reducing redundancy. This enables efficient video understanding without exceeding memory budgets. Experimental validation reveals that SMORE achieves state-of-the-art performance on QVHighlights, Charades-STA, and ActivityNet-Captions benchmarks.

Problem

• Video Moment Retrieval
• Memory Efficiency
• Frame Sampling
• Multimodal Large Language Models
• Information Loss

Innovation

• Video Moment Retrieval
• Memory Efficiency
• Query-Guided Captioning
• Adaptive Frame Compression
• Multimodal Large Language Models