FLARE: Full-Modality Long-Video Audiovisual Retrieval Benchmark with User-Simulated Queries

2026-05-11

AI Summary

Existing video retrieval benchmarks are largely confined to short clips, single modalities, and descriptive text, limiting their ability to evaluate models’ capacity to understand cross-modal (audiovisual) user queries in real-world long-form video scenarios. This work proposes the first full-modality audiovisual retrieval benchmark specifically designed for long videos, introducing large-scale long-form content, realistic user-style queries, and a stringent dual-modality constraint mechanism. Through multimodal annotations and a dual-modality filtering strategy, the benchmark supports evaluation across visual, audio, and unified audiovisual dimensions for both descriptive and query-based retrieval tasks. Experiments on 15 representative models reveal that user-style queries significantly impact performance, strong descriptive retrieval capabilities do not readily transfer to query-based tasks, and audio–language alignment remains a critical bottleneck.

Abstract

As video becomes increasingly central to information dissemination and multimodal large language models (MLLMs) continue to advance, evaluating video retrieval has become increasingly important. In realistic search scenarios, this requires matching short user queries to long-form content using both visual and auditory evidence. Yet existing retrieval benchmarks are still dominated by short clips, single modalities, and caption-based evaluation. We introduce FLARE, a full-modality long-video audiovisual retrieval benchmark with user-simulated queries. Built from 399 carefully screened Video-MME videos (10--60 min, 225.4 h) to ensure source quality and diversity, FLARE contains 87,697 clips annotated with vision, audio, and unified audiovisual captions, together with 274,933 user-style queries. Cross-modal queries are further filtered by a hard bimodal constraint, requiring retrieval to fail under either modality alone but succeed when both are combined. FLARE evaluates models under two regimes, caption-based and query-based retrieval, across vision, audio, and unified audiovisual settings. Experiments with 15 representative retrievers show that user-style queries substantially change model behavior, strong caption-based performance does not always transfer to query-based retrieval, and audio--language alignment remains a key bottleneck for unified audiovisual retrieval. Our code and data are released at https://flarebench.github.io/

Problem

• long-video retrieval
• audiovisual retrieval
• user-simulated queries
• multimodal benchmark
• cross-modal matching

Innovation

• long-video retrieval
• audiovisual benchmark
• user-simulated queries
• bimodal constraint
• multimodal evaluation