Conditional Multi-Event Temporal Grounding in Long-Form Video

2026-06-13

AI Summary

Existing methods struggle to simultaneously localize multiple events in long videos that satisfy complex spatiotemporal conditions, and there is a lack of a unified benchmark for evaluating multi-event localization, counting, and negative query recognition. To address this, this work introduces CoMET-Bench, comprising 600 long videos and 2,789 composite spatiotemporal queries, along with a standardized evaluation protocol and a novel Rejection-F1 metric designed to discourage speculative model behavior. Furthermore, the authors develop CoMET-Agent, a training-free framework that leverages multimodal large language models to enable structured search and aggregation for joint reasoning over multi-event temporal localization and negative queries. Experiments show that CoMET-Agent outperforms GPT-5 by 6.1% on F1@0.5, highlighting limitations in current approaches regarding fine-grained entity tracking, uniform retrieval, and causal event pairing.

Abstract

Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing every event that satisfies compositional temporal and spatial conditions. Existing benchmarks fall short: they localize only a single moment per query, count without temporal conditions, or treat grounding and counting as disjoint tasks. We introduce CoMET-Bench for Conditional Multi-Event Temporal Grounding in long-form video, comprising 2789 queries over 600 videos averaging 33.8 minutes across five real-world domains, with each query composed from 4 temporal conditions, 3 spatial conditions, and a dedicated negative-query subset. We further propose a unified evaluation protocol jointly measuring counting, grounding, and negative-query recognition, including a new Rejection-F1 metric that prevents trivial gaming by lazy "always-empty" models. Benchmarking a broad suite of MLLMs, agent-based, and grounding-specialized methods reveals that existing approaches remain far from solving this task. Building on these findings, we propose CoMET-Agent, a training-free agentic framework that reformulates the task as structured search-and-aggregate, improving F1@0.5 by 6.1% over GPT-5 purely through structural reasoning. Failure analysis further surfaces three open directions: fine-grained entity tracking, position-uniform retrieval, and causal event pairing.

Problem

• temporal grounding
• multi-event localization
• compositional conditions
• long-form video
• video understanding

Innovation

• temporal grounding
• multimodal large language models
• conditional multi-event localization
• negative-query recognition
• structured reasoning