FineVAU: A Novel Human-Aligned Benchmark for Fine-Grained Video Anomaly Understanding
Current evaluation methods for video anomaly understanding (VAU) struggle to accurately assess models’ fine-grained descriptive capabilities regarding anomalous events, often diverging from human perception. This work reframes VAU as a tripartite parsing task—capturing the anomaly’s “What,” the involved entities “Who,” and the spatial context “Where”—and introduces FineW3, a new benchmark dataset, along with FVScore, a human-aligned evaluation metric. FVScore enables the first interpretable, fine-grained assessment of large vision-language models (LVLMs) based on their coverage of critical visual elements. Structured automatic annotations augment manual labeling, and scoring is grounded in key visual components. Human evaluations demonstrate that FVScore significantly outperforms existing metrics. Experiments further reveal that LVLMs underperform on tasks requiring spatiotemporal fine-grained reasoning but excel in scenarios with static or strong visual cues.