Learning to Deny: Action Denial in Multimodal Large Language Models

2026-06-30

AI Summary

This work addresses a critical yet underexplored limitation in multimodal large language models (MLLMs): their frequent inability to accurately deny actions that did not occur in videos, even when contextual cues strongly suggest their absence. To systematically investigate this action denial capability, the authors introduce UCF101-AD, a novel benchmark comprising paired video clips depicting both the presence and explicit denial of specific actions. They further propose CausalAct, a learnable causal graph reasoning mechanism grounded in natural language prompts that leverages causal relationships to improve inference. Comprehensive experiments across 20 state-of-the-art multimodal models reveal a widespread deficiency in action denial performance. Crucially, integrating causal cues through CausalAct significantly reduces false positive rates, demonstrating the efficacy of causal reasoning in enhancing video understanding robustness.

Abstract

Multimodal large language models (MLLMs) have rapidly advanced video understanding, achieving strong zero-shot and few-shot recognition across standard benchmarks. Yet their ability to deny an action by recognizing when an activity is not happening despite strong contextual cues remains largely unexplored. We introduce UCF101-AD, a large-scale benchmark consisting of paired Action-Presence and Action-Denial clips, designed to evaluate this capacity for denial. Each negative video in UCF101-AD preserves the same contextual and motion cues, including persons, objects, and locations, as its positive counterpart, but the defining action itself is explicitly absent. Evaluating 20 state-of-the-art MLLMs reveals a consistent failure: models that exceed 85% accuracy on the positive action classes collapse below 50% on their action-denial counterparts, indicating a strong inclination to affirm plausible actions rather than verify that they truly occur. This exposes a critical blind spot in modern video understanding: the inability to reason causally about whether a motion actually happens. To probe this issue, we explore a causal graph formulation, CausalAct, which expresses scene structure through natural-language prompts linking context, interaction, and motion. Incorporating such causal cues substantially reduces false positives, demonstrating that denial is a learnable reasoning skill. UCF101-AD provides a new lens for diagnosing and improving causal reasoning in multimodal models. Dataset and relevant code: https://github.com/raiyaan-abdullah/Learn-to-Deny.

Problem

• action denial
• multimodal large language models
• video understanding
• causal reasoning
• negative recognition

Innovation

• action denial
• multimodal large language models
• causal reasoning
• video understanding
• UCF101-AD