Responsibility Distribution Estimation in Ego-View Accident Videos with Multimodal Large Language Models

2026-07-03

AI Summary

This work addresses the limitations of existing traffic accident analyses, which struggle to objectively reconstruct the observable information available to drivers prior to collisions and lack fine-grained modeling of responsibility allocation. To bridge this gap, the paper introduces a novel task—estimating responsibility distributions from first-person driver-view accident videos—and proposes a multimodal large language model (MLLM)-based reasoning framework. This framework integrates multiple input sources, including raw video frames, semantic segmentation maps, and textual descriptions, and incorporates an LLM-assisted pipeline for responsibility annotation. Experimental results establish a strong baseline for this new task and demonstrate the effectiveness of MLLMs in performing complex, socially and legally constrained attribution reasoning, thereby expanding the applicability of accident video analysis.

Abstract

Recent studies on multimodal traffic accident understanding have mainly relied on infrastructure-camera footage, satellite imagery, or structured crash records. However, such data sources are costly to deploy and maintain at large scale, and they cannot objectively capture what the driver was actually able to observe before the accident. In contrast, ego-view accident videos directly represent the driver's visual perspective, making them suitable for reasoning about avoidability and driver responsibility. In this paper, we introduce responsibility distribution estimation for ego-view traffic accident videos, a new task in which a model predicts the percentage of responsibility assigned to each involved agent. We construct an LLM-assisted responsibility annotation pipeline and fine-tune multimodal large language models under multiple input settings, including raw frames, segmentation-enhanced input, and textual descriptions. Experimental results establish a strong initial benchmark, demonstrating that multimodal LLMs can effectively perform this nuanced, constraint-based reasoning task. Our findings suggest that ego-centric accident videos provide a promising foundation for socially and legally meaningful multimodal reasoning beyond conventional accident classification and explanation tasks.

Problem

• responsibility distribution
• ego-view accident videos
• multimodal large language models
• driver responsibility
• traffic accident understanding

Innovation

• responsibility distribution estimation
• ego-view accident videos
• multimodal large language models
• driver responsibility
• constraint-based reasoning