VisChronos: Revolutionizing Image Captioning Through Real-Life Events

2026-06-22

AI Summary

This work addresses the limitations of conventional image captioning approaches in capturing the contextual narrative of real-world events by proposing an event-centric image understanding framework. For the first time, real historical events are integrated as external knowledge sources into image description generation. The framework synergistically combines large language models, dense image captioning models, and an event knowledge base, supported by a newly curated dataset, EventCap, to enable automatic mapping from a single image to context-aware, semantically rich event descriptions. User studies demonstrate that the proposed method significantly outperforms existing approaches in terms of descriptive accuracy, coherence, and event relevance, thereby advancing event-oriented image understanding research.

Abstract

This paper aims to bridge the semantic gap between visual content and natural language understanding by leveraging historical events in the real world as a source of knowledge for caption generation. We propose VisChronos, a novel framework that utilizes large language models and dense captioning models to identify and describe real-life events from a single input image. Our framework can automatically generate detailed and context-aware event descriptions, enhancing the descriptive quality and contextual relevance of generated captions to address the limitations of traditional methods in capturing contextual narratives. Furthermore, we introduce a new dataset, EventCap (https://zenodo.org/records/14004909), specifically constructed using the proposed framework, designed to enhance the model's ability to identify and understand complex events. The user study demonstrates the efficacy of our solution in generating accurate, coherent, and event-focused descriptions, paving the way for future research in event-centric image understanding.

Problem

• image captioning
• real-life events
• contextual narratives
• semantic gap
• event understanding

Innovation

• event-centric image captioning
• large language models
• dense captioning
• context-aware description
• EventCap dataset