NewsRECON: News article REtrieval for image CONtextualization

2026-01-20

AI Summary

This work addresses the challenge of accurately inferring the spatiotemporal context of news images when reverse image search (RIS) fails. To overcome the reliance on RIS evidence, the authors propose a novel approach that first retrieves candidate news articles associated with a given image using a dual-encoder architecture trained on over 90,000 news articles. Subsequently, two cross-encoders independently re-rank these candidates based on geographic and event consistency, enabling high-precision estimation of the image’s capture time and location. The framework is designed to seamlessly integrate multimodal large language models and demonstrates significant performance gains over existing methods on the TARA and 5Pils-OOC benchmarks, establishing a new state-of-the-art for RIS-free spatiotemporal grounding of news images.

Technology Category

• Computer Vision: Image and Video Retrieval
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We introduce NewsRECON, a method that links images to relevant news articles to infer their date and location from article metadata. NewsRECON leverages a corpus of over 90,000 articles and integrates: (1) a bi-encoder for retrieving event-relevant articles; (2) two cross-encoders for reranking articles by location and event consistency. Experiments on the TARA and 5Pils-OOC show that NewsRECON outperforms prior work and can be combined with a multimodal large language model to achieve new SOTA results in the absence of RIS evidence. We make our code available.

Problem

• image contextualization
• news article retrieval
• reverse image search
• geolocation
• temporal localization

Innovation

• NewsRECON
• image contextualization
• bi-encoder
• cross-encoder
• multimodal retrieval