Time Imprint: Learning Time-Aware Representations in Multi-Modal Knowledge Graphs

2026-07-08

AI Summary

This work addresses the challenge of distinguishing entities with highly similar textual and visual representations in multimodal knowledge graphs, a task further complicated by semantic sparsity and noisy multi-timestamp signals that existing methods fail to adequately leverage. To overcome these limitations, the study introduces time as an independent modality on par with text and images and proposes a tri-view contrastive learning framework that jointly aligns multimodal representations. Additionally, it designs a compact timestamp subset selection strategy coupled with an attention-based pooling mechanism to produce discriminative and robust temporal embeddings. Evaluated on three benchmark datasets, the approach achieves state-of-the-art performance, improving Hits@1 by up to 6.07% overall and yielding a remarkable 58% performance gain on the most challenging 1% of samples.

Abstract

Multi-Modal Knowledge Graphs (MMKGs) enrich entities with multiple modalities such as text and images, yet entities with highly similar multi-modal features remain difficult to distinguish. Temporal information of an entity can serve as an additional modality to disambiguate such entities, but existing approaches rarely treat time as a separate modality alongside text and images due to two major challenges: (1) sparse temporal semantics, which hinder alignment with richer modalities, and (2) multiple timestamps, which introduce noise or reduce robustness in representation learning. To address these challenges, we propose Time Imprint, a framework that treats time as an entity-level modality and jointly aligns temporal, textual, and visual representations via a three-view contrastive objective. Additionally, to mitigate multi-timestamp ambiguity, Time Imprint studies a compact timestamp subset selection design space and aggregates the selected timestamps into a discriminative temporal embedding with attention pooling, balancing temporal specificity and robustness. Experiments on three MMKG benchmarks demonstrate that Time Imprint achieves state-of-the-art link prediction performance, improving Hits@1 by up to 6.07\% overall and yielding up to 58\% gains on the subset of the top-1\% ambiguity samples. We further examine different fusion strategies and the sensitivity to timestamp availability and quality, clarifying when and why time-as-modality is most beneficial, while adding only modest training overhead. We release our code at https://anonymous.4open.science/r/Time-Imprint.

Problem

• Multi-Modal Knowledge Graphs
• Entity Disambiguation
• Temporal Information
• Time-Aware Representations
• Modality Alignment

Innovation

• time-as-modality
• multi-modal knowledge graphs
• contrastive learning
• temporal embedding
• timestamp selection