TADACap: Time-series Adaptive Domain-Aware Captioning

2025-04-15

AI Summary

Existing time-series image captioning methods are generic and lack domain-specific adaptation capability, necessitating extensive retraining for cross-domain transfer. Method: We propose TADACap, a training-free, retrieval-based domain-aware captioning framework tailored for high-specialization domains such as finance and healthcare, addressing both domain-specific description deficiency and zero-shot cross-domain adaptation. Contribution/Results: (1) We introduce the first training-free domain adaptation mechanism for time-series images; (2) we design a diversity-aware retrieval strategy, TADACap-diverse, to enhance domain coverage and semantic robustness; (3) we construct a cross-domain image–caption pair repository with efficient nearest-neighbor matching. Evaluated on multiple benchmarks, TADACap achieves state-of-the-art semantic accuracy while reducing annotation cost by over 60% and generalizing to unseen domains without fine-tuning.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Image and Video Retrieval
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

While image captioning has gained significant attention, the potential of captioning time-series images, prevalent in areas like finance and healthcare, remains largely untapped. Existing time-series captioning methods typically offer generic, domain-agnostic descriptions of time-series shapes and struggle to adapt to new domains without substantial retraining. To address these limitations, we introduce TADACap, a retrieval-based framework to generate domain-aware captions for time-series images, capable of adapting to new domains without retraining. Building on TADACap, we propose a novel retrieval strategy that retrieves diverse image-caption pairs from a target domain database, namely TADACap-diverse. We benchmarked TADACap-diverse against state-of-the-art methods and ablation variants. TADACap-diverse demonstrates comparable semantic accuracy while requiring significantly less annotation effort.

Problem

• Generating domain-aware captions for time-series images
• Adapting to new domains without retraining existing models
• Reducing annotation effort while maintaining semantic accuracy

Innovation

• Retrieval-based framework for domain-aware captions
• Adapts to new domains without retraining
• Novel retrieval strategy for diverse image-caption pairs