EmoLat: Text-driven Image Sentiment Transfer via Emotion Latent Space

2026-01-17

AI Summary

This work addresses the challenge of cross-modal semantic and visual affect alignment in text-driven fine-grained image emotion transfer by proposing a novel paradigm. The approach constructs an emotion semantic graph to model complex relationships among emotions, objects, and visual attributes, and introduces EmoLat—a latent emotional space—and EmoSpace Set, a large-scale annotated dataset. High-fidelity, text-controllable emotional editing is achieved through cross-modal joint embedding, adversarial regularization, and multi-objective optimization. Experimental results demonstrate that the proposed method significantly outperforms existing techniques on EmoSpace Set, achieving state-of-the-art performance in both quantitative metrics and visual emotion transfer quality.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We propose EmoLat, a novel emotion latent space that enables fine-grained, text-driven image sentiment transfer by modeling cross-modal correlations between textual semantics and visual emotion features. Within EmoLat, an emotion semantic graph is constructed to capture the relational structure among emotions, objects, and visual attributes. To enhance the discriminability and transferability of emotion representations, we employ adversarial regularization, aligning the latent emotion distributions across modalities. Building upon EmoLat, a cross-modal sentiment transfer framework is proposed to manipulate image sentiment via joint embedding of text and EmoLat features. The network is optimized using a multi-objective loss incorporating semantic consistency, emotion alignment, and adversarial regularization. To support effective modeling, we construct EmoSpace Set, a large-scale benchmark dataset comprising images with dense annotations on emotions, object semantics, and visual attributes. Extensive experiments on EmoSpace Set demonstrate that our approach significantly outperforms existing state-of-the-art methods in both quantitative metrics and qualitative transfer fidelity, establishing a new paradigm for controllable image sentiment editing guided by textual input. The EmoSpace Set and all the code are available at http://github.com/JingVIPLab/EmoLat.

Problem

• image sentiment transfer
• text-driven editing
• emotion representation
• cross-modal correlation
• controllable image editing

Innovation

• emotion latent space
• text-driven image sentiment transfer
• cross-modal alignment
• adversarial regularization
• emotion semantic graph