What Images Cannot Say: Language-Guided Olfactory Representation Learning

2026-07-07

AI Summary

This work addresses the challenge of aligning electronic nose (e-nose) signals with visual content, which is hindered by the absence of explicit olfactory cues in images. To bridge this modality gap, the authors propose SCENT, a multimodal framework that leverages vision-language models to generate scene descriptions encompassing objects, environmental context, and implicit odor semantics, thereby establishing a semantic link between vision and olfaction. SCENT further introduces a language-guided latent decomposition mechanism that, for the first time, disentangles object-specific odors from ambient background scents, yielding interpretable olfactory representations. By mapping e-nose signals into a shared embedding space aligned with vision-text pairs, SCENT substantially outperforms vision-only baselines on the New York Smells dataset and achieves state-of-the-art performance in both odor-to-image and odor-to-text retrieval tasks.

Abstract

Images tell us what a scene looks like, but rarely what it would feel like to be there. While recent datasets pair visual scenes with electronic-nose measurements, aligning smell signals with images remains challenging because many olfactory cues arise from contextual environmental factors that are not directly visible in pixels. We introduce SCENT, a multimodal framework that uses language guidance as a semantic bridge between vision and olfaction. Our approach leverages Vision-Language Models (VLMs) to generate scene descriptors capturing objects, environmental context, and plausible ambient smell cues suggested by the visual scene. These descriptors provide semantic guidance for learning olfactory representations. We train a smell encoder that maps electronic-nose signals into a shared embedding space aligned with both visual and textual representations, and introduce a languageguided latent decomposition that separates object-specific odors from contextual environmental contributions. Experiments on the New York Smells dataset demonstrate that SCENT significantly improves crossmodal retrieval compared to vision-only baselines, achieving state-of-theart performance on smell-to-image and smell-to-text retrieval tasks. In addition, our framework produces interpretable olfactory representations that enable the disentanglement of complex smell mixtures. Our results reveal the importance of contextual semantic information for grounding olfactory perception in multimodal learning and pave the way for future research in this area.

Problem

• olfactory representation
• vision-olfaction alignment
• contextual semantics
• multimodal learning
• electronic-nose

Innovation

• language-guided learning
• multimodal representation
• olfactory perception
• vision-language models
• latent decomposition