Food Image Generation on Multi-Noun Categories

2025-12-09

AI Summary

Multi-noun food categories (e.g., “egg noodles”) are frequently misparsed as independent lexical units in text-to-image generation, leading to semantic distortion and spatial misalignment. To address this, we propose FoCULR (Food Category Understanding and Layout Refinement), the first framework to embed a food knowledge graph into the text encoder. It explicitly models semantic dependencies among multi-noun constituents via a relation-decoupling module and enforces structural fidelity through a novel spatial layout constraint loss. Built upon diffusion models, FoCULR achieves significant improvements on benchmarks including UEC-256: FID and CLIP-Score improve markedly; accuracy for multi-noun food categories increases by 32.7%; and spurious ingredient generation decreases by 58.4%. This work establishes an interpretable, controllable, and domain-adaptive paradigm for fine-grained food image synthesis.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Computer Vision: Language and Vision
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Generating realistic food images for categories with multiple nouns is surprisingly challenging. For instance, the prompt "egg noodle" may result in images that incorrectly contain both eggs and noodles as separate entities. Multi-noun food categories are common in real-world datasets and account for a large portion of entries in benchmarks such as UEC-256. These compound names often cause generative models to misinterpret the semantics, producing unintended ingredients or objects. This is due to insufficient multi-noun category related knowledge in the text encoder and misinterpretation of multi-noun relationships, leading to incorrect spatial layouts. To overcome these challenges, we propose FoCULR (Food Category Understanding and Layout Refinement) which incorporates food domain knowledge and introduces core concepts early in the generation process. Experimental results demonstrate that the integration of these techniques improves image generation performance in the food domain.

Problem

• Generates realistic images for multi-noun food categories
• Addresses misinterpretation of compound names in generative models
• Improves spatial layouts by incorporating food domain knowledge

Innovation

• Incorporates food domain knowledge early
• Refines spatial layouts for multi-noun categories
• Enhances text encoder understanding of relationships