An Integrated Framework for Contextual Personalized LLM-Based Food Recommendation

2025-04-25

AI Summary

Existing food recommendation systems suffer from fragmented component design, poor generalization under massive imbalanced data, and insufficient domain adaptation of general-purpose large language models (LLMs). To address these limitations, we propose Food-RLP—a novel context-aware food recommendation paradigm integrating multimodal food logging, geospatial modeling, and domain-specific LLM fine-tuning. We construct a multimedia food logging platform and the World Food Atlas to enable fine-grained, geography-aware food representation. Compared to generic RLP approaches, Food-RLP significantly improves recommendation accuracy and interpretability, enabling context-sensitive, personalized, and cross-regional dietary recommendations in real-world settings. Key innovations include: (1) a food-domain-specific architectural design; (2) joint geospatial–nutritional modeling; and (3) food-semantic-enhanced LLM adaptation mechanisms.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Personalized food recommendation systems (Food-RecSys) critically underperform due to fragmented component understanding and the failure of conventional machine learning with vast, imbalanced food data. While Large Language Models (LLMs) offer promise, current generic Recommendation as Language Processing (RLP) strategies lack the necessary specialization for the food domain's complexity. This thesis tackles these deficiencies by first identifying and analyzing the essential components for effective Food-RecSys. We introduce two key innovations: a multimedia food logging platform for rich contextual data acquisition and the World Food Atlas, enabling unique geolocation-based food analysis previously unavailable. Building on this foundation, we pioneer the Food Recommendation as Language Processing (F-RLP) framework - a novel, integrated approach specifically architected for the food domain. F-RLP leverages LLMs in a tailored manner, overcoming the limitations of generic models and providing a robust infrastructure for effective, contextual, and truly personalized food recommendations.

Problem

• Fragmented understanding and imbalanced data hinder food recommendation systems
• Generic LLM approaches lack specialization for complex food domain needs
• Absence of contextual frameworks for personalized food recommendations using LLMs

Innovation

• Multimedia food logging platform for contextual data
• World Food Atlas for geolocation-based food analysis
• Tailored F-RLP framework leveraging LLMs for food