DietGlance: Dietary Monitoring and Personalized Analysis at a Glance with Knowledge-Empowered AI Assistant

2025-02-03

AI Summary

Existing dietary monitoring tools exhibit significant limitations in complex food recognition and personalized nutritional analysis. This study proposes the first end-to-end dietary analysis framework integrating multimodal eating-behavior sensing (eye-tracking + low-resolution imaging) with knowledge-enhanced retrieval-augmented generation (RAG). The framework incorporates a lightweight food recognition model, a structured nutritional knowledge graph, and a privacy-preserving image processing mechanism. It enables unobtrusive daily dietary monitoring, fine-grained food-and-portion identification, and real-time personalized nutritional assessment. A short-term usability study (N=33) confirms high system acceptability; a four-week longitudinal study (N=16) demonstrates a 41% improvement in dietary logging accuracy and a 78% adoption rate of personalized recommendations—substantially overcoming traditional methods’ bottlenecks in parsing composite dishes and performing deep domain-knowledge reasoning.

Technology Category

• Computer Vision: Multi-modal Vision
• Humans and AI: Brain-Sensing and Analysis
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Growing awareness of wellness has prompted people to consider whether their dietary patterns align with their health and fitness goals. In response, researchers have introduced various wearable dietary monitoring systems and dietary assessment approaches. However, these solutions are either limited to identifying foods with simple ingredients or insufficient in providing analysis of individual dietary behaviors with domain-specific knowledge. In this paper, we present DietGlance, a system that automatically monitors dietary in daily routines and delivers personalized analysis from knowledge sources. DietGlance first detects ingestive episodes from multimodal inputs using eyeglasses, capturing privacy-preserving meal images of various dishes being consumed. Based on the inferred food items and consumed quantities from these images, DietGlance further provides nutritional analysis and personalized dietary suggestions, empowered by the retrieval augmentation generation module on a reliable nutrition library. A short-term user study (N=33) and a four-week longitudinal study (N=16) demonstrate the usability and effectiveness of DietGlance.

Problem

• Dietary Monitoring
• Complex Food Identification
• Personalized Nutrition Analysis

Innovation

• DietGlance
• Non-invasive Monitoring
• Personalized Nutrition Analysis