DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain

2026-04-11

AI Summary

Existing vision-language models struggle with fine-grained food recognition and accurate nutritional reasoning in dietary contexts due to reliance on coarse-grained categories, single-view images, and unreliable metadata. To address these limitations, this work introduces DiningBench, a hierarchical multi-view benchmark comprising multi-perspective images and expert-verified nutritional data for 3,021 dishes. DiningBench incorporates same-menu “hard negatives” and establishes a three-tier cognitive evaluation framework encompassing fine-grained classification, nutrient estimation, and visual question answering. Systematic evaluation of 29 state-of-the-art models reveals five prevalent failure modes in fine-grained discrimination and nutritional inference. The study further demonstrates that leveraging multi-view inputs and chain-of-thought reasoning can partially mitigate these shortcomings, offering clear directions for future research.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Recent advancements in Vision-Language Models (VLMs) have revolutionized general visual understanding. However, their application in the food domain remains constrained by benchmarks that rely on coarse-grained categories, single-view imagery, and inaccurate metadata. To bridge this gap, we introduce DiningBench, a hierarchical, multi-view benchmark designed to evaluate VLMs across three levels of cognitive complexity: Fine-Grained Classification, Nutrition Estimation, and Visual Question Answering. Unlike previous datasets, DiningBench comprises 3,021 distinct dishes with an average of 5.27 images per entry, incorporating fine-grained "hard" negatives from identical menus and rigorous, verification-based nutritional data. We conduct an extensive evaluation of 29 state-of-the-art open-source and proprietary models. Our experiments reveal that while current VLMs excel at general reasoning, they struggle significantly with fine-grained visual discrimination and precise nutritional reasoning. Furthermore, we systematically investigate the impact of multi-view inputs and Chain-of-Thought reasoning, identifying five primary failure modes. DiningBench serves as a challenging testbed to drive the next generation of food-centric VLM research. All codes are released in https://github.com/meituan/DiningBench.

Problem

• Vision-Language Models
• food domain
• fine-grained classification
• nutritional reasoning
• multi-view benchmark

Innovation

• multi-view benchmark
• fine-grained food recognition
• nutritional reasoning
• vision-language models
• hierarchical evaluation