The Proof is in the Almond Cookies

2025-01-03

AI Summary

This work addresses the challenge of enabling robotic cooking assistants to comprehend and execute unstructured natural-language recipes—particularly for elderly, disabled, and professional kitchen users. We propose a narrative-driven recipe modeling framework that integrates domain-specific ontologies, mental simulation, and semantic parsing to achieve zero-shot coreference resolution, task-level understanding evaluation, and language-agnostic cross-lingual recipe annotation. Unlike conventional instruction-parsing paradigms, our approach formalizes recipes as causally and temporally grounded narratives. Evaluated on the “Almond New Mooncake Cookies” recipe, the method demonstrates effectiveness in plan optimization, ambiguity resolution, and interpretable understanding assessment. Results show significant improvement in robots’ deep comprehension of culinary intent, enabling robust, explainable, and generalizable execution across diverse linguistic and contextual settings.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper presents a case study on how to process cooking recipes (and more generally, how-to instructions) in a way that makes it possible for a robot or artificial cooking assistant to support human chefs in the kitchen. Such AI assistants would be of great benefit to society, as they can help to sustain the autonomy of aging adults or people with a physical impairment, or they may reduce the stress in a professional kitchen. We propose a novel approach to computational recipe understanding that mimics the human sense-making process, which is narrative-based. Using an English recipe for almond crescent cookies as illustration, we show how recipes can be modelled as rich narrative structures by integrating various knowledge sources such as language processing, ontologies, and mental simulation. We show how such narrative structures can be used for (a) dealing with the challenges of recipe language, such as zero anaphora, (b) optimizing a robot's planning process, (c) measuring how well an AI system understands its current tasks, and (d) allowing recipe annotations to become language-independent.

Problem

• Recipe Understanding
• Language Processing
• Kitchen Assistance

Innovation

• Recipe Understanding
• Cognitive Robotics
• Language-Agnostic Processing