EgoCogNav: Cognition-aware Human Egocentric Navigation

2025-11-15

AI Summary

Existing embodied navigation models largely neglect human cognitive processes under partial observability. This work proposes EgoCogNav, a multimodal framework that— for the first time—models perceptual uncertainty as a latent state, explicitly linking it to cognitively grounded behaviors such as pausing, hesitating, and backtracking. By jointly encoding egocentric visual features with multi-source sensory cues, the model co-predicts navigation trajectories and head motion. To support this, we introduce and release CEN, the first real-world dataset (6 hours) annotated with fine-grained cognitive behaviors. Experiments demonstrate that EgoCogNav accurately captures uncertainty dynamics strongly correlated with human behavior and exhibits strong zero-shot generalization to unseen environments. This work establishes a novel, cognition-aware paradigm for embodied navigation modeling.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Modeling the cognitive and experiential factors of human navigation is central to deepening our understanding of human-environment interaction and to enabling safe social navigation and effective assistive wayfinding. Most existing methods focus on forecasting motions in fully observed scenes and often neglect human factors that capture how people feel and respond to space. To address this gap, We propose EgoCogNav, a multimodal egocentric navigation framework that predicts perceived path uncertainty as a latent state and jointly forecasts trajectories and head motion by fusing scene features with sensory cues. To facilitate research in the field, we introduce the Cognition-aware Egocentric Navigation (CEN) dataset consisting 6 hours of real-world egocentric recordings capturing diverse navigation behaviors in real-world scenarios. Experiments show that EgoCogNav learns the perceived uncertainty that highly correlates with human-like behaviors such as scanning, hesitation, and backtracking while generalizing to unseen environments.

Problem

• Modeling cognitive factors in human navigation
• Predicting perceived path uncertainty from sensory cues
• Forecasting trajectories with human-like behavioral patterns

Innovation

• Models perceived path uncertainty as latent state
• Fuses scene features with sensory cues
• Forecasts trajectories and head motion jointly