HAVEN: Hierarchical Adversary-aware Visibility-Enabled Navigation with Cover Utilization using Deep Transformer Q-Networks

2025-11-29

AI Summary

In partially observable environments, autonomous navigation faces challenges including limited field-of-view, severe occlusions, and difficulty in predicting dynamic obstacles. To address these, this paper proposes a hierarchical visibility-aware navigation framework. At the high level, a Deep Transformer Q-Network (DTQN) integrates odometry, goal direction, obstacle distance, and explicit visibility cues to select task-driven sub-goals; at the low level, a potential-field controller executes local trajectory tracking. A novel visibility-aware candidate generation mechanism is introduced, employing a mask-expose penalty to encourage proactive exploitation of occluders, while temporal memory enhances prediction of dynamic obstacles. Evaluated in both 2D simulation and a 3D Unity-ROS environment, the method significantly improves task success rate, minimum safety distance, and arrival efficiency. Results demonstrate robustness, cross-environment transferability, and practical deployability.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

Autonomous navigation in partially observable environments requires agents to reason beyond immediate sensor input, exploit occlusion, and ensure safety while progressing toward a goal. These challenges arise in many robotics domains, from urban driving and warehouse automation to defense and surveillance. Classical path planning approaches and memoryless reinforcement learning often fail under limited fields of view (FoVs) and occlusions, committing to unsafe or inefficient maneuvers. We propose a hierarchical navigation framework that integrates a Deep Transformer Q-Network (DTQN) as a high-level subgoal selector with a modular low-level controller for waypoint execution. The DTQN consumes short histories of task-aware features, encoding odometry, goal direction, obstacle proximity, and visibility cues, and outputs Q-values to rank candidate subgoals. Visibility-aware candidate generation introduces masking and exposure penalties, rewarding the use of cover and anticipatory safety. A low-level potential field controller then tracks the selected subgoal, ensuring smooth short-horizon obstacle avoidance. We validate our approach in 2D simulation and extend it directly to a 3D Unity-ROS environment by projecting point-cloud perception into the same feature schema, enabling transfer without architectural changes. Results show consistent improvements over classical planners and RL baselines in success rate, safety margins, and time to goal, with ablations confirming the value of temporal memory and visibility-aware candidate design. These findings highlight a generalizable framework for safe navigation under uncertainty, with broad relevance across robotic platforms.

Problem

• Enables safe navigation in partially observable environments with occlusions
• Integrates hierarchical planning with visibility-aware subgoal selection using transformers
• Transfers from 2D simulation to 3D robotic platforms without architectural changes

Innovation

• Hierarchical navigation with Deep Transformer Q-Network subgoal selector
• Visibility-aware candidate generation using masking and exposure penalties
• Feature projection enabling 2D to 3D transfer without architectural changes