The DeepXube Software Package for Solving Pathfinding Problems with Learned Heuristic Functions and Search

2026-03-24

AI Summary

This work proposes an automated framework that integrates deep reinforcement learning with heuristic search to address the reliance on manual expertise in designing heuristic functions for path planning. The approach combines limited-horizon Bellman updates, hindsight experience replay, and batched heuristic search algorithms—such as batch weighted A*, Q*, and beam search—and leverages answer set programming to flexibly specify task objectives. Built upon a multiple-inheritance object-oriented architecture, the system simplifies domain modeling and supports automatic parallelization across GPU and CPU for both training and inference. The project is open-sourced, offering efficient command-line utilities and visualization tools, and demonstrates significant improvements in heuristic learning efficiency and solution performance across multiple benchmarks.

Technology Category

• Search and Optimization: Heuristic Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

DeepXube is a free and open-source Python package and command-line tool that seeks to automate the solution of pathfinding problems by using machine learning to learn heuristic functions that guide heuristic search algorithms tailored to deep neural networks (DNNs). DeepXube is comprised of the latest advances in deep reinforcement learning, heuristic search, and formal logic for solving pathfinding problems. This includes limited-horizon Bellman-based learning, hindsight experience replay, batched heuristic search, and specifying goals with answer-set programming. A robust multiple-inheritance structure simplifies the definition of pathfinding domains and the generation of training data. Training heuristic functions is made efficient through the automatic parallelization of the generation of training data across central processing units (CPUs) and reinforcement learning updates across graphics processing units (GPUs). Pathfinding algorithms that take advantage of the parallelism of GPUs and DNN architectures, such as batch weighted A* and Q* search and beam search are easily employed to solve pathfinding problems through command-line arguments. Finally, several convenient features for visualization, code profiling, and progress monitoring during training and solving are available. The GitHub repository is publicly available at https://github.com/forestagostinelli/deepxube.

Problem

• pathfinding
• heuristic functions
• deep neural networks
• heuristic search
• automated planning

Innovation

• learned heuristic functions
• deep reinforcement learning
• batched heuristic search
• hindsight experience replay
• GPU parallelization