Active Inference in Contextual Multi-Armed Bandits for Autonomous Robotic Exploration

2024-08-07

arXiv.org

AI Summary

To address challenges in real-world mineral exploration—including dynamically shifting expert preferences, strong sensor noise, and highly uncertain environments—this paper proposes a neuro-inspired active inference framework that introduces expected free energy minimization to contextual multi-armed bandits (CMAB) for the first time. The method integrates hyperspectral remote sensing features (AVIRIS-NG) with online Bayesian learning to enable autonomous robotic sampling decisions and adaptive preference tracking. Compared to classical bandit algorithms, it achieves faster iterative convergence on real geological data—reducing required sampling rounds by ~35%—while demonstrating significantly improved robustness to annotation bias and observation noise, and maintaining higher selection accuracy under abrupt expert preference shifts. The core contribution is the establishment of the first active-inference-driven CMAB paradigm tailored to geological exploration, effectively bridging cognitive robotics and real-world resource discovery.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Autonomous selection of optimal options for data collection from multiple alternatives is challenging in uncertain environments. When secondary information about options is accessible, such problems can be framed as contextual multi-armed bandits (CMABs). Neuro-inspired active inference has gained interest for its ability to balance exploration and exploitation using the expected free energy objective function. Unlike previous studies that showed the effectiveness of active inference based strategy for CMABs using synthetic data, this study aims to apply active inference to realistic scenarios, using a simulated mineralogical survey site selection problem. Hyperspectral data from AVIRIS-NG at Cuprite, Nevada, serves as contextual information for predicting outcome probabilities, while geologists' mineral labels represent outcomes. Monte Carlo simulations assess the robustness of active inference against changing expert preferences. Results show that active inference requires fewer iterations than standard bandit approaches with real-world noisy and biased data, and performs better when outcome preferences vary online by adapting the selection strategy to align with expert shifts.

Problem

• Robot Autonomy
• Exploration vs Exploitation
• Data Collection in Uncertain Environments

Innovation

• Neuro-inspired Active Inference
• Spectral Data Analysis
• Expert Knowledge Integration