Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach

2026-05-07

AI Summary

This work proposes the first neuro-symbolic imitation learning framework that integrates privileged information to overcome the limitations of purely neural and purely symbolic approaches. While purely neural methods suffer from low sample efficiency and susceptibility to overfitting, and purely symbolic methods struggle with high-dimensional perceptual inputs, the proposed framework leverages privileged signals—such as human gaze during training—to guide neural networks in extracting task-relevant perceptual features. These features are then jointly optimized with a symbolic reasoning module. By synergistically combining the strengths of high-dimensional perception and symbolic generalization, the method achieves both high sample efficiency and significantly improved task performance, while also demonstrating enhanced cross-scenario generalization capabilities.

Abstract

Imitation learning is widely used for learning to act in complex environments. While pure neural-based methods handle high dimensional data effectively, they suffer from the requirement of large number of samples and are prone to overfitting. Pure symbolic approaches, while generalize well, do not handle high-dimensional data effectively. We propose a neurosymbolic approach that achieves the best of both worlds, i.e, handling high-dimensional data while achieving generalization. The key advantage of our approach is that it can effectively exploit additional privileged information that is available only during training (in our case, gaze data). Our empirical evaluations demonstrate the effectiveness, efficiency and the generalization capability of our proposed approach.

Problem

• Imitation Learning
• Neurosymbolic
• Privileged Information
• Generalization
• High-dimensional Data

Innovation

• neurosymbolic
• imitation learning
• privileged information
• gaze data
• generalization