Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection

2026-02-28

AI Summary

This work addresses the limitations of unsupervised graph-level out-of-distribution (OOD) detection, where reliance solely on in-distribution data leads to insufficient characterization of the feature space and ambiguous decision boundaries. To overcome these challenges, the authors propose PGOS, a policy-guided outlier synthesis framework that introduces, for the first time, a learnable reinforcement learning exploration policy into graph OOD detection. By deploying an agent that actively explores low-density regions in a structured latent space, PGOS adaptively generates high-quality pseudo-OOD graphs to refine decision boundaries. Integrating graph neural networks, reinforcement learning, latent space modeling, and graph decoding techniques, PGOS establishes an end-to-end anomaly synthesis pipeline. The method achieves state-of-the-art performance across multiple graph OOD and anomaly detection benchmarks, significantly enhancing detection robustness.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Detecting out-of-distribution (OOD) graphs is crucial for ensuring the safety and reliability of Graph Neural Networks. In unsupervised graph-level OOD detection, models are typically trained using only in-distribution (ID) data, resulting in incomplete feature space characterization and weak decision boundaries. Although synthesizing outliers offers a promising solution, existing approaches rely on fixed, non-adaptive sampling heuristics (e.g., distance- or density-based), limiting their ability to explore informative OOD regions. We propose a Policy-Guided Outlier Synthesis (PGOS) framework that replaces static heuristics with a learned exploration strategy. Specifically, PGOS trains a reinforcement learning agent to navigate low-density regions in a structured latent space and sample representations that most effectively refine the OOD decision boundary. These representations are then decoded into high-quality pseudo-OOD graphs to improve detector robustness. Extensive experiments demonstrate that PGOS achieves state-of-the-art performance on multiple graph OOD and anomaly detection benchmarks.

Problem

• out-of-distribution detection
• graph neural networks
• unsupervised learning
• outlier synthesis
• decision boundary

Innovation

• Policy-Guided Outlier Synthesis
• Graph Out-of-Distribution Detection
• Reinforcement Learning
• Latent Space Exploration
• Pseudo-OOD Graph Generation