Out-of-Distribution Detection in Molecular Complexes via Diffusion Models for Irregular Graphs

2025-12-20

AI Summary

Out-of-distribution (OOD) detection for 3D irregular molecular graphs—characterized by continuous geometric coordinates, discrete atomic types, and permutation-invariant structure—remains challenging in protein–ligand complex analysis. Method: We propose the first unsupervised diffusion-based OOD detection framework tailored to molecular complexes. It (i) introduces a unified graph diffusion process jointly modeling continuous coordinates and discrete atom types; (ii) employs an analytically differentiable posterior mean interpolation scheme to estimate discrete feature scores; and (iii) constructs sample-level log-likelihoods via probability flow ODEs (PF-ODEs), augmented with multi-scale trajectory dynamics statistics—including path curvature, flow stiffness, and field instability—to yield self-consistent typicality scores and OOD criteria. Results: On a rigorous protein-family-level OOD benchmark, our method significantly improves OOD identification. PF-ODE likelihood strongly correlates with prediction error, and incorporating trajectory statistics markedly enhances OOD separation over likelihood-only baselines, providing a reliable evaluation tool for label-free 3D geometric deep learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Predictive machine learning models generally excel on in-distribution data, but their performance degrades on out-of-distribution (OOD) inputs. Reliable deployment therefore requires robust OOD detection, yet this is particularly challenging for irregular 3D graphs that combine continuous geometry with categorical identities and are unordered by construction. Here, we present a probabilistic OOD detection framework for complex 3D graph data built on a diffusion model that learns a density of the training distribution in a fully unsupervised manner. A key ingredient we introduce is a unified continuous diffusion over both 3D coordinates and discrete features: categorical identities are embedded in a continuous space and trained with cross-entropy, while the corresponding diffusion score is obtained analytically via posterior-mean interpolation from predicted class probabilities. This yields a single self-consistent probability-flow ODE (PF-ODE) that produces per-sample log-likelihoods, providing a principled typicality score for distribution shift. We validate the approach on protein-ligand complexes and construct strict OOD datasets by withholding entire protein families from training. PF-ODE likelihoods identify held-out families as OOD and correlate strongly with prediction errors of an independent binding-affinity model (GEMS), enabling a priori reliability estimates on new complexes. Beyond scalar likelihoods, we show that multi-scale PF-ODE trajectory statistics - including path tortuosity, flow stiffness, and vector-field instability - provide complementary OOD information. Modeling the joint distribution of these trajectory features yields a practical, high-sensitivity detector that improves separation over likelihood-only baselines, offering a label-free OOD quantification workflow for geometric deep learning.

Problem

• Detects out-of-distribution molecular complexes using diffusion models
• Handles irregular 3D graphs with continuous geometry and categorical features
• Provides likelihood and trajectory-based OOD detection for protein-ligand data

Innovation

• Diffusion model for 3D graph OOD detection
• Unified continuous diffusion over coordinates and features
• Multi-scale trajectory statistics enhance OOD sensitivity