A Dependent Feature Allocation Model Based on Random Fields

2025-12-19

AI Summary

Traditional nonparametric feature allocation models struggle to explicitly capture dependency structures, covariate effects, and temporal-spatial dynamics. To address this, we propose a Bayesian nonparametric framework grounded in Gaussian Markov random fields (GMRFs), the first to embed GMRFs into the logit layer of the feature paintbox, thereby jointly modeling local dependencies among latent variables. Our approach innovatively integrates a data-adaptive k-nearest-neighbor graph structure with an Ornstein–Uhlenbeck process, enabling principled modeling of spatiotemporal evolution and low-rank feature correlations. The resulting method achieves both high-dimensional scalability and interpretability. Evaluated on multi-drug combination clinical data, it successfully infers patients’ latent health states, significantly improving accuracy in dependency-aware feature allocation and fidelity in covariate-response modeling.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce a flexible framework for modeling dependent feature allocations. Our approach addresses limitations in traditional nonparametric methods by directly modeling the logit-probability surface of the feature paintbox, enabling the explicit incorporation of covariates and complex but tractable dependence structures. The core of our model is a Gaussian Markov Random Field (GMRF), which we use to robustly decompose the latent field, separating a structural component based on the baseline covariates from intrinsic, unstructured heterogeneity. This structure is not a rigid grid but a sparse k-nearest neighbors graph derived from the latent geometry in the data, ensuring high-dimensional tractability. We extend this framework to a dynamic spatio-temporal process, allowing item effects to evolve via an Ornstein-Uhlenbeck process. Feature correlations are captured using a low-rank factorization of their joint prior. We demonstrate our model's utility by applying it to a polypharmacy dataset, successfully inferring latent health conditions from patient drug profiles.

Problem

• Models dependent feature allocations with covariates and complex dependencies
• Uses Gaussian Markov Random Field to separate structural and intrinsic heterogeneity
• Extends to dynamic spatio-temporal processes for evolving item effects

Innovation

• Gaussian Markov Random Field for feature allocation
• Sparse k-nearest neighbors graph for tractability
• Dynamic spatio-temporal Ornstein-Uhlenbeck process evolution