DAO-GP Drift Aware Online Non-Linear Regression Gaussian-Process

2025-12-09

AI Summary

Real-world data streams frequently exhibit concept drift, degrading the performance of online Gaussian process (GP) regression models with fixed hyperparameters. Existing approaches lack drift awareness, principled adaptive hyperparameter updating, theoretically grounded data forgetting mechanisms, and efficient sparse approximations. This paper proposes the first fully adaptive, hyperparameter-free sparse online GP regression framework. It integrates sliding-window-based drift detection with confidence-driven dynamic data decay, jointly optimizes inducing points and kernel hyperparameters via Bayesian online adaptive annealing, and ensures inference consistency by avoiding data peeking. The method achieves state-of-the-art performance across abrupt, gradual, incremental, and stationary drift scenarios, while maintaining memory efficiency and enabling real-time uncertainty quantification.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Real-world datasets often exhibit temporal dynamics characterized by evolving data distributions. Disregarding this phenomenon, commonly referred to as concept drift, can significantly diminish a model's predictive accuracy. Furthermore, the presence of hyperparameters in online models exacerbates this issue. These parameters are typically fixed and cannot be dynamically adjusted by the user in response to the evolving data distribution. Gaussian Process (GP) models offer powerful non-parametric regression capabilities with uncertainty quantification, making them ideal for modeling complex data relationships in an online setting. However, conventional online GP methods face several critical limitations, including a lack of drift-awareness, reliance on fixed hyperparameters, vulnerability to data snooping, absence of a principled decay mechanism, and memory inefficiencies. In response, we propose DAO-GP (Drift-Aware Online Gaussian Process), a novel, fully adaptive, hyperparameter-free, decayed, and sparse non-linear regression model. DAO-GP features a built-in drift detection and adaptation mechanism that dynamically adjusts model behavior based on the severity of drift. Extensive empirical evaluations confirm DAO-GP's robustness across stationary conditions, diverse drift types (abrupt, incremental, gradual), and varied data characteristics. Analyses demonstrate its dynamic adaptation, efficient in-memory and decay-based management, and evolving inducing points. Compared with state-of-the-art parametric and non-parametric models, DAO-GP consistently achieves superior or competitive performance, establishing it as a drift-resilient solution for online non-linear regression.

Problem

• Addresses concept drift in online non-linear regression models
• Eliminates reliance on fixed hyperparameters for dynamic data adaptation
• Improves memory efficiency and drift detection in Gaussian Processes

Innovation

• Drift detection and adaptation mechanism
• Hyperparameter-free and fully adaptive design
• Decayed and sparse memory management