OLR-WAA: Adaptive and Drift-Resilient Online Regression with Dynamic Weighted Averaging

2025-12-14

AI Summary

Online regression over non-stationary data streams suffers from performance degradation due to concept drift, while conventional methods with fixed hyperparameters lack adaptability. Method: We propose a hyperparameter-free online regression framework that jointly integrates a drift-aware mechanism and dynamic weighted averaging to automatically balance stability and adaptivity. A high-confidence conservative update strategy is introduced, incorporating exponential weighted moving averages, real-time drift detection and quantification, adaptive dynamic weight adjustment, and incremental model updates. Contribution/Results: Experiments demonstrate that our method matches batch-learning approaches on stationary data and significantly outperforms state-of-the-art online algorithms under concept drift—achieving higher R² scores and faster convergence. It effectively bridges the performance gap between static and dynamic scenarios, offering robust, self-adaptive regression without manual tuning.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Real-world datasets frequently exhibit evolving data distributions, reflecting temporal variations and underlying shifts. Overlooking this phenomenon, known as concept drift, can substantially degrade the predictive performance of the model. Furthermore, the presence of hyperparameters in online models exacerbates this issue, as these parameters are typically fixed and lack the flexibility to dynamically adjust to evolving data. This paper introduces "OLR-WAA: An Adaptive and Drift-Resilient Online Regression with Dynamic Weighted Average", a hyperparameter-free model designed to tackle the challenges of non-stationary data streams and enable effective, continuous adaptation. The objective is to strike a balance between model stability and adaptability. OLR-WAA incrementally updates its base model by integrating incoming data streams, utilizing an exponentially weighted moving average. It further introduces a unique optimization mechanism that dynamically detects concept drift, quantifies its magnitude, and adjusts the model based on real-time data characteristics. Rigorous evaluations show that it matches batch regression performance in static settings and consistently outperforms or rivals state-of-the-art online models, confirming its effectiveness. Concept drift datasets reveal a performance gap that OLR-WAA effectively bridges, setting it apart from other online models. In addition, the model effectively handles confidence-based scenarios through a conservative update strategy that prioritizes stable, high-confidence data points. Notably, OLR-WAA converges rapidly, consistently yielding higher R2 values compared to other online models.

Problem

• Addresses concept drift in online regression with adaptive, hyperparameter-free design
• Balances model stability and adaptability for evolving data streams
• Dynamically detects drift and adjusts using real-time data characteristics

Innovation

• Hyperparameter-free model for non-stationary data streams
• Dynamic weighted averaging with concept drift detection
• Conservative update strategy prioritizing high-confidence data