Comparison of Data Reduction Criteria for Online Gaussian Processes

2025-08-14

AI Summary

To address the computational intractability arising from unbounded data accumulation in online Gaussian process (GP) regression, this paper systematically compares multiple data reduction criteria and proposes a unified evaluation framework along with a novel adaptive acceptance criterion to efficiently identify and discard redundant data points. The method integrates sparse GP modeling, online learning mechanisms, and multi-strategy reduction, and is validated on both regression and dynamic system identification tasks. Key contributions include: (1) a quantitative evaluation metric that jointly balances computational complexity and information preservation; and (2) an adaptive acceptance criterion that significantly improves data selection quality. Experiments on benchmark functions and real-world streaming datasets demonstrate that the proposed approach reduces long-term computational overhead by 40–65% while maintaining predictive accuracy—yielding less than a 3% increase in normalized mean squared error (NMSE). The method provides a reproducible, deployable guideline for practical online GP applications.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Data Mining & Knowledge Management: Data Compression

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Gaussian Processes (GPs) are widely used for regression and system identification due to their flexibility and ability to quantify uncertainty. However, their computational complexity limits their applicability to small datasets. Moreover in a streaming scenario, more and more datapoints accumulate which is intractable even for Sparse GPs. Online GPs aim to alleviate this problem by e.g. defining a maximum budget of datapoints and removing redundant datapoints. This work provides a unified comparison of several reduction criteria, analyzing both their computational complexity and reduction behavior. The criteria are evaluated on benchmark functions and real-world datasets, including dynamic system identification tasks. Additionally, acceptance criteria are proposed to further filter out redundant datapoints. This work yields practical guidelines for choosing a suitable criterion for an online GP algorithm.

Problem

• Compare data reduction criteria for Online Gaussian Processes
• Analyze computational complexity and reduction behavior
• Propose guidelines for selecting suitable reduction criteria

Innovation

• Compares data reduction criteria for Online GPs
• Analyzes computational complexity and reduction behavior
• Proposes acceptance criteria to filter redundant datapoints