Hybrid least squares for learning functions from highly noisy data

2025-07-02

AI Summary

Estimating conditional expectations under high-noise regimes remains challenging, and conventional least-squares function approximation suffers from poor robustness to noise. To address this, we propose a hybrid least-squares method that integrates Christoffel-weighted sampling with optimal experimental design. Our approach jointly optimizes sampling strategy and noise suppression, achieving—under strong noise—the first simultaneous near-optimality in both sampling and approximation. We further extend the framework to settings with convex constraints and adaptive random subspaces. Theoretical analysis establishes improved sample complexity and accelerated convergence rates. Numerical experiments on synthetic benchmarks and financial stochastic simulations demonstrate that the proposed method significantly outperforms state-of-the-art baselines in approximation accuracy, stability, and computational efficiency.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Motivated by the need for efficient estimation of conditional expectations, we consider a least-squares function approximation problem with heavily polluted data. Existing methods that are powerful in the small noise regime are suboptimal when large noise is present. We propose a hybrid approach that combines Christoffel sampling with certain types of optimal experimental design to address this issue. We show that the proposed algorithm enjoys appropriate optimality properties for both sample point generation and noise mollification, leading to improved computational efficiency and sample complexity compared to existing methods. We also extend the algorithm to convex-constrained settings with similar theoretical guarantees. When the target function is defined as the expectation of a random field, we extend our approach to leverage adaptive random subspaces and establish results on the approximation capacity of the adaptive procedure. Our theoretical findings are supported by numerical studies on both synthetic data and on a more challenging stochastic simulation problem in computational finance.

Problem

• Estimating functions from highly noisy data efficiently
• Improving accuracy in large noise scenarios
• Extending method to convex-constrained settings

Innovation

• Hybrid least squares for noisy data
• Christoffel sampling with optimal design
• Adaptive random subspaces for expectations