Geometric Integration for Neural Control Variates

2025-09-18

AI Summary

To address the challenge of high variance in Monte Carlo integration—where analytical integration is infeasible for nonlinear control variates—this paper proposes using a multi-layer perceptron (MLP) with continuous piecewise-linear activations as a control variate. Leveraging computational geometry techniques, we introduce the first exact analytical integration method for such MLPs over 2D domains. Specifically, the input domain is partitioned into convex polygonal subregions, on each of which the MLP reduces to an affine function, enabling closed-form integration. The resulting control variate remains unbiased and effectively captures the nonlinearity of the integrand. In light-transport rendering, our approach significantly reduces estimator variance, accelerates convergence, and improves rendering efficiency. Our core contribution is a geometric framework for exact MLP integration, overcoming the long-standing bottleneck of analytically integrating high-dimensional or nonlinear control variates.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Control variates are a variance-reduction technique for Monte Carlo integration. The principle involves approximating the integrand by a function that can be analytically integrated, and integrating using the Monte Carlo method only the residual difference between the integrand and the approximation, to obtain an unbiased estimate. Neural networks are universal approximators that could potentially be used as a control variate. However, the challenge lies in the analytic integration, which is not possible in general. In this manuscript, we study one of the simplest neural network models, the multilayered perceptron (MLP) with continuous piecewise linear activation functions, and its possible analytic integration. We propose an integration method based on integration domain subdivision, employing techniques from computational geometry to solve this problem in 2D. We demonstrate that an MLP can be used as a control variate in combination with our integration method, showing applications in the light transport simulation.

Problem

• Analytically integrating neural networks for control variates
• Solving integration challenge via computational geometry techniques
• Applying method to variance reduction in Monte Carlo

Innovation

• Neural networks as control variates
• Analytic integration via domain subdivision
• Computational geometry for 2D integration