Monte Carlo Rendering to Diffusion Curves with Differential BEM

2026-02-05

AI Summary

This work proposes the first end-to-end framework for directly generating editable vector representations—specifically diffusion curves—with explicit shading from noisy Monte Carlo renderings, a task that existing methods struggle to accomplish. The approach formulates diffusion curve generation as a stochastic optimization problem over the geometry and color of curve control points, solved efficiently via the Levenberg–Marquardt algorithm. A key innovation is the integration of a differential boundary element method (Differential BEM) to reconstruct colors and compute gradients without requiring a converged image. This enables robust handling of noisy inputs, supports complex geometric configurations such as intersecting curves, and necessitates only a single matrix factorization. Experiments demonstrate high-quality, robust vector reconstructions across diverse scenes, significantly extending the applicability of diffusion curves to real-world rendering data.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

We present a method for generating vector graphics, in the form of diffusion curves, directly from noisy samples produced by a Monte Carlo renderer. While generating raster images from 3D geometry via Monte Carlo raytracing is commonplace, there is no corresponding practical approach for robustly and directly extracting editable vector images with shading information from 3D geometry. To fill this gap, we formulate the problem as a stochastic optimization problem over the space of geometries and colors of diffusion curve handles, and solve it with the Levenberg-Marquardt algorithm. At the core of our method is a novel differential boundary element method (BEM) framework that reconstructs colors from diffusion curve handles and computes gradients with respect to their parameters, requiring the expensive matrix factorization only once at the beginning of the optimization. Unlike triangulation-based techniques that require a clean domain decomposition, our method is robust to geometrically challenging scenarios, such as intersecting diffusion curves, and to color noise in the target image, enabling the direct use of noisy Monte Carlo samples without requiring a converged, error-free input image. We demonstrate the robustness and broad applicability of our approach across several test cases. Finally, we highlight several open questions raised by our work, which spans both theory and applications.

Problem

• vector graphics
• diffusion curves
• Monte Carlo rendering
• stochastic optimization
• boundary element method

Innovation

• diffusion curves
• differential BEM
• Monte Carlo rendering
• vector graphics reconstruction
• stochastic optimization