Gradient Flow for Finding E-optimal Designs

2026-01-20

AI Summary

This work addresses the non-smooth optimization problem of E-optimal design in regression models by introducing, for the first time, a Wasserstein gradient flow framework that transforms the infinite-dimensional optimization into a computable finite-particle system. To handle the non-differentiability and multiplicity of the minimum eigenvalue, the authors employ semidefinite programming (SDP) relaxation to precisely characterize the steepest ascent direction and accommodate constrained designs. Theoretical analysis provides convergence guarantees for the information matrix under empirical measure approximation. Experimental results demonstrate that the proposed method efficiently generates high-precision E-optimal designs across a range of linear and nonlinear regression models, exhibiting significantly superior scalability and performance compared to existing heuristic algorithms.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We investigate the use of Wasserstein gradient flows for finding an $E$-optimal design for a regression model. Unlike the commonly used $D$- and $L$-optimality criteria, the $E$-criterion finds a design that maximizes the smallest eigenvalue of the information matrix, and so it is a non-differentiable criterion unless the minimum eigenvalue has geometric multiplicity equals to one. Such maximin design problems abound in statistical applications and present unique theoretical and computational challenges. Building on the differential structure of the $2$-Wasserstein space, we derive explicit formulas for the Wasserstein gradient of the $E$-optimality criterion in the simple-eigenvalue case. For higher multiplicities, we propose a Wasserstein steepest ascent direction and show that it can be computed exactly via a semidefinite programming (SDP) relaxation. We develop particle approximations that connect infinite-dimensional flows with finite-dimensional optimization, and provide approximation guarantees for empirical measures. Our framework extends naturally to constrained designs via projected Wasserstein gradient flows. Numerical experiments demonstrate that the proposed methods successfully recover $E$-optimal designs for both linear and nonlinear regression models, with competitive accuracy and scalability compared to existing heuristic approaches. This work highlights the potential of optimal transport-based dynamics as a unifying tool for studying challenging optimal design problems.

Problem

• E-optimal design
• maximin optimization
• non-differentiable criterion
• optimal experimental design
• information matrix

Innovation

• Wasserstein gradient flow
• E-optimal design
• semidefinite programming
• optimal transport
• particle approximation