Variable Selection for Additive Global Fréchet Regression

2025-09-17

AI Summary

This paper addresses variable selection in Fréchet regression over metric spaces—particularly for non-Euclidean data such as probability distributions and covariance matrices. We propose an interpretable additive Fréchet regression framework that models nonlinear covariate effects via reproducing kernel Hilbert spaces (RKHS), circumventing Euclidean structure assumptions through squared-distance transformation. To the best of our knowledge, this is the first work to incorporate folded concave penalties—including Elastic Net and SCAD—into the RKHS-based Fréchet regression setting, achieving both variable selection consistency and strong oracle properties. A locally linear approximation algorithm is developed to ensure computational feasibility. Theoretically, we establish estimation consistency and selection consistency under mild regularity conditions. Empirically, the method demonstrates high predictive accuracy, interpretability, and scalability on distribution-valued and matrix-valued datasets.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We present a novel framework for variable selection in Fréchet regression with responses in general metric spaces, a setting increasingly relevant for analyzing non-Euclidean data such as probability distributions and covariance matrices. Building on the concept of (weak) Fréchet conditional means, we develop an additive regression model that represents the metric-based discrepancy of the response as a sum of covariate-specific nonlinear functions in reproducing kernel Hilbert spaces (RKHS). To address the absence of linear structure in the response space, we transform the response via squared distances, enabling an interpretable and tractable additive decomposition. Variable selection is performed using Elastic Net regularization, extended to the RKHS setting, and further refined through a local linear approximation scheme that incorporates folded concave penalties such as the SCAD. We establish theoretical guarantees, including variable selection consistency and the strong oracle property, under minimal assumptions tailored to metric-space-valued responses. Simulations and applications to distributional and matrix-valued data demonstrate the scalability, interpretability, and practical effectiveness of the proposed approach. This work provides a principled foundation for statistical learning with random object data.

Problem

• Variable selection for non-Euclidean response data
• Additive regression modeling in metric spaces
• Elastic Net regularization for object responses

Innovation

• Additive regression model in RKHS
• Elastic Net regularization for variable selection
• Local linear approximation with SCAD penalties