Method-of-Moments Inference for GLMs and Doubly Robust Functionals under Proportional Asymptotics

2024-08-12

AI Summary

This paper addresses the joint estimation of regression coefficients and signal-to-noise ratio (SNR) in high-dimensional generalized linear models (GLMs), and extends inference to doubly robust functionals—such as the average treatment effect—in observational studies. We propose a moment-based estimator that avoids nuisance function estimation and hyperparameter tuning. Theoretically, we establish consistent asymptotic normality (CAN) for the estimator under a proportional asymptotic regime—marking the first such result that dispenses with Gaussianity assumptions on covariates and does not require knowledge of the covariate covariance matrix. Methodologically, we introduce a sample covariance inverse matrix correction to ensure robust implementation under unknown covariance. Numerical experiments demonstrate that the proposed method significantly outperforms existing approaches in finite samples, particularly in high-dimensional sparse settings.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

In this paper, we consider the estimation of regression coefficients and signal-to-noise (SNR) ratio in high-dimensional Generalized Linear Models (GLMs), and explore their implications in inferring popular estimands such as average treatment effects in high-dimensional observational studies. Under the ``proportional asymptotic'' regime and Gaussian covariates with known (population) covariance $Sigma$, we derive Consistent and Asymptotically Normal (CAN) estimators of our targets of inference through a Method-of-Moments type of estimators that bypasses estimation of high dimensional nuisance functions and hyperparameter tuning altogether. Additionally, under non-Gaussian covariates, we demonstrate universality of our results under certain additional assumptions on the regression coefficients and $Sigma$. We also demonstrate that knowing $Sigma$ is not essential to our proposed methodology when the sample covariance matrix estimator is invertible. Finally, we complement our theoretical results with numerical experiments and comparisons with existing literature.

Problem

• Estimate regression coefficients in high-dimensional GLMs
• Infer average treatment effects in observational studies
• Develop CAN estimators without nuisance function estimation

Innovation

• Method-of-Moments estimators bypass nuisance functions
• Consistent estimators under Gaussian covariates
• Universality shown for non-Gaussian covariates