Generative Flexible Latent Structure Regression (GFLSR) model

2025-08-06

AI Summary

Existing linear continuous latent variable methods are largely black-box algorithms, lacking generative modeling, statistical inference capabilities, and parameter identifiability. Method: We propose Generative-PLS—a generative, flexible latent structure regression model—that establishes the first inferential latent variable regression framework. It abandons restrictive distributional assumptions to enable non-probabilistic statistical inference; unifies classical approaches including PLS, CCA, and SEM under a single representation; develops a parameter convergence theory with residual diagnostics; and introduces a recursive-structure-driven customized bootstrap algorithm to quantify both parameter and predictive uncertainty. Results: Extensive simulations and empirical studies demonstrate that Generative-PLS achieves strong interpretability, robustness, and principled uncertainty quantification—surpassing conventional latent variable methods in both theoretical rigor and practical applicability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Latent structure methods, specifically linear continuous latent structure methods, are a type of fundamental statistical learning strategy. They are widely used for dimension reduction, regression and prediction, in the fields of chemometrics, economics, social science and etc. However, due to the lack of model inference, generative form, and unidentifiable parameters, most of these methods are always used as an algorithm, instead of a model. This paper proposed a Generative Flexible Latent Structure Regression (GFLSR) model structure to address this problem. Moreover, we show that most linear continuous latent variable methods can be represented under the proposed framework. The recursive structure allows potential model inference and residual analysis. Then, the traditional Partial Least Squares (PLS) is focused; we show that the PLS can be specialised in the proposed model structure, named Generative-PLS. With a model structure, we analyse the convergence of the parameters and the latent variables. Under additional distribution assumptions, we show that the proposed model structure can lead to model inference without solving the probabilistic model. Additionally, we proposed a novel bootstrap algorithm that enables uncertainty on parameters and on prediction for new datasets. A simulation study and a Real-world dataset are used to verify the proposed Generative-PLS model structure. Although the traditional PLS is a special case, this proposed GFLSRM structure leads to a potential inference structure for all the linear continuous latent variable methods.

Problem

• Addresses lack of model inference in latent structure methods
• Proposes generative framework for linear latent variable models
• Enables parameter convergence and uncertainty analysis

Innovation

• Generative Flexible Latent Structure Regression model
• Recursive structure enables model inference
• Novel bootstrap algorithm for uncertainty estimation