A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity

2026-08-30

AI Summary

本文提出一种深度潜在变量框架，综合处理缺失数据、测量误差和人群异质性问题，采用分层树路由变分自编码器和校准去噪方法。

Abstract

Missing data, measurement error, and population heterogeneity are pervasive challenges in analyzing data arising from modern observational studies and machine learning applications. Although these problems frequently coexist and interact, they are often treated separately in existing works. We propose a unified probabilistic framework that jointly addresses these issues utilizing deep latent variable representation. The proposed method integrates a novel hierarchical tree-routed variational autoencoder with pattern-aware latent representations and calibration-based denoising. The framework accommodates missing data mechanisms, including MCAR, MAR, and MNAR, while simultaneously learning subgroup-specific and globally shared latent structure. The introduced reconvergent routing mechanism enables selective parameters to be shared across related subpopulations, which offers flexibility as well as improved statistical efficiency. Simulation studies demonstrate substantial improvements over existing deep generative imputation approaches under complex heterogeneous missingness and measurement-error settings. The proposed framework provides a principled approach for learning from noisy and incomplete data in modern healthcare and other high-dimensional applications.

Problem

• missing data
• measurement error
• population heterogeneity

Innovation

• deep latent variable
• hierarchical tree-routed variational autoencoder
• pattern-aware latent representation
• reconvergent routing mechanism