Adaptive Estimation and Inference in Semi-parametric Heterogeneous Clustered Multitask Learning via Neyman Orthogonality

2026-05-03

AI Summary

This work addresses the challenge in semi-parametric multi-task learning where task heterogeneity—induced by infinite-dimensional nuisance parameters—coexists with a shared latent clustering structure. The authors propose an adaptive fusion orthogonal estimator that integrates Neyman-orthogonal losses with a data-driven pairwise fusion penalty. By calibrating the penalty using task-level preliminary estimates and employing orthogonalization together with adaptive aggregation, the method effectively mitigates estimation errors from nuisance parameters. Theoretically, it recovers the true latent clusters with high probability and achieves asymptotic efficiency comparable to an oracle estimator that knows the ground-truth clustering, while attaining parameter convergence rates matched to cluster sizes. Empirical results on both simulated data and U.S. residential electricity consumption demonstrate substantial improvements over strong baselines and successfully uncover regional clusters in price elasticity.

Abstract

We study clustered multitask learning in a semiparametric setting where tasks share a latent cluster structure in their target parameters but exhibit heterogeneous, potentially infinite-dimensional nuisance components. Such heterogeneity poses a major challenge for existing multitask learning methods, which typically rely on aligned feature spaces or homogeneous task structures. To address this challenge, we propose an adaptive fused orthogonal estimator that integrates Neyman-orthogonal losses with data-driven pairwise fusion penalties. Our framework leverages task-specific pilot estimates to calibrate the fusion penalties and combines adaptive aggregation with orthogonalization to mitigate the impact of nuisance-parameter estimation error. Theoretically, we show that the proposed estimator achieves exact recovery of the latent clustering with high probability and attains pooled parametric convergence rates proportional to cluster size. Moreover, we establish asymptotic normality and show that, asymptotically, our estimator matches the performance of an oracle procedure that knows the true clustering in advance. Empirically, we show that the proposed method consistently outperforms strong baselines in various simulation setups. A real-world application to U.S. residential energy consumption demonstrates the effectiveness of our approach in uncovering meaningful regional clustering in electricity price elasticity, showcasing the efficacy of our method.

Problem

• clustered multitask learning
• semi-parametric
• heterogeneous nuisance components
• latent cluster structure
• Neyman orthogonality

Innovation

• Neyman orthogonality
• clustered multitask learning
• semi-parametric estimation
• adaptive fusion penalty
• heterogeneous nuisance components