Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsity

2026-07-01

AI Summary

This study addresses the challenge in traditional multi-task learning where heterogeneous output types—such as continuous and binary responses—lead to incomparable loss functions, hindering effective information sharing. To overcome this, the authors propose a multi-task transformation framework that unifies diverse response variables through unknown monotonic transformations. The approach integrates a shared first-layer deep neural network with group Lasso regularization, enabling joint modeling in high-dimensional settings. Notably, it is the first to combine monotonic transformations with a shared sparse structure, establishing a unified multi-task learning framework suitable for mixed output types. Theoretical guarantees for consistent variable selection are provided. Empirical results demonstrate that the method significantly outperforms existing approaches on both simulated data and real-world gene expression analysis, successfully identifying biologically meaningful shared predictors.

Abstract

Most existing multitask learning approaches are limited by their reliance on task-specific loss functions tailored to the scale and type of each outcome. When outcomes differ across tasks, these losses are generally not directly comparable, which makes it difficult to formulate a unified objective and may limit information sharing across tasks. We propose a multitask transformation framework in which task-specific responses may differ through unknown monotone transformations. Motivated by high-dimensional biological applications in which the predictor dimension may diverge with the sample size while only a common subset of predictors is informative, we consider shared sparsity across tasks. Under this framework, we estimate the target functions and identify important predictors by optimizing a smoothed rank-based criterion with a group-Lasso penalty, implemented through a multitask deep neural network with a shared first layer. We establish the nonasymptotic excess-risk bounds, and variable-selection consistency for the proposed estimator. Simulation studies show that the proposed method achieves competitive prediction and variable-selection performance compared with competing approaches. Analyses of gene-expression studies with continuous, binary, and mixed outcomes further illustrate that the proposed method improves prediction and identifies biologically meaningful shared predictors.

Problem

• multitask learning
• mixed-type outcomes
• shared sparsity
• non-comparable losses
• information sharing

Innovation

• multitask learning
• shared sparsity
• monotone transformation
• deep neural network
• group-Lasso