Differential Subgroup Discovery: Characterizing Where Two Populations Differ, and Why

2026-04-30

AI Summary

This study addresses the challenge of pronounced outcome disparities across groups in feature space by introducing the concept of “differential subgroups”—subpopulations that share similar covariates yet exhibit markedly divergent target outcomes. The work formally defines this notion and establishes discovery conditions amenable to causal interpretation, enabling fine-grained characterization of the sources underlying group-level differences. To efficiently identify such subgroups in tabular data, the authors propose DiffSub, a gradient-based algorithm that balances heterogeneity in outcomes with homogeneity in covariates to yield interpretable results. Empirical evaluations on synthetic data, clinical analyses, model error diagnostics, and treatment effect estimation demonstrate the method’s effectiveness in uncovering key covariate combinations and latent mechanisms driving population heterogeneity.

Abstract

We study the problem of understanding where two populations differ within a feature space, which we formalize in the concept of a differential subgroup: a subset of individuals from both populations who, despite sharing similar characteristics, exhibit exceptional differences in a target outcome. Differential subgroups reveal the regions of the feature space where population-level gaps are most pronounced and can help practitioners identify the covariate combinations that are structurally responsible for these differences, e.g.~in clinical analysis, model diagnostics, or treatment-effect studies. We introduce a general optimization objective for discovering differential subgroups and establish conditions under which the resulting subgroups admit a causal interpretation of population differences. We propose DiffSub, a gradient-based approach that discovers interpretable differential subgroups in tabular data. Across synthetic benchmarks, medical case studies, model-error analyses, and treatment-effect settings, DiffSub identifies informative subgroups that reveal where population differences arise and why.

Problem

• differential subgroup
• population difference
• feature space
• target outcome
• covariate combinations

Innovation

• differential subgroup discovery
• causal interpretation
• gradient-based optimization
• interpretable subgroups
• population differences