Quantifying inconsistency in one-stage individual participant data meta-analyses of treatment-covariate interactions: a simulation study

2025-10-28

AI Summary

Quantifying heterogeneity in treatment–covariate interactions within one-stage individual participant data meta-analyses (IPD-MAs) remains challenging—particularly when subgroup sizes are imbalanced or covariates are continuous, rendering the conventional I² statistic inapplicable. Method: We extend the I² statistic to the one-stage IPD-MA framework by developing a flexible mixed-effects model accommodating imbalanced subgroups and continuous covariates, and conduct comprehensive simulation studies across varying numbers of trials, sample sizes, and heterogeneity levels. Contribution/Results: Our proposed I² estimator demonstrates excellent agreement with the standard two-stage approach—mean absolute differences range from −1.0 to 0.0 percentage points—exhibiting robust performance even under small-sample conditions. This advancement enhances the interpretability and comparability of one-stage IPD-MA results and establishes a novel, principled standard for assessing interaction-effect heterogeneity in IPD-MAs.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

It is recommended that measures of between-study effect heterogeneity be reported when conducting individual-participant data meta-analyses (IPD-MA). Methods exist to quantify inconsistency between trials via I^2 (the percentage of variation in the treatment effect due to between-study heterogeneity) when conducting two-stage IPD-MA, and when conducting one-stage IPD-MA with approximately equal numbers of treatment and control group participants. We extend formulae to estimate I^2 when investigating treatment-covariate interactions with unequal numbers of participants across subgroups and/or continuous covariates. A simulation study was conducted to assess the agreement in values of I^2 between those derived from two-stage models using traditional methods and those derived from equivalent one-stage models. Fourteen scenarios differed by the magnitude of between-trial heterogeneity, the number of trials, and the average number of participants in each trial. Bias and precision of I^2 were similar between the one- and two-stage models. The mean difference in I^2 between equivalent models ranged between -1.0 and 0.0 percentage points across scenarios. However, disparities were larger in simulated datasets with smaller samples sizes with up to 19.4 percentage points difference between models. Thus, the estimates of I^2 derived from these extended methods can be interpreted similarly to those from existing formulae for two-stage models.

Problem

• Extending I² estimation for treatment-covariate interactions in meta-analyses
• Assessing consistency between one-stage and two-stage IPD meta-analysis models
• Evaluating I² performance with unequal subgroup sizes and continuous covariates

Innovation

• Extended I² formula for one-stage meta-analysis
• Simulation study comparing one-stage and two-stage models
• Assessed bias and precision of heterogeneity estimates