Who's Winning? Clarifying Estimands Based on Win Statistics in Cluster Randomized Trials

2026-02-11

AI Summary

This study addresses the discrepancy in treatment effect estimation based on win ratios in cluster randomized trials when cluster size is informative. It demonstrates that estimates targeting individual-pair versus cluster-pair win probabilities can yield fundamentally different—even contradictory—conclusions. The paper is the first to explicitly distinguish these two estimands, deriving consistent estimators for each and proposing a leave-one-cluster-out jackknife procedure to accurately estimate their variances. Theoretical analysis and finite-sample simulations show that the cluster-pair win ratio estimator is unbiased, whereas the individual-pair estimator may exhibit bias in small samples. By clarifying the necessity of precisely defining the target estimand, this work provides tailored inferential frameworks for both objectives, thereby preventing misleading interpretations of treatment effects.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Treatment effect estimands based on win statistics, including the win ratio, win odds, and win difference are increasingly popular targets for summarizing endpoints in clinical trials. Such win estimands offer an intuitive approach for prioritizing outcomes by clinical importance. The implementation and interpretation of win estimands is complicated in cluster randomized trials (CRTs), where researchers can target fundamentally different estimands on the individual-level or cluster-level. We numerically demonstrate that individual-pair and cluster-pair win estimands can substantially differ when cluster size is informative: where outcomes and/or treatment effects depend on cluster size. With such informative cluster sizes, individual-pair and cluster-pair win estimands can even yield opposite conclusions regarding treatment benefit. We describe consistent estimators for individual-pair and cluster-pair win estimands and propose a leave-one-cluster-out jackknife variance estimator for inference. Despite being consistent, our simulations highlight that some caution is needed when implementing individual-pair win estimators due to finite-sample bias. In contrast, cluster-pair win estimators are unbiased for their respective targets. Altogether, careful specification of the target estimand is essential when applying win estimators in CRTs. Failure to clearly define whether individual-pair or cluster-pair win estimands are of primary interest may result in answering a dramatically different question than intended.

Problem

• win statistics
• cluster randomized trials
• estimands
• informative cluster size
• treatment effect

Innovation

• win statistics
• cluster randomized trials
• estimands
• jackknife variance estimator
• informative cluster size