ETZ: A Modeling Principle for Confirmability of Drug-Development Studies

2025-10-27

AI Summary

In drug development, the phase II-to-III transition success rate is only ~40%, and the phase III-to-approval rate falls below 50%, primarily due to insufficient confirmatory evidence and statistically unrigorous endpoint selection. To address this, we propose the Confidence-Based Quantification (CBQ) framework, which replaces conventional multiple testing with confidence-set inference to enhance confirmatory strength. We introduce the Endpoint Trajectory Zoning (ETZ) modeling principle—first of its kind—to disentangle intra- and inter-subject baseline and trajectory variability, thereby quantifying how three distinct sources of variability impact confirmatory power. Leveraging partitioning principles, we unify pivotal inference with Neyman-type confidence sets and extend Tukey’s Confidence-Based Assessment (CBA). A Shiny application enables practical implementation. CBQ enables experience-free statistical decision-making, ensuring both inferential validity and operational utility, thereby substantially improving phase III trial design rigor and regulatory approval probability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Transitioning from Phase 2 to Phase 3 in drug development, at a rate of $approx$40%, is the most stringent among phase transitions (Hay et al. (2014)). Yet, success rate at Phase 3 leading to approval is only $approx$50% (Arrowsmith (2011b)). To improve Confirmability, we propose a methodological shift: replacing multiple hypothesis testing with inference based on confidence sets, and substituting conventional power and sample size calculations with a Confidently Bounded Quantile (CBQ) framework. Our confidence set inferences to answer the questions of whether to transition to a Confirmatory study as well as what to designate as the endpoint in that study. Construction of our directed confidence sets follows the Partitioning Principle, taking the best of each of Pivoting and Neyman Confidence Set Construction. Rooted in Tukey's Confidently Bounded Allowance (CBA) (Tukey (1994a)), our proposed CBQ makes the transitioning decision following the Correct and Useful Inference principle in Hsu (1996). CBQ removes from "power" the probability of rejecting for wrong reasons, eliminating the need for informal discounting in power calculation that has existed in the biopharmaceutical industry. ETZ, the modeling principle proposed in Wang et al. (2025), quantifies the impact of three variability components on confirmability. In repeated-measures RCTs, it separates within-subject and between-subject variability, further dividing the latter into baseline and trajectory components. This enables informed investment decisions for the sponsors on targeting variability reduction to improve confirmability. A Shiny-based Confirmability App supports all computations.

Problem

• Improving drug development confirmability through methodological shifts
• Replacing multiple hypothesis testing with confidence set inference
• Quantifying variability impact on confirmability via ETZ modeling principle

Innovation

• Replaces multiple hypothesis testing with confidence set inference
• Introduces Confidently Bounded Quantile framework for power calculations
• Quantifies three variability components using ETZ modeling principle