Uncovering All Highly Credible Binary Treatment Hierarchy Questions in Network Meta-Analysis

2025-10-09

AI Summary

In network meta-analysis (NMA), multi-treatment comparisons lead to combinatorial explosion in binary treatment hierarchy questions (e.g., “A is better than B”, “A is most likely best”), and conventional approaches require pre-specification of hypotheses, hindering systematic identification of all high-confidence conclusions. Method: We propose the first scalable algorithmic framework that formally defines and automatically enumerates six canonical types of binary treatment hierarchy questions—without prior specification—using empirically calibrated probability thresholds to generate, deduplicate, and filter high-confidence answers. The method integrates NMA ranking probabilities, pairwise comparisons, and systematic enumeration, implemented as an open-source R package enabling end-to-end automation. Contribution/Results: Applied to real-world diabetes and depression datasets, our approach robustly identifies multiple clinically meaningful treatment hierarchies, substantially enhancing interpretability and decision support for complex treatment networks. It is broadly applicable to evidence-based, multi-option decision-making contexts.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In recent years, there has been growing research interest in addressing treatment hierarchy questions within network meta-analysis (NMA). In NMAs involving many treatments, the number of possible hierarchy questions becomes prohibitively large. To manage this complexity, previous work has recommended pre-selecting specific hierarchy questions of interest (e.g., ``among options A, B, C, D, E, do treatments A and B have the two best effects in terms of improving outcome X?") and calculating the empirical probabilities of the answers being true given the data. In contrast, we propose an efficient and scalable algorithmic approach that eliminates the need for pre-specification by systematically generating a comprehensive catalog of highly credible treatment hierarchy questions, specifically, those with empirical probabilities exceeding a chosen threshold (e.g., 95%). This enables decision-makers to extract all meaningful insights supported by the data. An additional algorithm trims redundant insights from the output to facilitate interpretation. We define and address six broad types of binary hierarchy questions (i.e., those with true/false answers), covering standard hierarchy questions answered using existing ranking metrics - pairwise comparisons and (cumulative) ranking probabilities - as well as many other complex hierarchy questions. We have implemented our methods in an R package and illustrate their application using real NMA datasets on diabetes and depression interventions. Beyond NMA, our approach is relevant to any decision problem concerning three or more treatment options.

Problem

• Systematically generates highly credible treatment hierarchy questions in network meta-analysis
• Eliminates need for pre-specification by creating comprehensive catalog of meaningful insights
• Addresses six types of binary hierarchy questions using scalable algorithmic approach

Innovation

• Automated generation of credible treatment hierarchy questions
• Systematic elimination of redundant insights for clarity
• Scalable algorithm applicable beyond network meta-analysis