Ordinal regression for meta-analysis of test accuracy: a flexible approach for utilising all threshold data

2025-05-29

AI Summary

Existing meta-analytic methods for diagnostic test accuracy—such as bivariate and network meta-analysis—are limited by threshold-wise modeling, leading to information loss when studies report incomplete or heterogeneous thresholds. Multi-threshold models either rely on restrictive assumptions or fail to jointly synthesize sensitivity and specificity across thresholds. To address these limitations, we propose a unified Bayesian framework based on ordinal regression, introducing the induced Dirichlet prior for the first time to enable full-threshold data integration, threshold-specific inference, and flexible fixed- or random-effects modeling. Implemented via Stan for MCMC inference, the method supports sROC curve estimation, meta-regression, and covariate-adjusted analyses. In both real-world anxiety/depression screening data and simulation studies, it achieves significantly improved estimation accuracy. An open-source R package, MetaOrdDTA, provides automated analysis and visualization tools.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Standard methods for meta-analysis and network-meta-analysis of test accuracy do not fully utilise available evidence, as they analyse thresholds separately, resulting in a loss of data unless every study reports all thresholds - which rarely occurs. Furthermore, previously proposed"multiple threshold"models introduce different problems: making overly restrictive assumptions, or failing to provide summary sensitivity and specificity estimates across thresholds. To address this, we proposed a series of ordinal regression-based models, representing a natural extension of established frameworks. Our approach offers notable advantages: (i) complete data utilisation: rather than discarding information like standard methods, we incorporate all threshold data; (ii) threshold-specific inference: by providing summary accuracy estimates across thresholds, our models deliver critical information for clinical decision-making; (iii) enhanced flexibility: unlike previous"multiple thresholds"approaches, our methodology imposes fewer assumptions, leading to better accuracy estimates; (iv) our models use an induced-Dirichlet framework, allowing for either fixed-effects or random-effects cutpoint parameters, whilst also allowing for intuitive cutpoint priors. Our (ongoing) simulation study - based on real-world anxiety and depression screening data - demonstrates notably better accuracy estimates than previous approaches, even when the number of categories is high. Furthermore, we implemented these models in a user-friendly R package - MetaOrdDTA (https://github.com/CerulloE1996/MetaOrdDTA). The package uses Stan and produces MCMC summaries, sROC plots with credible/prediction regions, and meta-regression. Overall, our approach establishes a more comprehensive framework for synthesising test accuracy data, better serving systematic reviewers and clinical decision-makers.

Problem

• Standard methods lose data by analyzing test accuracy thresholds separately
• Existing multiple threshold models have restrictive assumptions or lack summary estimates
• Need flexible ordinal regression to utilize all threshold data effectively

Innovation

• Ordinal regression models for all threshold data
• Flexible induced-Dirichlet framework for cutpoints
• User-friendly R package with Stan implementation