False Discovery Rate and Localizing Power

2024-01-07

AI Summary

In neuroimaging, conventional two-tailed false discovery rate (FDR) correction controls the overall false discovery rate but fails to ensure reliable directional inference (e.g., “activation” vs. “deactivation”), leading to directional error rates as high as 100%. This work first systematically characterizes the mechanism underlying directional error inflation in two-tailed FDR procedures. We propose two novel statistical paradigms: (1) direction-separated FDR correction, which applies independent FDR control to positive and negative test statistics, and (2) asymmetric thresholding, wherein distinct critical values are employed for positive and negative effects. Through theoretical analysis, extensive simulations on synthetic data, and empirical validation using real fMRI datasets, we demonstrate that direction-separated FDR strictly bounds the directional false discovery rate at ≤5%, substantially outperforming standard two-tailed p-value approaches. Our framework advances statistical thresholding methodology in neuroimaging and provides a principled foundation for implementing user-configurable asymmetric thresholds in neuroimaging software.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

False discovery rate (FDR) is commonly used for correction for multiple testing in neuroimaging studies. However, when using two-tailed tests, making directional inferences about the results can lead to vastly inflated error rate, even approaching 100% in some cases. This happens because FDR only provides weak control over the error rate, meaning that the proportion of error is guaranteed only globally over all tests, not within subsets, such as among those in only one or another direction. Here we consider and evaluate different strategies for FDR control with two-tailed tests, using both synthetic and real imaging data. Approaches that separate the tests by direction of the hypothesis test, or by the direction of the resulting test statistic, more properly control the directional error rate and preserve FDR benefits, albeit with a doubled risk of errors under complete absence of signal. Strategies that combine tests in both directions, or that use simple two-tailed p-values, can lead to invalid directional conclusions, even if these tests remain globally valid. To enable valid thresholding for directional inference, we suggest that imaging software should allow the possibility that the user sets asymmetrical thresholds for the two sides of the statistical map. While FDR continues to be a valid, powerful procedure for multiple testing correction, care is needed when making directional inferences for two-tailed tests, or more broadly, when making any localized inference.

Problem

• FDR control fails in directional inferences with two-tailed tests
• Strategies needed to properly control directional error rates
• Imaging software should allow asymmetric thresholds for valid inference

Innovation

• Directional FDR control for two-tailed tests
• Asymmetrical thresholds for statistical maps
• Separate hypothesis tests by direction