Empirical Power Analysis of a Statistical Test to Quantify Gerrymandering

2025-01-10

AI Summary

This study evaluates the practical statistical power of Chikina et al.’s hypothesis test for detecting partisan gerrymandering in North Carolina’s congressional districting plans. Method: Leveraging geospatial presidential election data from 2012 and 2016, we generate multiple skewed districting plans and conduct large-scale MCMC sampling coupled with outlier analysis to empirically assess test power across partisan affiliations, election years, and chain lengths. Contribution/Results: The test exhibits robustness across parties, years, and sampling durations; however, its statistical power depends critically on the choice of bias metric—not on party affiliation, year, or chain length. This work provides the first computational benchmark for the evidentiary reliability of statistical gerrymandering tests in judicial settings, establishing metric design as the primary determinant of test validity and offering essential methodological grounding for courts evaluating such evidence.

Technology Category

• Machine Learning: Ethics, Bias, and Fairness
• Game Theory and Economic Paradigms: Social Choice / Voting
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Gerrymandering is a pervasive problem within the US political system. In the past decade, methods based on Markov Chain Monte Carlo (MCMC) sampling and statistical outlier tests have been proposed to quantify gerrymandering and were used as evidence in several high-profile legal cases. We perform an empirical power analysis of one such hypothesis test from Chikina et al (2020). We generate a family of biased North Carolina congressional district maps using the 2012 and 2016 presidential elections and assess under which conditions the outlier test fails to flag them at the specified Type I error level. The power of the outlier test is found to be relatively stable across political parties, election years, lengths of the MCMC chain and effect sizes. The main effect on the power of the test is shown to be the choice of the bias metric. This is the first work that computationally verifies the power of statistical tests used in gerrymandering cases.

Problem

• Statistical Tests
• Gerrymandering Detection
• Electoral Districting

Innovation

• Statistical Testing
• Gerrymandering Detection
• Bias Measurement