🤖 AI Summary
This work addresses a critical limitation in current automated decision-making systems, which prioritize predictive accuracy while neglecting their systemic impact on organizational workflows and decision processes, thereby compromising downstream societal outcomes. To overcome this “prediction-first” paradigm, the study proposes an intervention-oriented integrative framework that systematically incorporates organizational dynamics and intervention mechanisms into the design and evaluation of automated decision systems. By synthesizing insights from social science theory and decision systems analysis, the research establishes an interdisciplinary pathway tailored to real-world deployment contexts. This approach offers a novel paradigm for understanding and optimizing the societal consequences of automated decision-making in high-stakes domains such as criminal justice, medical triage, and educational support.
📝 Abstract
Automated decision systems (ADS) leverage predictions about individual future outcomes to inform consequential decision-making in organizational settings. Across various settings - including criminal pretrial release, clinical triage, student support, and more - it is often assumed that improved predictive accuracy is the priority consideration in determining better downstream outcomes upon the deployment of ADS. In practice, real-world case studies reveal that this is far from the case: introducing individual predictions into decision-making modifies organizational workflows, assessment, and decision-making processes in ways that require a complete re-consideration of our approach to the design, evaluation, and deployment of ADS. As a result, this Perspective develops an integrated framework for studying ADS in social systems, shifting current priorities from a purely prediction-based paradigm towards an intervention-oriented view that accounts for real-world conditions. Our aim is to improve our understanding of ADS and more meaningfully anticipate its downstream societal and organizational consequences.