Search papers, labs, and topics across Lattice.
This paper critiques the prevailing focus on predictive accuracy in automated decision systems (ADS) and highlights the need for an integrated framework that emphasizes intervention-oriented approaches. By analyzing real-world case studies across various domains, the authors reveal that the introduction of predictions alters organizational workflows and decision-making processes, necessitating a reevaluation of ADS design and deployment. The key result is a proposed shift in priorities that enhances our understanding of the societal and organizational impacts of ADS beyond mere prediction metrics.
Predictions alone can mislead; understanding how they reshape decision-making is crucial for effective automated systems.
Automated decision systems (ADS) leverage predictions about individual future outcomes to inform consequential decision-making in organizational settings. Across various settings - including criminal pretrial release, clinical triage, student support, and more - it is often assumed that improved predictive accuracy is the priority consideration in determining better downstream outcomes upon the deployment of ADS. In practice, real-world case studies reveal that this is far from the case: introducing individual predictions into decision-making modifies organizational workflows, assessment, and decision-making processes in ways that require a complete re-consideration of our approach to the design, evaluation, and deployment of ADS. As a result, this Perspective develops an integrated framework for studying ADS in social systems, shifting current priorities from a purely prediction-based paradigm towards an intervention-oriented view that accounts for real-world conditions. Our aim is to improve our understanding of ADS and more meaningfully anticipate its downstream societal and organizational consequences.
