Search papers, labs, and topics across Lattice.
This paper derives a theoretical "Likelihood Ratio Wall" demonstrating fundamental limits on the achievable precision of pretrial risk assessment tools when predicting rare violent re-offense. It shows that even with respectable overall performance, these tools will inevitably have low positive predictive value (PPV) for rare events like violent re-arrest (2-5%). The paper also proves a "Surveillance Ceiling," where over-policing inflates risk factors and further reduces achievable precision, even with equal offense rates.
Expect pretrial risk assessment tools to be wrong more often than right when flagging someone as "high risk" for rare violent re-offense, regardless of recalibration efforts.
Pretrial risk assessment tools are used on over one million U.S. defendants each year, yet their use for predicting rare violent re-offense faces a basic statistical barrier. We derive a universal precision bound -- the Likelihood Ratio Wall -- showing that when violent re-arrest rates are low (2-5%), achieving even a 50% hit rate among people labeled"high risk"(positive predictive value, or PPV) would require tools far more discriminative than current instruments appear to be. For rare outcomes, a tool can have respectable-looking performance metrics and still be wrong most of the time it flags someone as"high risk for violence."We show that post-hoc score recalibration cannot solve this problem because it does not improve the tool's underlying ability to separate true positives from false positives. We further prove a Surveillance Ceiling: when over-policing inflates recorded"risk factors"among those who would not re-offend, the maximum achievable precision is structurally lower for over-policed groups, even at equal offense rates. We translate these results into the Number Needed to Detain (how many people must be detained to prevent one violent offense), and propose that risk reports should communicate this uncertainty explicitly. Our findings suggest that for rare violent outcomes, debates about fairness metrics alone are incomplete: under current data regimes, the available features may not support high-confidence individualized detention decisions.
