Abstract
Predictive policing tools have cycled through controversy for over a decade, yet their deployment has continued to expand across municipal police departments in the United States, the United Kingdom, and several EU member states. The current generation of these tools - including place-based risk scoring systems and network analysis platforms that flag individuals for proactive contact - raises accountability questions that existing legal frameworks are ill-equipped to answer. Who bears liability when an algorithmic risk score contributes to a wrongful arrest? What disclosure obligations apply to the vendors who build these systems? This article examines the accountability gap at the intersection of predictive policing and AI governance, and evaluates recent legislative and institutional attempts to close it.
The Accountability Gap in Current Deployments
Most predictive policing contracts are structured between software vendors and municipal law enforcement agencies under terms that limit public disclosure. Vendors including PredPol (rebranded as Geolitica), ShotSpotter, and various social network analysis tools have successfully argued that their model architectures constitute trade secrets, shielding them from the kind of independent auditing that would be routine for, say, a credit scoring algorithm subject to FCRA oversight. The result is a class of high-stakes decision-support tools that receive less regulatory scrutiny than the models banks use to approve credit cards.
This opacity matters because the feedback loops embedded in place-based prediction are well-documented. A model trained on historical arrest data predicts elevated risk in neighborhoods that were over-policed historically; deploying officers there generates more arrests, which reinforces the model’s predictions. ProPublica’s 2016 analysis of the COMPAS recidivism tool exposed an early version of this dynamic; more recent work from the Stanford Computational Policy Lab in 2024 found similar feedback amplification in place-based systems operating in three US cities.
Recent Legislative Attempts
California’s ACLU-backed AB 1545, which would have required audits and public disclosure for predictive tools used in law enforcement, stalled in committee in 2024 but has been re-introduced with broader coalition support in 2025. At the federal level, the Algorithmic Accountability Act has been reintroduced repeatedly without advancing. More traction has come at the city level: New York City’s Local Law 49 (2021) established an algorithm review task force, and Portland, Oregon banned predictive policing tools outright in 2020 - a ban that remains in effect.
The EU AI Act, which entered its phased implementation in 2024, classifies “AI systems used by law enforcement for individual risk assessments” as high-risk under Annex III. This imposes conformity assessments, logging requirements, and human oversight mandates on EU-deployed tools, but enforcement mechanisms are still being operationalized by member state market surveillance authorities.
What Meaningful Accountability Requires
Technical accountability for predictive policing tools needs to address three things that current frameworks largely skip. First, training data provenance: a model trained on arrest records rather than crime incidence inherits the enforcement patterns of prior policing, a distinction that audit standards need to make explicit. Second, deployment drift monitoring: performance metrics must be tracked in operational settings, not just at validation time, because neighborhood demographics and policing priorities shift. Third, vendor liability: municipal contracts should include enforceable accuracy and disparity thresholds with financial penalties, rather than best-effort language. Without these structural requirements, accountability frameworks remain aspirational.