UK Police AI Rollout Proceeds Despite Audit Revealing Unreliable Predictive Models
A WIRED investigation exposes significant flaws in regional predictive analytics, including burglary and child exploitation models with precision rates below 10 per cent, even as the College of Policing leads a national push for artificial intelligence in law enforcement.
An investigation by WIRED, in partnership with Liberty Investigates, the Bristol Cable, and Lighthouse Reports, has revealed that predictive analytics systems developed by Avon and Somerset Police and Bristol City Council suffered from poor performance and a lack of transparency. The inquiry focused on the Think Family Database and the Offender Management App, which utilised risk-scoring models for child sexual exploitation (CSE), child criminal exploitation (CCE), and burglary. Independent audits, including one by AI auditing firm Eticas, found that most of the 13 models reviewed suffered from low precision, with some operating at less than 10 per cent accuracy for over three years.
Consequently, several models were quietly abandoned after staff deemed them unfit for operational use, with no public record kept of these decisions. At least two risk-scoring models (CSE and CCE) were abandoned after Bristol City Council staff deemed them unreliable, following a switch to using only police data which reduced accuracy. An independent audit by Eticas found that most of the 13 models reviewed produced low precision scores, with a burglary prediction model operating with a precision rating lower than 10 per cent for over three years.
John Pegram, a local accountability group leader, confirmed in early 2024 that he was on the Offender Management App after filing a public records request; he is now mounting a legal challenge to have his details removed and the program scrapped. The Centre for Data Ethics and Innovation raised "ethical tensions" in 2021 regarding the use of "legal gateways" rather than consent for data collection. Police data disclosed to WIRED comprised more than 36,000 model performance scores, which an independent analyst described as showing "genuinely poor predictive performance."
Despite these findings, the UK government is expanding AI adoption in policing through the new £75 million PoliceAI initiative, led by the College of Policing under former Avon and Somerset chief constable Andy Marsh. The initiative aims to roll out AI tools to 43 police forces across England and Wales, with Policing Minister Sarah Jones describing it as "the future of policing." Marsh has stated that effective AI should be "injected like heroin" to speed up British police work, with his organisation examining around 100 currently deployed AI tools.
The contrast between the local failures and the national expansion highlights a growing divergence in how predictive policing is managed. While Avon and Somerset Police struggled with data sharing and model accuracy, the College of Policing is moving forward with standardising AI deployment. Critics, including Rob Procter of the University of Warwick, argue that the lack of documentation and transparency in Bristol illustrates the critical need for public debate before such tools are widely adopted.
