US Police Tech ‘Gold Rush’ Raises Bias and Transparency Concerns

A surge in artificial intelligence sales to US police departments has drawn sharp criticism over transparency and potential bias, according to a report from The Verge. The trend was prominently displayed at the International Association of Chiefs of Police (IACP) Technology Conference in Fort Worth, Texas, where vendors pitched a wide array of automated tools, including facial recognition, predictive analytics, and report-writing software. The event underscored a “gold rush” in the police-tech industrial complex, driven by major market players such as Axon Enterprise and Motorola Solutions, alongside a growing number of startups seeking to capitalise on law enforcement’s need for efficiency.

Axon Enterprise has emerged as a dominant force in this expanding market, reporting a 700 per cent year-on-year increase in AI product revenue. Between the first quarter of last year and the same period this year, subscriptions to its “AI Era Plan” rose by 140 per cent. The company, which acquired surveillance firm Fusus in early 2024 to launch its Axon Fusus real-time crime centre, argues that its tools alleviate the administrative burden on officers. A 2024 Axon study found that police officers spend approximately 40 per cent of their shifts writing reports, a statistic used to justify the adoption of AI-driven narrative generation tools like Draft One.

Despite these efficiency claims, critics warn that the reliance on proprietary “black box” algorithms risks eroding accountability and perpetuating historical biases. Legal experts and law enforcement officials note that the absence of comprehensive federal oversight leaves departments vulnerable to adopting unproven technologies. Concerns are particularly acute regarding predictive policing, a sector that has faced significant backlash after earlier systems were shown to over-predict crime in Black and Latino neighbourhoods. In 2024, four US Democratic senators urged the Department of Justice to halt grants for such programs, citing evidence that these algorithms reinforced racial disparities rather than mitigating them.

Transparency issues have also been highlighted by incidents involving AI-generated reports. In one notable case, Axon’s Draft One tool produced a report stating an officer in Utah had “morphed into a frog,” a hallucination attributed to audio from the Disney movie The Princess and the Frog playing at the scene. Originally, the tool was designed not to save original copies of generated narratives to avoid “disclosure headaches,” a feature that drew criticism for obscuring the line between human and machine input. Axon updated the software in December 2024 to allow departments to retain unedited AI-generated narratives, responding to growing demands for clarity in legal proceedings.

The commercialisation of these technologies has attracted significant outside investment, with approximately one-quarter of attendees at the IACP showroom floor identified as representatives from equity firms. This influx of capital has intensified competition among vendors to monopolise the police technology stack, from data collection to strategic decision-making. As police departments face budget constraints and staffing shortages, they are increasingly pressured to adopt these tools, often without clear industry standards. Experts caution that without rigorous scrutiny, the integration of AI into policing could further distance law enforcement from community trust and due process.