Crypto news

The Avon and Somerset Police, in conjunction with the Bristol City Council, was forced to discontinue the use of at least two machine learning algorithms designed to assess the risk of crimes against children. The reason: catastrophically low accuracy and complete opacity of the systems, which proved virtually impossible to audit independently.

A large-scale journalistic investigation, covering hundreds of pages of internal documents, revealed a shocking picture. At the root of the problems was the Think Family Database, launched in 2016. This digital behemoth collected information on nearly 500,000 residents of Bristol, merging police reports, data on housing status, mental health, school attendance, and even information about receiving free meals. All of this was done without the direct consent of citizens, under the guise of legal grounds for inter-agency data sharing.

Based on this database, 23 machine learning models were built, ranging from predicting burglaries to assessing the risk of becoming a victim of domestic violence. However, it was the models for child protection that failed. In addition to police and social data, they included anonymized information on 1,000 children who had been victims of crimes, sourced from the charity Barnardo's. The algorithms considered a child's status as needing help, chronic school absences, and mental health issues.

Ethical Collapse and the "Black Box"

As early as 2016, the police ethics committee warned of the risk of algorithmic bias, but its recommendations were ignored. Later, the independent consulting organization Social Finance delivered a damning assessment: the risk scoring was called the weakest link of the project. It turned out that the quality of the models sharply deteriorated when the police tried to scale the system to the entire region but failed to agree on data sharing with all local councils. As a result, the algorithms lost critically important social indicators and began to rely primarily on police data.

Employees of Bristol's city services openly complained that the system put the most vulnerable at risk. One worker described an egregious case: children who had recently become victims of crimes received a lower risk score than individuals involved in theft cases. Others refused to rely on the assessments due to the complete opacity of the methodology. An audit conducted by the company Eticas at the request of journalists confirmed the worst fears: most models had critically low positive predictive accuracy. For example, the model for identifying potential burglars showed an accuracy below 10% for over three years, meaning the system erroneously "stigmatized" more than 90% of people.

Lessons for PoliceAI

This incident is a grim omen ahead of the launch of the national PoliceAI center with a budget of £75 million. Ironically, the center is headed by former Avon and Somerset Police Chief Constable Andy Marsh, under whose leadership these controversial models were developed. The Bristol case clearly demonstrates that the main danger of AI in law enforcement is not the algorithms themselves, but the "garbage" at the input: poor-quality data, lack of transparency, and the impossibility of independent verification.

Expert opinion: The Bristol story is a classic example of how blind faith in technology, without proper control over data quality and ethical standards, leads to the opposite effect. A system designed to protect children itself became a source of injustice and distrust. If PoliceAI does not learn this lesson, millions of pounds of taxpayers' money risk being spent on creating an even larger and more dangerous "black box."