Written evidence submitted by the Human Rights, Big Data and Technology Project

Executive summary

• This submission highlights opportunities and risks associated with the adoption of new technologies by police in England and Wales.
• Opportunities include the implementation of data driven technologies which improve police efficiency by facilitating the prediction and pre-emption of crime that has yet to take place.
• Risks include the ineffective and/or discriminatory use of data driven technologies due to low quality or biased data, as well as privacy issues related to bulk and indiscriminate data collection.
• Recommendations include the creation of oversight and regulatory mechanisms to assess the implementation of data driven technologies and to address associated risks.

Introduction

1. This submission is made by the Human Rights, Big Data and Technology Project (‘the Project’), funded by the Economic and Social Research Council and based at the University of Essex’s Human Rights Centre.[1]

2. The Project analyses the opportunities and risks presented by the use of big data and associated technologies from a human rights perspective. Drawing on the expertise of its interdisciplinary researchers and partner organisations, the Project considers how human rights approaches are adapted to meet the rapidly evolving technological landscape. The Project also brings together State, industry and United Nations officials, and academics in the fields of human rights, big data and associated technologies to assess existing regulatory responses and whether reforms are needed to protect human rights.

3. The Project is grateful to the Home Affairs Committee for the opportunity to provide this submission on ‘Policing for the future: changing demands and new challenges’.

Policing for the Future: Opportunities and Risks Associated with the Adoption of Data Driven Technologies

4. Among the changing demands and challenges facing police in England and Wales are the opportunities and risks associated with the adoption of new technologies. Based on our ongoing research and review of the academic literature, this submission offers insights into some of the specific opportunities and risks concerning the police’s adoption of ‘data driven technologies,’ which aid the bulk collection, storage, and analysis of data about persons, places, and activities.

5. Data driven technologies can improve the efficiency of police work by offering law enforcement agencies methods of analysing crime data and generating predictions about crime that is yet to take place. For example, police can use data driven technology to make decisions regarding the most-effective allocation of scarce police resources. However, existing research[2] indicates a series of risks that must be considered when adopting data driven technologies. This submission highlights three risks associated with data driven technologies, focusing on: the quality of the input data, the potential for discriminatory decision-making, and privacy harms.

Opportunities Associated with Police Use of Data Driven Technologies

6. In the era of big data and advanced information communication technologies, data driven technologies are central to daily interactions with economic, political, and social institutions. For example, online retailers rely on data driven technologies to: collect data on consumer habits, analyse this data to discover consumer trends, and use knowledge of these trends to make predictions about future consumer behaviour. Predictions are then capitalised on by retailers in the form of purchase recommendations and personalised advertisements.

7. When adopted by police, data driven technologies serve a similar function by collecting data related to criminal behaviour, analysing this data to determine trends, and using knowledge of these trends to make predictions about future crimes. Predictions are subsequently used to make decisions about the allocation of police resources, hence the term ‘predictive policing.’ Contemporary predictive policing uses software that amalgamates the police's crime data (information from police databases) and data from other sources. Relevant data in this regard may include publicly available open source information, or more personal information obtained by law enforcement or intelligence agencies based on the powers established in the Investigatory Powers Act.

8. There are many types of predictive policing.[3] For example, certain predictive technologies attempt to identify specific individuals. This may involve: predicting victims (those most likely to be victims of crime); predicting perpetrators (those most likely to be guilty of having committed a crime); or predicting future offenders (those most likely to commit crime in the future). Predicting future offenders can involve calculating and applying “threat scores” that assign labels (usually numerical digits or colour-codes) to individuals representing the likelihood that they will commit crimes.[4] Threat scores are based on datasets populated with information such as: demographic information, personal data, an individual’s contact network, or an individual’s criminal history.[5] A contemporary example of threat scoring is the Fresno California Police Departments’ use of Beware Software,[6] which analyses publicly-available data to assign a colour-coded threat level to an individual previously identified as being of interest to the police.[7]

9. Another type of predictive policing involves producing "crime maps" and predicting areas where crime is likely to occur in the future, also known as “hot spots.” A contemporary example of this type of predictive policing is the Los Angeles Police Department’s (‘LAPD’) use of PredPol software,[8] which uses three data points (time, place, and type of crime) to identify likely hotspots. PredPol employs advanced algorithms capable of examining crime data and identifying hotspots at a speed that human crime analysts cannot. It is argued that its speed and accuracy mean that PredPol holds the potential to increase the efficiency of police work by allowing officers to quickly locate and pre-empt crime, resulting in a reduction in overall crime rates.[9] By focusing on the geography of crime rather than details such as the demographics of victims or perpetrators, the designers of PredPol and other location-based predictive software claim that predictive policing avoids discriminating against members of certain groups.[10] 

10. PredPol is now used in several LAPD divisions (and over 60 other departments in the US),[11] and predictive technologies are currently migrating to England and Wales. For example, Kent Police have adopted PredPol, and early studies suggest the software may have contributed to a 6% reduction in crime rates in North Kent.[12] The Metropolitan Police Service in London have also trialled a predictive software developed by Accenture to predict gang associations and gang violence.[13] Similar predictive technologies will be trialled in South Wales and Gwent Forces as part of a test of a virtual reality program known as "Gotham," designed to help police analyse crime patterns.[14]

11. Despite the potential benefits of predictive software, including their potential to reduce overall crime rates, existing research has identified significant risks that must be addressed. Three key risks are presented here for illustrative purposes.

Three Risks of Data Driven Policing and Predictive Software

12. Data Quality: The effectiveness of predictive software relies on the quality of input data. If the data that is analysed by predictive software is inaccurate, incomplete, or skewed, this will significantly affect the quality of predictions. For example, predictive algorithms based on deficient data may result in false positives (no crime in alleged "hot spots") or false negatives (crimes in areas identified as low-risk).[15] There are many reasons that data may be deficient, including a heightened emphasis on more visible forms of crime and disorder, whilst concealing others that are less visible. For example, some crimes are consistently under reported – in relation to both the type of crime, and the areas in which crimes are typically reported as occurring – meaning crime data is often incomplete and cannot provide algorithms with a quality representation of the actual recurrence of crimes in a space.[16] Furthermore, police discretion introduces a level of subjectivity into the production of crime data, which may affect the quality of that data in terms of its depiction of crimes rates.[17] Resulting crime data will present a skewed portrayal of the persistence crimes in a space and predictive software will be unable to accurately locate hot spots.[18]

13. Discriminatory Capacities: Data driven policing and the use of predictive software can result in discriminatory outcomes.[19] Despite police efforts to adopt an objective methodology, concerns about biased policing persist, particularly in terms of decision making about the locations in which police officers patrol. Evidence suggests that some police officers continue to target members of marginalised groups and impoverished neighbourhoods.[20] if this approach is taken, crime data will falsely suggest that crime rates are particularly high in impoverished neighbourhoods, not because they are the locations where crime is most common, but because they are the locations in which police focus most of their patrols, overlooking similar crimes occurring in areas where there is limited police presence.[21] Once this skewed data is introduced to predictive software, the software will most likely notice that crime data suggests that most crime takes place in impoverished neighbourhoods, and predict that future crimes are likely to follow this pattern.[22] Such predictions will funnel more police into already over-policed spaces, resulting in a self-fulfilling cycle in which the criminality of impoverished spaces is given priority by police, and crime data continues to suggest a correlation between impoverished spaces and crime. This self-fulfilling cycle is sometimes referred to as the ‘ratchet effect’[23] as it persistently narrows the criminal justice system’s focus on impoverished spaces.

14. Therefore, despite claims about the objectivity of algorithms, there is a risk that predictive software reinforces the targeting of impoverished spaces and the residents within these spaces, often racial and ethnic minorities.[24] This conclusion is supported by studies in the United States including the Human Rights Data Analysis Group’s research finding that algorithms that are trained on biased input data tend to make predictions encouraging the police to focus on impoverished spaces.[25] For instance, the study found that, when applied to Oakland Police Department’s data on drug crimes, the algorithm suggested police almost exclusively target lower income, minority neighbourhoods.[26] Similarly a study by the RAND Corporation suggested racial and ethnic biases among the targets of police interventions persisted despite the implementation of predictive software that claimed to remove biases from the police’s decision making.[27] A related consideration here is the degree to which any inaccuracies, or inbuilt biases, are reviewed and evaluated.

15. Privacy Harms: The use of data driven technologies requires the collection of large quantities of data. Mass data collection raises questions about the police’s contributions to bulk and indiscriminate surveillance. Despite the argument that surveillance in service of crime control should not concern those with 'nothing to hide’,[28] surveillance poses significant risks.[29] Risks include violations of privacy rights through the mass surveillance of large numbers of citizens.[30] Without privacy, citizens are left without a retreat from which to carry out unmonitored communication, thought, and self-expression.[31] This may affect the functioning of democracy,[32] or may result in a "chilling effect"[33] based on the premise that citizens inhibit their behaviour (related to political opinion, sexuality, or religious belief for example) and conceal information (related to one’s health or political views for example) that, if publicised, could result in social exclusion and/or the denial of opportunities.[34] The European Court of Human Rights has repeatedly highlighted the risk that ‘a system of secret surveillance for the protection of national security may undermine or even destroy democracy under the cloak of defending it.’[35]

Recommendations: Regulation and Oversight

16. The future demands and challenges of policing will depend on the police’s ability to address the opportunities and risks of data driven technologies, particularly those involved in predicting crime. Among the challenges faced by police is the need to simultaneously make effective use of data driven technologies to improve their ability to serve the safety and security expectations of the public, while addressing challenges related to the social implications of data driven technologies. Of particular concern is the fact that the risks identified with such policies bring into play legal requirements relating to, for example, the prohibition of discrimination, the right to privacy, and the right to freedom of expression.[36]

17. Addressing these social and legal concerns will require the adoption of oversight and regulations facilitating independent assessment of crime data. It will also require the police to create systems and policies that ensure use of data driven technologies comply with human rights law and address concerns about the far-reaching and intrusive nature of mass surveillance. Accordingly, the Project recommends the creation of detailed oversight and regulation mechanisms with particular attention paid to if and how police address the limitations of crime data, and the discriminatory capacities and privacy harms related to the use of data driven technologies.

18. The Project[37] has dedicated resources to an empirical case study exploring police organisations use of predictive and crime mapping technologies to determine how police organisations have addressed the risks discussed in this submission, and further oversight and regulatory mechanisms that may be required.

Human Rights, Big Data and Technology Project

University of Essex’s Human Rights Centre