19
Examination of Witnesses
Anna Thomas, Jeremias Adams-Prassl and Sian McKinley.
Q20 Chair: Good afternoon and welcome to today’s meeting of the Joint Committee on Human Rights. We are a cross-party committee and a Joint Committee, which means we have members from both the House of Commons and the House of Lords.
This is the committee’s third oral evidence session as part of our inquiry into human rights at work. Today we will focus on the use of artificial intelligence and surveillance in the workplace, which we will consider through a focus on Articles 8 and 14 of the European Convention on Human Rights.
We have three witnesses to assist us today. I will introduce them in no particular order, starting with Anna Thomas, who is the co-founder and founding director of the Institute for the Future of Work, an independent research institute exploring the impacts of technology on work and working lives. Anna has advised the Government and the Centre for Data Ethics and Innovation on technology policy, including on the review into bias in algorithm decision-making. Formerly, Anna was a barrister at Devereux Chambers specialising in equality and labour law. Thank you for joining us, Anna.
Next, we have Professor Jeremias Adams-Prassl, who is a professor of law at Magdalen College at the University of Oxford and a senior research associate at the Institute for Ethics in AI. The professor’s research focuses on technology, innovation policy and the future of work. We are very grateful to you for joining us this afternoon.
Third is Sian McKinley, who is a senior associate at Herbert Smith Freehills in the employment and data protection team. Sian has experience in cases spanning the range of employment and discrimination law, and she has represented both employers and employees. She also has a significant data protection practice both in an employment context and in relation to wider general privacy issues.
Thank you to all three of you for joining us this afternoon. I will take the Chair’s privilege and kick off with the first question, which is really quite a general question. The use of surveillance technologies in the workplace engages Article 8 of ECHR, which protects the right to private and family lives. To what extent do employers use surveillance and, if so, why and what kind?
Anna Thomas: Yes, employers are using AI and related technologies increasingly at work. Adoption is broadly going up, and fairly quickly, but results and application are very uneven. I should perhaps flag the incremental nature of the change and the impacts. They span the full range of the job lifecycle, right from advertisement and recruitment to the full range of management functions, both traditional and new, such as making predictions about how the workforce, or future workforce, will behave. In a sense, this comes with a shift from the assessment of performance to the assessment of who someone is and how they will behave, which is perhaps important in the context of Article 8.
On the broad themes, we have identified that there is experimentation and unpredictability in how AI is used, so it is not always clear how it will be used, although it tends to be used to perform tasks and functions that have previously been performed by humans. That is the traditional way of looking at it.
Chair: That is AI, but I am thinking more about surveillance at work. To me, that means that employees are being monitored, whether remotely or, if we think about delivery drivers, through the use of apps to track where they are going and how quickly they are doing things. We will come to AI in a minute; David has a few questions about that. Can we focus on how employers use surveillance?
Anna Thomas: That is a very helpful clarification. It is not only about AI; it almost always comes with data gathering from a range of other technologies and software, from wearables and CCTV to heat sensors and facial recognition. It is put together and then analysed, often by AI or algorithmic systems, in order to perform functions or make predictions, classifications or recommendations. Our work suggests that those are not just about individuals or groups but about working models and practices. The transformation as a whole is not just of the nature of work but of the conditions of work, the type of work and even working models.
Chair: Sian, in your legal practice have you come across the use of surveillance in the workplace by employers? If so, what kind are they using and why?
Sian McKinley: Yes, absolutely. We tend to use the phrase “monitoring”, because “surveillance” can have quite negative connotations. Some of these technologies have been around for a very long time and are used for what I think we can all agree objectively are some very important reasons.
The concept of surveillance and monitoring captures quite a wide range of technologies. Loss prevention technologies for email traffic are used to capture confidential information going out, simply to prevent data breaches when something is sent to the wrong person, or to capture email traffic coming in to prevent phishing attacks or other cyberattacks. Something as simple as swipe cards counts as surveillance or monitoring. That is technically also an algorithm; it uses analytics to identify whether someone is in or out. Likewise, CCTV is an example of monitoring that does not necessarily use an algorithm. It is used to protect the security of people, for health and safety or to determine who is in the building and who is not.
You mentioned the logistics and distribution sector. Tachometers are used to track the driving time of drivers to ensure that they are safe and are staying in compliance with the law. In warehouses, we know that technologies are used to maximise efficiencies and to track whether lone workers are on their own or have not moved for a long time so that someone can be alerted.
We see the recording of telephone conversations for training and quality purposes—I am sure we have all heard those messages—but also for compliance with legal obligations in the financial sector. In professional services, we see monitoring activities again to comply with regulatory obligations; for example, document management systems are used to ensure that not only records but time recording and billing are accurate.
The idea of surveillance or monitoring is extremely wide-ranging. Sometimes it overlaps with analytical algorithms, and sometimes it does not; generative AI, for example, might not be used in the context of surveillance.
Chair: That is very helpful.
Anna Thomas: I agree very much with that evidence, but I would add that it is hard to see surveillance or monitoring in isolation. Often it is a window or access point to a whole range of impacts, such as intensification, tacit knowledge elicitation or refractive surveillance, that can bear on other fundamental and human rights, including economic ones, such as systems for pay or the allocation of work.
Professor Jeremias Adams-Prassl: I have one very small point. We should not underestimate the role of the Covid pandemic in the massive explosion of these surveillance techniques. Imagine if in February 2020 employers all over the country had told people that we had to install video cameras in our bedrooms and kitchens. You would have had the biggest unionisation drive in history on your hands. Yet suddenly, two months later, the game changed completely.
It is really important not to take the pandemic times as the baseline for going forward. We need to think very hard about this idea that surveillance has suddenly become so omnipresent. There are potentially very good reasons for that, but we may have to make sure that that does not become a baseline.
Chair: That is really interesting.
Q21 Lord Henley: Can I just pick up on that “potentially for very good reasons” point? Presumably there are many occasions when an employer would be failing in their duty of care by not providing some sort of surveillance. Sian mentioned people working remotely, which might be in an unsafe environment, and therefore the employer has a duty to have some means of knowing what they are doing for their own good.
Professor Jeremias Adams-Prassl: Absolutely. I sometimes volunteer as a lighthouse keeper in Canada, and I wear GPS and monitoring systems of exactly that kind. I am deeply suspicious of surveillance normally, but in those circumstances, when I am by myself on a little island, I am deeply grateful that somebody is monitoring me at all times.
My other point is one that Sian has already made. Of course, employers also have other obligations beyond health and safety such as security or the prevention of insider trading. There are good reasons for doing it. The question then becomes more about what is proportionate in the particular circumstances.
Lord Henley: Conversely, one should also add that employees have now discovered, particularly when working from home, that there are things they can do to their laptops to make it appear as though they are working. I know this from my own children; it is not for the record.
Chair: Do share.
Lord Henley: It implies activity on their machine and the machine not being idle, so the employer thinks they are working when they are not.
Professor Jeremias Adams-Prassl: There is a whole range of gadgets you can buy on the internet that will simulate mouse movement, for example.
Chair: I had no idea. That is completely new to me.
Professor Jeremias Adams-Prassl: You can also get it for your Fitbit. You can set it up to do a 10 kilometre run and then go for a swim while you pop down to the pub.
Q22 David Simmonds: We are learning evermore every time. This question moves on to artificial intelligence more specifically. We know that artificial intelligence in the context of business is not new. Automated credit scoring could be considered a form of that. It has grown in the context of the workplace specifically where it engages Article 8 of the ECHR.
Could you perhaps explain, building on what was said earlier on, what is meant specifically by “artificial intelligence” in this context? Could you give us some of the current examples of AI used in the workplace and any of the anticipated future uses that are on your radar?
Anna Thomas: In a sense, AI is a marketing term. It can take many forms. It is perhaps easier to think of it as a scientific field than a particular thing, but it tends to be used as a shorthand for machine learning, which learns from collected data how to perform tasks defined by humans and finds patterns to make recommendations or predictions.
We have toolkits that set out some examples on our website, which I can perhaps share later, and machine learning case studies, which again illustrate how it may be used in the course of hiring and job advertisement. Often it is used to combine or fuse complex and very large datasets. It can be used in new or creative ways, but it does not have to be. For example, it could be used to combine social media profiles with performance data, as we have said in one of our machine-learning case studies, as a basis for future recruitment.
As you said, Lord Henley, it can be used in either way. It could be used positively to help us to understand past patterns of behaviour in order to identify the places where we can make correct and make things better. Equally, if we are not careful, as Lord Sales has said a few times, those potential advantages could be reversed. By not being aware of them, they could be projected into the future.
Professor Jeremias Adams-Prassl: It is really important to say, as Anna said, that there are some positive use cases. Health and safety tends to be one of the areas where we are seeing a lot of positive uses of AI. It can also be used for fairly mundane things, such as a big bank trying to work out which teams should sit together and which teams you can offshore to a cheaper location, whether that is in Manhattan or northern New York.
At the same time, we are also starting to see some pretty dark use cases, although not necessarily in the UK yet. To give you some ideas, call centres increasingly use AI software that listens in on every call and then provides live instructions to workers. Imagine I am talking to you right now. If I were running the software, I would have a little screen saying, “Jeremias, you have to slow down now”, “Make a joke”, “Be a bit more empathetic” or “Maybe calm down a bit”.
Baroness Kennedy of The Shaws: Or “Charge them more, Jeremias”.
Professor Jeremias Adams-Prassl: Think about performing a job on a day-to-day basis or giving a speech in the House of Commons while you have this constant monitoring that is not just surveillance but giving you instructions live.
Something you see in the United States, which I am not aware of being deployed in the UK or in Europe, is the use of predictive analytics to try to determine when workers will exercise certain rights. A classic thing is the targeting of trade unionists, for example, by trying to score how likely somebody is to join a trade union. Very large US retailers deploy those systems extensively. Those are some of the darker use cases that we have to think about.
It is also quite important to point out that, increasingly, these AI technologies are not freestanding software. We still think of them as sort of freestanding software—you buy in a recruitment system. Increasingly, they are built into the day-to-day software that businesses use, such as Microsoft Teams, SAP and Oracle. Those systems increasingly have a lot of algorithmic management technology built into them. Employees might not even be aware that every day you use Microsoft Teams it creates a productivity score for you. The university version shows us, before we teach, how many students have read the materials we have set them. I have stopped looking at that because it is too depressing, as you can surmise. If you are using day-to-day technology such as Microsoft Word, it suddenly has a lot of these things built into it.
Sian McKinley: I agree with my fellow panellists that there really is no set definition for artificial intelligence. Many organisations have tried to land on one. For now, we tend to use AI to mean, as Anna said, when computers carry out a task that you would expect to be completed by a human. Machine learning means when a computer program is trained on data and then gives itself instructions or calibrates its outcomes and learns from previous attempts. An algorithm is just the instructions that the computer uses. One of my colleagues uses the analogy of a cake. The algorithm is the recipe, the training data that goes in is the ingredients and the output is the delicious cake.
I would suggest that there are two different ways in which we see it currently. Anna has been talking about one of those, which is where employers use it in respect of their employees. That is often where you see the overlap between surveillance or monitoring and algorithms. Commonly, increasingly, we are seeing the use of it by employees, often in a sanctioned way. Their employers are happy for employees to use it.
A recent study carried out by Herbert Smith Freehills found that 31% of respondents had used generative AI to produce the first draft of professional emails or for letter drafting, 28% had used it to rewrite text on a company website, and 27% had used a chatbot as part of their job. There is a very much positive side to it, where it is beneficial to both employees and employers.
Then there is the other side where we see it being used by employers to get efficiencies in the workplace. That is where we can see the key aspect of what this inquiry is looking into. Are the uses of the technology proportionate? Do they interfere with any Article 8 or Article 14 rights? Do they go too far? It really depends on what the technology is being used to do and what the employer or the organisation is trying to achieve. These things are so wide and varied that it is important to be clear on the specifics of what we are talking about.
David Simmonds: On the last point that Sian raised, we have seen the growth of predictive analytics in the public sector for things like public health, or predictions of the likelihood of homelessness in order to enable interventions to prevent it happening. Are you aware of any lessons that have been learned that might illustrate where that boundary might be drawn, exactly in the way you have described it, where the purpose would justify a particular course of action that, for another purpose, is not appropriate?
Sian McKinley: Jeremias gave one example a few minutes ago of the pandemic. At the beginning of the pandemic, having a webcam in the house might have been welcomed by an employee who was not seeing anyone else, was living by themselves or in their bubble, and that was the only interaction they had with people. In other jurisdictions—for example, in the Netherlands—by the end of the pandemic people were objecting to the instruction to have their webcams on and kept saying that was an interference with their rights.
That was obviously in the pandemic, so it is very specific, but that is a neat example of how a particular type of monitoring technology can be justified in one scenario—by “justified” I mean that it is proportionate to achieve a legitimate aim—but in another scenario is considered too invasive.
Q23 Baroness Kennedy of The Shaws: I am not totally following this. Most of the time when people were going home and doing their work they were having meetings on Zoom, Teams or whatever their choice of delivery was. Then, they would be visible on the screen. Are you saying that employers were saying to people, “Between nine and five, while you’re working at your desk, we want you to have your webcam on so we can see you and come to speak to you if we need to”? Was that going on?
Sian McKinley: I am aware that there was a case in the Netherlands in which a US company insisted that, between certain hours, an individual had their webcam on.
Baroness Kennedy of The Shaws: So the company could see they were at their desk.
Sian McKinley: Exactly, yes. I imagine the company believed that it was conducive to replicating face-to-face conversations and that there was a value in that. In this particular case—again, I should be clear that this was in the Netherlands—the individual brought a claim and said that they considered it to be too intrusive to demand them to be on camera all the time. I am aware of it in other jurisdictions. It has been a real issue.
Baroness Kennedy of The Shaws: What was the outcome of that case?
Sian McKinley: I think the individual won, from memory.
Q24 Baroness Kennedy of The Shaws: That is interesting. I have looked to see whether there is any mention of this in any of the other questions because I do not want to steal anyone’s questions, but I want to ask this. Is there an obligation for employers to let their workforce know that “We have technology at work that tells us when you are at your computer. We know from the intelligence that’s passed to us from our system when you are working through your list of emails, whether you are typing and whether you are moving your mouse”?
Do employers let their staff know that those things are going on? Should there be an obligation to do so?
Sian McKinley: There is a case in the European Court of Human Rights—I do not mean to cut across anyone else’s questions; I know there is a later question on this—called Bărbulescu v Romania, in which the Grand Chamber of the European Court of Human Rights set down a number of guidelines for employers to follow in order to be able to use surveillance or monitoring in the workplace. In that case it was not technology, but the principles still apply. There were certain steps they had to take before they could gather and fairly deploy that evidence. In that case, it was to dismiss the employee. One of those steps is to make sure that the employee knows not just what you are monitoring or the fact you are monitoring but that you are looking at the content of websites they are looking at; you are reading the content of emails. You have to give that kind of information.
Having said that, one can envisage scenarios in which even giving that information would tip someone off. The ICO used to have an employment practices code before it removed it, and it is bringing out some new, more specific information. It used to identify that there is a possibility for covert monitoring in very exceptional circumstances. The example given by the ICO was if you suspected theft in the changing rooms of an employer.
Baroness Kennedy of The Shaws: It might be insider trading or anything that could have a criminal connection.
Sian McKinley: Yes, exactly.
Chair: That pretty much pertains to Article 8, does it not? That is what Oliver was going to ask about next. Then we may explore one or two of the other cases you were coming on to, Sian.
Q25 Lord Henley: It was really just to get on to the law. What protection does Article 8 provide for workers’ rights to private and family life? Could you expand on any other cases or rulings from the court on privacy and non-discrimination that could provide guidance to us? This is obviously a matter for lawyers, but you are all lawyers. I do not know who wants to lead. Perhaps you would, Sian, since you started on this.
Sian McKinley: I am very happy to continue talking about Bărbulescu and then I will pass on to Jeremias for some of the other cases.
As I was saying, a series of guidelines was set down by the European Court of Human Rights, the first of which was that notification has to be given, except in exceptional circumstances, in advance to employees, which must be clear about the nature of the monitoring. The employer also has to consider the extent of monitoring, the degree of intrusion into the employee’s privacy and the consequences for the employee. The employer has to have identified, again in advance, legitimate reasons to justify the monitoring of communications and technology.
This is where we see proportionality and this idea of balance. The more invasive monitoring will require weightier justification.
Q26 Baroness Meyer: I use my personal laptop and my personal email; I send stuff to my personal email because I do not like to use my work one. On the work one, once a week we get a thing saying, “How many hours have you spent talking to your partners or working hard?” I obviously do not work very hard because I do not use it. If you use your personal computer, there is no way there can be any interference from AI or surveillance, is there?
Sian McKinley: If you are using your personal device and there is no software or program on your computer at the point of using it, you are right: your employer cannot see what you are doing. However, that does not mean to say that is the end of it. In the event of the employer needing to investigate or, in the event of litigation for example, there are circumstances in which the employer may ask to see a personal device, at which point the Article 8 rights would be engaged. Part of the balancing act in Article 8 is ensuring that purely private things, where appropriate, are not looked at. That was one of the issues in Bărbulescu, because the individual had been sending messages to his brother-in-law and his partner, and those were all captured.
Having said that—this is a typical lawyer’s answer—there are scenarios where, just because something is marked “private”, it does not mean that is the end of it. Say that an individual is accused of sending confidential information out of the company’s systems in order to derive a competitive advantage or share it with the competitor. The specific concern could be that there has been a data breach and information has been moved off the estate, in which case the very fact that it is marked “private” may not be enough to take it out of the balancing act.
Baroness Kennedy of The Shaws: Catherine’s question goes deeper than that. What she is wanting to know—I would like to know the answer to this too—is this. Imagine you are working on your computer in your office and somebody sends you an email. You think, “I’ll deal with that later. I’m going to be in the House of Lords”. You send it to your handheld device so you can walk around the House of Lords, where you are required to vote and so on. In sending it through and transferring it from your office computer to your handheld device here in the House, can access be gained to that email on your handheld device?
Sian McKinley: I will take it out of that scenario. If your employer does not have software on your device, they cannot have real-time access to it. However, in the event of a data subject access request, for example—
Baroness Kennedy of The Shaws: You are talking about after the event. I understand all that.
Sian McKinley: Yes, after the event there may be a need to call on it.
Baroness Kennedy of The Shaws: I am a lawyer too, so I know all about that. I want to know whether they could penetrate her phone in order to see because they want to read that email.
Sian McKinley: If no software has been placed on her device, there is no physical way to do that. Some employers say, “You can bring your own device”—BYOD—“to work, but you must install our software on it so that we can ensure that you are treating our data appropriately”.
Chair: Equally, other employers will prohibit people from transferring emails from their employee network on to their private device.
I just want to get back to the focus on Article 8. To summarise, if an employer wants to take action that is quite intrusive, they have to provide a justification for it and show there is no less intrusive way of making sure something adverse is not happening, such as the theft of information or goods. Would that be a fair summary of where the balance lies? Oliver, did you want to follow up on that at all?
Lord Henley: I just wanted to hear particularly from Professor Adams-Prassl about what the court itself is saying and where we are on that front, having heard from Sian.
Professor Jeremias Adams-Prassl: Bărbulescu is very much the leading case. It was a decision by the Grand Chamber. The reason why Bărbulescu is so important is because the court departs from its previous jurisprudence. Initially, in the 1990s, you got questions such as “Should Article 8 apply to work at all? It says ‘private and family life’. Is my work actually my private and family life?”
In a case called Niemietz in 1992 the court for the first time says that work is so important to the development of our personal life that, in principle, Article 8 applies. In the early jurisprudence, in cases like Copland, the court essentially just develops a notification principle. Until Bărbulescu, as long as you told people that you were doing the most intrusive thing imaginable, that was okay. The importance in Bărbulescu is that it says that you can never, even through explicit notification, reduce the expectation of privacy to zero. There will always be this proportionality approach where you have to explain why you are doing it. Then, depending on the purposes, you might be able to justify more or less intrusive measures.
Lord Henley: Bărbulescu is the case to look at.
Jeremias Adams-Prassl: Yes, exactly. It is a fundamental shift. This goes back to Baroness Kennedy’s earlier question as well. Notification used to be enough. These days, even if you notify employees explicitly, that, in and of itself, is not enough to comply with Article 8.
Chair: You still have to justify the extent of the intrusion and show that you could not have done something less intrusive to address the issue you are attempting to address.
Jeremias Adams-Prassl: Yes, exactly.
Q27 Lord Alton of Liverpool: Thank you very much for the responses you have been giving to the questions, especially on Article 8. Can I move you on to Article 14 and the potential infringement on questions of discrimination as a result of the use of surveillance or artificial intelligence?
Jeremias Adams-Prassl: My team in Oxford focuses on both the discrimination and the data protection aspects of algorithmic management. Among the challenges with AI, bias and discrimination are probably the most extensively documented. To some extent, some issues with bias are inherent in the way machine learning works, because ultimately it is about identifying patterns, and identifying patterns is also about identifying outliers. Therefore, it is inherent in some deployment of AI technology that there will be bias against people who are less than typical, which might of course often then overlap with protected characteristics.
One illustration of this that is very extensively studied in the computer science literature is facial recognition. Timnit Gebru and others have done extensive work looking at facial recognition systems that work particularly well with white male faces, but the further you move away from that, the worse the recognition gets. We are not just talking about a gradual decline; we are talking about a serious uptick in how much worse systems are.
How does this play out in the employment context? Various rideshare operators, for example, might say to drivers, “Before you start your shift, you have to look at your phone to identify yourself”. Somebody who is not recognised will then be accused of fraud by the system for trying to illegitimately log on to someone else’s account and might eventually be terminated from their employment, not because they have actually committed any fraud but simply because the technology discriminates.
Lord Alton of Liverpool: So the technology itself is not perfect. In some cases, especially for people of colour or people with different orientations or whatever characteristics that may be engaged, could end up being discriminated against as a result of the use of this technology.
Jeremias Adams-Prassl: Absolutely, particularly in intersectional cases of discrimination, when the various protected characteristics start adding up.
Turning to the legal problem, I have published work with Aislinn Kelly-Lyth and Reuben Binns, which I think is on record with the clerk of the committee. Traditionally, the analysis is then to say that this is indirect discrimination. There is this assumption that the algorithm is a neutral practice or criterion and that therefore, crucially, the employer can adduce a justification. Of course, in the context of facial recognition for a driver, you could say that public safety is an important justification for this; you do not want some random person in the back of your rideshare. However, that means that you de facto legitimise and legalise a significant amount of discrimination.
One thing we have been arguing in our work is that direct discrimination should be applied to some cases such as facial recognition discrimination, because ultimately that is discrimination on grounds of or because of the protected characteristic. In our work, we have shown and suggested that you should bring these claims as direct discrimination. Particularly with things like facial recognition systems that are known to be discriminatory, employers should never be able to justify their deployment.
Anna Thomas: Perhaps this is worth flagging. I very much agree with what Jeremias has said on that. Our own work and our analysis as part of the equality taskforce, which was chaired by Helen Mountfield KC, points very strongly to the fact that it is very difficult to access information. There is a real transparency and explainability issue, which starts off with understanding basics about the purpose—as we have already highlighted—the model and factors that are being considered, and how the trade-offs are being made. Our work suggests that there are a full range of impacts right across the conditions and quality of work.
You need a new AI literacy, knowledge and shared language to kick off, which is one of the ways in which human rights can help. It is also right that key human decisions that shape the outcomes are often diffuse and span multiple organisations. For example, recruitment tools or people analytics may well combine in-house tools, procured tools and data purchased from data brokers. The language of statistics, too, often obscures the human roles and gives an air of certainty or neutrality that needs to be unpicked, and it is difficult to unpick it. Of course, we have huge information asymmetries, so tools are simultaneously requiring more granular data, often as part of work, and sharing less about what happens to it.
Lord Alton of Liverpool: With the committee’s previous inquiry, which was into the Illegal Migration Bill, we had an issue with age verification. In fact, Parliament will be debating it further. We were told that this was never certain; it could not identify with any degree of certainty the age of a young person or a child and where they were on a wide spectrum.
We have seen the growth of surveillance equipment. There are 1 million Hikvision cameras around the country, some of them being used in places of work, but owned by a company that is directly affiliated to the Chinese Communist Party. What alarm bells should be ringing in our heads about the scale of these operations that are under way, the people who own them and the potential, therefore, for their abuse?
Sian McKinley: When we are talking about AI and large-scale surveillance, obviously it is important to understand who operates CCTV. Although AI in the workplace is often at a much smaller scale, we have clear principles whereby, if you are going to deploy it in the workplace, you need to carry out this assessment and identify the purpose for which you have used it or want to introduce it.
CCTV by itself—for example, to protect premises or to identify who is attending at your offices for the security of the people who work there—is much easier to justify, compared with surveillance where the purpose is unknown. In the workplace, again, it is important to be clear about what you are using it for, not least because, as Jeremias pointed out, in the event of an indirect discrimination claim we look at objective justification, which is a very similar test of looking at whether you have acted in a proportionate way to achieve a legitimate aim.
Picking up on what Jeremias and Anna said, there are a number of other ways in which Article 14 rights are engaged. Think about an algorithm that is trying to predict who the best candidate is for a CEO role; in 2020, for example, there were five female CEOs in the FTSE 100 and there were six CEOs called Peter, so we would certainly want to be very careful about the data over which that AI has been trained. If you are buying in technology from an organisation, you may not know what data it has been trained on.
I agree entirely with Anna that it is incredibly important to have transparency and explainability, not least because of the way in which the Equality Act works whereby, in a claim of direct discrimination, we have the treatment and we have a difference in protected characteristic. We then need something more, but it does not have to be very much more. In a recruitment context, you could say, “This person is a woman, this person is a man, and they appear on the face of their CVs to be equally qualified”. That raises a presumption of discrimination, and the burden then shifts to the potential employer to demonstrate that the reason for the treatment had nothing whatsoever to do with a protected characteristic.
There is European case law, albeit not in the context of technology, which says that, where there is a lack of transparency, that might itself be enough to shift the burden of proof. If the potential employer has brought in this technology, does not know what it is trained on—it is what we sometimes call a black box—and cannot explain how the technology has come to a conclusion, the potential employer is the one who is on the hook under the Equality Act for the decision, even though the potential employer may not be the one who has chosen to discriminate.
When we are thinking about what we need in order to explain, from my perspective it is important that there is an obligation on the developers of these technologies to provide information about what it has been trained on and how the decisions have been reached. That may involve being transparent about the jurisdiction in which something has been calibrated. Obviously, there can sometimes be a sales pitch, and organisations can say, “No, we’ve taken the necessary steps to ensure that there is no discrimination”.
If you do that in the US, for example, that may not be sufficient to defend a claim in the United Kingdom because of the difference in the jurisdictions. In the United Kingdom, it is the decision-maker, the employer, who is liable under a claim for unlawful discrimination. There are, I should add, aiding and abetting provisions, but that is probably too far for the purposes of the inquiry.
Chair: That is very interesting.
Q28 Baroness Meyer: Do you think that UK law strikes the right balance between the right of privacy of the employee or the worker and the employer’s way of managing his workforce?
Sian McKinley: That is a really interesting question. If you are an individual, there are avenues, primarily using the equalities legislation or data privacy legislation.
However, the reality of the situation is that an individual may be reluctant to sue their current employer. In the majority of the cases that we see, monitoring technologies or analytical algorithms have been used to gather evidence, but the decision has in fact been taken by a person—for example, “You’ve been committing misconduct on your timesheets, so we’re choosing to dismiss you”. The individual then brings a claim for unfair dismissal, for example, and argues that the information or material should not be used because of the way in which it was gathered or because it was done by technologies. That is when the Article 8 rights are engaged to assess whether the use of that material or technology was proportionate.
The issue then becomes slightly skewed. Does an individual get to hide behind their data privacy rights to defend their own wrongdoing? You are more likely to get claims like that than you are to get individuals proactively bringing claims about surveillance or artificial intelligence.
Anna Thomas: I agree. In addition to the problem of enforceability, to directly respond to your question about the Equality Act, we need more test cases and to test the boundaries of existing protection, for sure, but there are some areas that we already know are problematic.
It may be that behaviours or predictions are not linked with protected characteristics as they are identified at the moment. It could be postcode, voice or something else. It could be that there is a combination. In fact, it is likely, given the way algorithmic systems work, that it is not one isolated characteristic; it is a number of them. That is perhaps a particular area for attention from the committee.
The other area that is outstanding is group impacts. Again, because of the way in which the systems work, which is to find common denominators and to make predictions and recommendations about groups of people, it is not enough to rely on an individual after the event. There needs to be a paradigm shift—perhaps the overarching framework of human rights can help with this—towards pre-emptive evaluation of impacts, including and in particular human rights.
Jeremias Adams-Prassl: The employment context is very specific. When you think back again to the origins of Article 8, it is primarily designed to restrain the state. It is all about protecting citizens against extremely intrusive state surveillance. That also translates into data protection laws, where again there is some tension. On the one hand we have an omnibus regime such as the GDPR, with one set of rules that applies to consumer finance, work and healthcare.
In a blueprint we have recently published, we suggest that employment-specific rules are needed, to help both employers and employees, because there are also lots of employers out there who struggle, in my experience, particularly SMEs. As Sian has said, they do not develop their own software; they buy these things in and are at the mercy of the vendors, realistically, in commercial terms. Employers even struggle with translating things like the Bărbulescu criteria. If you have to decide what to do, it is difficult, so we need much more specific norms and guidance, particularly in the employment context.
Baroness Meyer: You think that could solve the problem and create a better balance.
Jeremias Adams-Prassl: That would go a long way towards having a better balance and would help both sides. It is interesting that the core of the compromise is hidden in the GDPR so far back that everyone is asleep by the time they get there. Article 88 of the GDPR specifically provides that member states can lay down rules for data protection in the employment context. One of my post‑docs, Dr Halefom Abraha, has done interesting work comparing what different jurisdictions have done. Increasingly, various member states are starting to develop very specific norms using Article 88.
Chair: I suppose if Article 14 in ECHR goes beyond the protected characteristics we have in our Equality Act, there is all the more reason to have a special charter for employment. As you say and Anna has just said, you go beyond the protected characteristics and into areas such as postcode, voice and group, and, as you said earlier, intersectional profiles of people.
Q29 Lord Dholakia: Thank you for the information you have supplied so far. Do you think that UK law adequately protects the right to privacy in workplaces?
Jeremias Adams-Prassl: For the reasons I have suggested, it does not go far enough at the moment. In this blueprint proposal for a new law that we have produced, we have identified two particular challenges. On the one hand, there are these information asymmetries and privacy harms, both individual and collective. For some things, we might just need a clear ban, for example predicting who might become pregnant so you can sack them beforehand, or predicting who might be a trade unionist so you do not hire them. That is a use of analytics that should be forbidden, full stop, without any further proportionality inquiry.
The second big thing we have identified is a lack of managerial agency. A lot of law, whether it is equality law or employment law, conditions how managers exercise their authority. The moment you start automating these functions with algorithmic management, that responsibility becomes diffused in the cloud.
To give you a practical illustration of this, in litigation in the United States against a major provider of online logistics and retail solutions, the allegation was that trade unionists had been targeted in dismissals. The company’s defence was to say that the local manager, who ran this enormous warehouse, neither understood nor controlled the system that sacked the workers. It was an automated productivity system.
On the facts of that case, that was a pretty clever defence to run, because you could show there was no animus regarding trade unions, but take a step back and think about this from a dignity perspective. You have been working somewhere and you get sacked by an automated system. You then try to inquire what happened, and you get an automated reply to that. It is a real challenge.
Baroness Kennedy of The Shaws: I just want to let everybody know that reforming the House of Lords will depend on the amount of speeches and questions that people ask, so we will be able to know just how active people are in the House of Lords. I hope you all realise that that is happening currently.
Baroness Meyer: That is the most stupid thing, because it is not about the number of times you speak. Sometimes less is more.
Jeremias Adams-Prassl: People are also trying to use this sort of analytics for litigation and judges. French law has banned analytics that predict what judges do. When I told a group of judges this at a recent judicial training, you could see them all drafting emails to the Ministry of Justice, so you might want to look into something similar.
Q30 Lord Dholakia: You mention the specific laws. At the moment, we have the Data Protection and Information (No. 2) Bill. Would that help in this particular situation?
Jeremias Adams-Prassl: The full automation of termination at the moment would probably be illegitimate under Article 22 of the GDPR as incorporated into UK law, so we have safeguards at the moment. Again, the main problem is that these norms are so general because they have to apply in so many areas. That is why it is so important that we recognise the specificity of the working context and, therefore, have specific norms for that.
That is not to say that existing norms would not apply. You would still be subject to unfair dismissal law; you would still be subject to discrimination law, but it would help both workers and employers to have more clarity and a very specific set of norms that address those challenges in the workplace.
Anna Thomas: We agree with that. We have also done a mapping, if you like, of how the data protection regime applies and the gaps between that regime and the Equality Act. We have found some such areas, many of which we have already touched on, including purpose information, not just about the model but about other things beyond the logic. One is identification and involvement of stakeholders. Another is ongoing monitoring. Given that some of these impacts are incremental and may not be apparent before the use has started, ongoing monitoring is important.
It is right that the data protection Bill as it is at the moment—it is not No. 2 yet—starts to deal with these issues in some important ways that we think should not be undermined, including a prior data protection impact assessment, when the type of processing is likely to result in high risks to rights. That is often the case in the workplace. There is added protection for solely automated processing, access to information, including the logic, and consultation. It is in fact those areas we have identified as being particularly key for modern users of AI and algorithmic systems that are at the moment subject to review in the Data Protection and Information (No. 2) Bill.
On the upside, it is also an opportunity to improve them, to think of building in additional specific rights in the context of work, as Jeremias said, in order to deal with the issues you have already identified in your questions, and to think of introducing, in the place of a data protection impact assessment, what we call a good work algorithmic impact assessment. That would deal with the challenges we have begun to identify today.
Sian McKinley: As Jeremias said, Article 22 would prohibit the kind of automated decision-making that he was talking about. I am aware of the proposed changes in the Bill that would enable automated decision-making to take place with consent as an alternative, but that does not have much of an impact in the workplace. Consent is already problematic to rely upon because of the inherent imbalance of power between employees and employers. It does not lessen the protection in its current form.
I completely agree with Jeremias and Anna that there is a need for legally binding AI rules, more than just an obligation to carry out an AI impact assessment, which will often be caught by the obligation to carry out a data protection impact assessment. In any event, good employers and good corporate organisations want to be compliant with the law. They want to act ethically in the use of this technology, which is not going anywhere.
At the moment, understanding what needs to be done to be on the right side is difficult. It is a little bit unclear, because it is sector-specific and organisations can operate cross-sector. There are very different approaches being taken in different jurisdictions. This is a particular issue for multinational organisations, especially when technology is sold in another jurisdiction and operated in this one.
I agree that we need legally binding AI rules and alignment across domestic and global policymakers in order to bring certainty and to prevent good employers and good corporates from being undercut by organisations that are not quite so diligent.
Q31 Baroness Kennedy of The Shaws: You mentioned the business about judges in France and a decision being made about this being a category or a place where the use of such technology would be inappropriate. Are there any other areas where you see that as something we should be thinking about? Should there be some places where it is just too dangerous to apply artificial intelligence or any of this kind of technology?
Jeremias Adams-Prassl: Generally speaking, the closer you get to soft human values, the more inaccurate and problematic AI becomes.
Baroness Kennedy of The Shaws: Would that include sentencing in criminal cases?
Jeremias Adams-Prassl: Yes.
Baroness Kennedy of The Shaws: What about medical decisions?
Jeremias Adams-Prassl: It might include some, but there is some medical AI that is pretty good and might outperform humans. I am thinking more about judging productivity or how capable somebody is as a team member. The closer we get into those soft factors, the harder it is.
Baroness Kennedy of The Shaws: Some people say that using technology in the workplace increases productivity, and that heavily regulating it will stifle the great productivity of the United Kingdom. I just wondered whether there is research on this. Does available research suggest that greater regulation of technology and strengthening the protections of human rights and of workers would lead to a decrease in productivity and creativity in the workplace?
Anna Thomas: The short answer is no. It is a very difficult area and there is nothing on causation, but we have a few pieces of research. One of them is out today; it is called Reframing Skills. One of them is a survey with Warwick Business School on how firms are using not just AI but other automation technologies. Some literature reviews are coming on the same subject. The survey, carried out by Professor James Hayton, shows that, as of March this year, 79% of firms are using automation technologies, including AI, for a wide range of both cognitive and non-cognitive tasks, which suggests that the transformation is well under way.
Speaking to your question, it is interesting that, because it was done from an organisational management human resources perspective, this particular survey with a business school shows good outcomes in general terms. Both on job quality and job creation, outcomes go up with regional innovation or readiness for technology adoption. That includes investment in education and human capital, as well as technology infrastructures. Human resource management practices in firms, including access to information and consultation and workforce engagement, were also an important driver of adoption and strengthened the relationship between technology and work outcomes at different levels.
Our lit reviews from different disciplines, including economics and even machine learning, show that higher attention to machine-human interaction and higher levels of information, engagement and involvement are associated with better outcomes. I need to caveat that by saying that the research is ongoing, but it tends to be consistent, in my opinion, with a rights-based approach.
Baroness Kennedy of The Shaws: It is that whole business: “We want to be able to watch what people are doing, make sure that they’re at it all the time and be on their backs”. It is this sort of thing about too many visits to the lavatory limiting productivity: “We want to see how often people are going”. What is the answer to this? Do we get better productivity if there is this? Is regulating technology in the workplace going to inhibit productivity, or, in fact, is it that trust and including people in the process has better outcomes?
Sian McKinley: From my perspective, I would say that trust is key. Trust is important in the employment relationship. You give the example of using an algorithm to watch whether someone is going to the bathroom. Is that any different from a manager standing on a platform overlooking the factory and watching when people are coming in and out or whether they are standing around talking for too long? It is not the technology in that scenario that is causing the breakdown in the relationship; it is the particular scenario of monitoring.
It may be justified; it may not be, but it is about trust between the employer and the employee, which can be achieved through clear notification and explanation of what this is trying to achieve. Listening at times to any concerns and having, at the employer’s fingertips, the information to pass on in answer to those concerns is really key.
Chair: I will have to interrupt there, because the committee is about to become inquorate. We have members who have to be elsewhere this afternoon, so we will not be able to go beyond 4 pm. It has been a really interesting session. We have other questions we wanted to ask you. If you bear with us, we will write to you with some follow-up questions. I am very grateful to you for the fascinating evidence this afternoon.
Oral Evidence: Human Rights at Work
