Digital, Culture, Media and Sport Committee 

Oral evidence: Connected tech: smart or sinister?, HC 157

Tuesday 19 July 2022

Ordered by the House of Commons to be published on 19 July 2022.

Watch the meeting 

Members present: Julian Knight (Chair); Kevin Brennan; Steve Brine; Clive Efford; Julie Elliott; Dr Rupa Huq; John Nicolson; Jane Stevenson; Giles Watling.

Questions 1 - 68

Witnesses

I: Silkie Carlo, Director, Big Brother Watch; Dr Lulu Shi, Research fellow, Oxford Internet Institute; Dr Leonie Tanczer, Lecturer, International Security and Emerging Technologies, UCL; Antony Walker, Deputy Chief Executive, techUK.

Examination of witnesses

Witnesses: Silkie Carlo, Dr Lulu Shi, Dr Leone Tanczer and Antony Walker.

Q1                Chair: This is the Digital, Culture, Media and Sport Select Committee’s hearing into connected tech. We are joined today by four witnesses. We have Antony Walker, deputy chief executive of techUK; Dr Leonie Tanczer, lecturer at International Security and Emerging Technologies at UCL; Dr Lulu Shi, a research fellow at Oxford Internet Institute’ and Silkie Carlo, director of Big Brother Watch. Silkie, Lulu, Leonie and Antony, thank you very much for joining us this morning.

Declaring an interest before we begin, I am the chair of the APPG on new and advanced technologies. Does anyone else have any interest to declare?

Julie Elliott: I am the chair of the all-party parliamentary group on digital skills.

Q2                Chair: Thank you very much. I will start with a general question: what do you think are the biggest benefits of connected tech—the current boom—and also the drawbacks?

Antony Walker: Good morning, everybody. I am Antony Walker, deputy CEO of techUK. We have about 900 technology companies in our membership. We have a very broad view of the technology ecosystem.

Looking at the connected technology landscape, we see connected devices becoming more ubiquitous in the home, in the workplace and in cars, for example. These are devices and services that deliver a huge amount of utility to the consumer. That is one of the primary reasons why they are so popular. They can range from things such as our ability to access entertainment in a hands-free way, very easily and seamlessly, and in a fun way, all the way through to things that can be life-saving for individuals, such as assisted-living technologies and monitors that can detect if somebody has had a fall in the home or something like that. The range of devices and applications is broad, rates of adoption are accelerating and there is potential for ubiquity in all sorts of different settings.

We see huge potential benefits, not least in things such as energy management in buildings, whether it is houses or large buildings, where by connecting heating systems, hot water and electricity and so on into a smart grid, you can drive energy benefits in a way that will help us to achieve net zero. There are potential benefits, but on the other side, of course, there are significant risks that we need to be aware of—risks around privacy, cyber-security and some unforeseen societal impacts and changes in norms that can be driven by these big technologies.

That is why the regulatory framework around these technologies, whether around privacy, cyber-security, AI, ethics and so on, is so important. That is why this Committee inquiry is well timed. It has a very broad remit when you begin to think about the very broad range of policy and regulatory frameworks and safeguards at play here, which we need to have in place to make sure that we can drive the benefits of these technologies safely.

Q3                Chair: Dr Tanczer, you were nodding when we talked about risks to privacy and cyber-security and about societal impacts. Do you want to expand on that?

Dr Tanczer: Absolutely, and thank you very much for having me. I am leading a project called Gender and IoT at UCL. We are looking at how smart devices are impacting victims and survivors of internet partner violence. That is not the only association I have with connected smart devices. I have been part of the PETRAS Internet of Things Research Hub, which is a large-scale, UK-wide project where researchers study the impact of IoT from an engineering perspective but also from a societal and regulatory perspective. Therefore, I am coming from these sides, but what I want to specifically talk about today is of course the risks for domestic abuse victims particularly.

Without a doubt, IoT-connected devices have a lot of benefits for victims and survivors, police services, support services and so on, but from our early research, in the context of the UK and with regard to smart devices, we are seeing that the IoT broadens and exacerbates the reach of perpetrators. You no longer need to be physically co-present with someone to not just impact them but change the physical environment that they are in, through for instance their heating, their temperature.

If you think about domestic abuse or intimate partner violence relationships, people are often gaslit. While for you it might seem mundane to think that your blinds go up and down, if you are at home and worried about things, and your partner tells you they have hacker friends that they may not have, or they have all these abilities that they may not have but still tell you that they have, people really are in an environment where they are becoming fearful. We have seen that through our recent research where, for example, people have been told that smart vacuum cleaners have cameras and knives installed in them, so this is about the broadening exacerbation of patterns of abuse that come together with remote control and automation but also data profiling.

Information collected from victims and survivors over years of abuse by perpetrators is coming together in these devices that we are wearing. That may not be reflective of the person they are, because they have been victims of abuse for 20 years, but suddenly their insurer says they have a certain sort of risk pattern or financial credit rating that is not reflective of the person they are, because they have been in that abusive relationship.

Another important point is that these devices are now disguised so they look like a normal kettle or a normal camera. There are also technical functionality features that we need to make clear: when are they on? When are they off? How are they connected when I have them connected?

For me, the biggest risk—if I can say it in one final sentence—is that we are in an environment now, a timely environment as we said earlier, where society is at risk of underestimating and overestimating the functionality. We see things such as “Black Mirror” where IoT, smart things, can do all these amazing things, so people have distorted views of it. Equally, people underestimate how much information IoT can collect, the impact on privacy and security, and the safety risk when they use these devices. That is my core message: we need to be emphasising education and digital literacy, challenging people’s under and overestimation and making it clear to them what they are buying into.

Q4                Chair: Drilling down, your first major point was about the potential for people to be abused in their homes by abusive partners using connected tech. I had no idea that that was something that happened.

Dr Rupa Huq: Question 5 is on this topic. Shall I take that now or come back to it?

Chair: I am asking the witness some questions. We will come back to that later.

I had no idea this was a phenomenon that was happening. Could you flesh it out? Is this something that is happening regularly?

Dr Tanczer: Without a doubt. Regular phone apps and everything else that is being discussed in the Online Safety Bill are the majority of the issues. That being said, IoT devices are not yet as widespread as smartphones and that is the problem. We also lack figures because crime in England and Wales does not specifically narrow down the abuses that—

Q5                Chair: Sorry to interrupt, but to emphasise the point, there is little data on this and what you are talking about is more anecdotal or potential.

Dr Tanczer: We are working with domestic abuse charities. They are seeing this. Refuge and other charities can talk about this more specifically. However, stalking charities, such as the Suzy Lamplugh Trust, have said that 100% of their cases have a cyber element. What is cyber? Is that just social media apps or is that IoT? I think we need to extract data specifically on the devices. However, we do have anecdotal evidence—as I mentioned earlier with the smart vacuum cleaner—and we have evidence from the charities that we are working with that say they have victims coming in asking about smart cameras, smart televisions, smart doorbells and smart speakers—things that are currently affordable and easily available. But the future of where we are going with tech will mean that this is expanding.

Chair: We will expand on this later. I just wanted to establish the empirical base of what you were saying. It seems like a mixture of anecdote and potential. I think that is fair to say, isn’t it?

Dr Tanczer: Tech abuse is happening, but the exact figure—

Q6                Chair: Yes, we do not know the numbers yet. The Committee will have to think about that.

Dr Shi, in relation to my original question about the benefits and drawbacks, what are your thoughts?

Dr Shi: I come from a slightly different background than Leonie’s. My expertise is in technology, education and children so I can perhaps expand a bit more about that.

Let me start with the benefits. There are some clear benefits from using technology, especially if we just think about the covid pandemic and the school closures. Children were sent home. No schools were open and we all had to use technology to enable schooling. That was a huge benefit. Otherwise, children would not have had any education at all.

Of course, there are risks involved. Maybe I can start with more of a macro or organisational level. If we think about using technology, we also need to think about where technology comes from and who the providers are. In most cases, those providers are not public service institutions, so not from the education system. They are large corporate companies in some cases, and also small tech companies. But many schools use Google Classroom or other kinds of apps to enable education. Most of these companies may not have the public interest at heart. They are profit-driven, for-profit industries, so that is something that we need to keep in mind when we are using that sort of technology.

Secondly, I want to mention what those education products do. They enable education but they also collect vast amounts of data. They do collect data from children—two kinds of data if you want a rough categorisation. They collect personal data about children, such as their age, ethnicity, gender and so on, and secondly they collect behavioural data. By behavioural data, I mean how children learn, how they perform at school, and what they do. They can track students in some cases in terms of where they go, and they can track students beyond the app they are using, in other online environments. Those are things that we need to keep in mind.

Lastly, I want to emphasise what is specific to my research. I look at schools, education and children. Children are a particularly vulnerable group because they may not have all the information and knowledge about what is happening to them. Research has shown that children understand interpersonal relationships. Above a certain age, they can estimate the risk in, say, whether to tell a secret to someone. But it is very difficult for them to estimate what online harms are—for instance, what it means if they opt in to something. Unfortunately, most education technologies do not have an opt-out option so either you use the product or you do not use the product. In the case of education technologies, either you participate in education or you do not participate in education.

To end my contribution to the first question, I would say we need to think about how much power we allocate to which social actor groups.

Q7                Chair: Is it fair to say that technology is age-blind to some degree? That the same data that could be collected on me could be collected on a child?

Dr Shi: Age-blind? Yes. Data is collected from everyone—from children, and teachers in the setting of schools. However, children may not be as informed as adults, so they cannot anticipate what might happen to their data and how their data is being used. There should be extra concern for children.

Q8                Chair: An interesting point. Silkie Carlo from Big Brother Watch, I have brought you in last. You have heard a lot about privacy, about children and also about the potential for abuse. Please give us an overarching view of where you think we are with connected tech and what the main challenges are that the Committee faces in this inquiry.

Silkie Carlo: We have had a rich introduction to the issues so I don’t want to repeat what has been said, but I would echo pretty much everything that has been said.

Big Brother Watch is a non-profit, non-partisan privacy and civil liberties organisation. We take an interest in connected tech, particularly with a privacy lens. We see that the potential of the enormous growth of connected technologies is radical from a privacy perspective because we are entering an environment of increasingly ambient surveillance. A lot of the time the word “smart” is synonymous with surveillance. We are talking about technologies that collect data on individuals. Often that is for commercial purposes, but with that comes the risk of data being used, as you have heard, for exploitative or abusive uses. It could also be for criminal activity and, invariably, it is also for commercial gain.

How often benefits to the individual are seriously involved with connected and smart technologies is a different question. Of course, there are benefits. I am sure we all have smartphones, but many people here will use smart speakers and all kinds of other connected technologies that make their lives easier or that are beneficial in some way.

When we think about larger-scale applications, like schools, the health sector and policing, which we have written about in our evidence, let alone the concept of smart cities, we face some very fundamental questions about what kind of future we are building and where people’s privacy rights fit into that, if they fit into it at all.

I think this is a very timely inquiry indeed. At the moment, there is an intention to reform data protection legislation, and in our view and the view of our colleague NGOs, there is a concern that that means watering down data protection. We are on the precipice of a technological revolution. We are looking at more and more surveillance devices and connected technologies across the public sector, let alone in people’s homes, workplaces and schools.

In our view, this is not a time to be watering down data protection at all. It is a time to have a very serious think about how we can bolster protection for individuals, whether that is against commercial data collection or abusive and exploitative uses of technology and all the kinds of risks that you will hear about in this inquiry.

Q9                Kevin Brennan: I have some questions about connected tech in the home. Antony, do you have a smart speaker, for example, in your home?

Antony Walker: Yes.

Kevin Brennan: Do you have one, Leonie?

Dr Tanczer: Yes.

Kevin Brennan: Do you have one, Lulu?

Dr Shi indicated dissent.

Kevin Brennan: Do you have one, Silkie?

Silkie Carlo indicated dissent. 

Q10            Kevin Brennan: Do you think Lulu and Silkie are a bit old-fashioned by not having one in their homes, Antony?

Antony Walker: No, I think it is absolutely up to everybody to make their own choices about what technology they want in their home. These devices can be very useful. I think it is entirely a matter of personal choice.

Q11            Kevin Brennan: Leonie, what do you think?

Dr Tanczer: Mine was a joke gift from my husband.

Kevin Brennan: He is not surveilling you with it, is he?

Dr Tanczer: Honestly, it is fine. It is not working very well. As you can hear, I have an accent so most of the time I have to repeat myself 10 times. Hopefully, they will get better. Before we got it—well, I did not have a choice; he bought it as a joke but, when we set it up, we were very clear about how we wanted to set it up.

Q12            Kevin Brennan: Lulu, why do you not have one?

Dr Shi: I guess mainly because of data privacy concerns. For example, my colleagues work on Alexa, and although this is not my area I do learn some things about Alexa and other devices—about how they collect information. One of the main points that I think is concerning is that many of these devices do not delete their data and you have to actively tell them, “Please delete my data,” otherwise it will be stored for a long period. That can be problematic because you don’t know how it will be used.

In that sense, there is some potential to regulate this part about forgetting the data and how it can be used, because in many cases—

Q13            Kevin Brennan: Do you mean by default?

Dr Shi: Yes, by default and not by consumers having to say, “Please delete my data.”

Q14            Kevin Brennan: Silkie, what is your reason for not having a smart speaker?

Silkie Carlo: I understand that, for different people’s lifestyles, they have different meanings and different uses. For me, a smart speaker is a live microphone and recording device. I do not want to hot-mic my home. It is a private space. Even professionally, as someone who scrutinises these kinds of technologies very closely, I cannot tell you with total confidence where the data goes. It also seems that the companies cannot tell people with total confidence where the data goes, because the privacy policies are lengthy, obscure and inaccessible. To not have that level of confidence over something that is potentially so intrusive is a very risky thing.

We do know that there have been whistleblowers from Amazon and Apple, for example, who have said that some of these devices accidentally trigger and accidentally record things that they are not supposed to record and that those recordings, voice recordings from inside the privacy of your home, can be listened to, and are listened to, by people who work for those companies in terms of product development and so on. There are cases where accidental recordings have related to highly confidential material, whether sexual or medical or about illegal interactions, that people would certainly not want others to be recording and would not want others to hear. With the kind of opacity that we see with these devices and the privacy policies and data flows, that is a very serious issue.

Another thing is that these are not just personal decisions, and that is something that I hope the inquiry considers. Often we think about these choices as purely individual choices. If I have a smart speaker, a recording device or a Ring doorbell in my home, it does not just impact me; it impacts other people that might come into the home. It impacts families, and it impacts communities, too, if it is something that captures the outside of your house, for instance. We need to think about the communal privacy aspect as well.

Q15            Kevin Brennan: Antony, having heard what Lulu and Silkie said, will you disconnect your smart speaker?

Antony Walker: No. The devices themselves have to work within the current regulatory framework of data protection.

Q16            Kevin Brennan: I put the same question to the previous Information Commissioner and to a cyber-security expert who appeared before us in a recent inquiry. Both of them gave very similar answers to what Silkie and Lulu said. The outgoing Information Commissioner, appointed by the Government and answerable to Parliament for the safety of information held on people, said that she did not trust these devices and would advise people not to have them in their homes. What is your response to that?

Antony Walker: I don’t think that is the current advice of the ICO, which is responsible for the regulation of these services from a data perspective. Rather than taking a view that technology cannot be made to work, I think we have to focus on how it is regulated. I totally agree about the timeliness of this inquiry, because you have a new data protection Bill coming through Parliament. That is why the detail of that Bill matters and why it requires careful scrutiny. Rather than just saying yes or no to these technologies, we have to build the right framework to assure their governance and appropriate use.

Q17            Kevin Brennan: According to my notes, you have said that connected home tech can support human rights. What did you mean by that?

Antony Walker: Connected home technology can do some extraordinary things, not least the bit that I think has the most transformational potential, which is around assisted living and social care. We have a problem today with ambulances queued up outside hospitals because they cannot discharge patients into hospitals because we cannot get people out of hospitals and back into their home environments. There are huge challenges in social care. Part of the solution there will be around better using smart connected devices, sensors and other things that can enable people to live safely in their homes when they need additional assistance. From a societal perspective, we can have big transformational benefits from the smart and intelligent use of these technologies.

I am not in any way dismissing the fact that with these technologies come risks. The question for all of you and for all of us is how we face up to those risks, mitigate them and address them.

Q18            Kevin Brennan: Silkie, techUK has also said that industry basically “has a pro-privacy business incentive to minimise cyberbreaches and privacy incidents”. What is your response to that? In other words, it is in their business interests to make sure that people are not worried about being surveilled.

Silkie Carlo: You would hope so. I think it is in their business interests to write and design privacy policies, or gain a public reputation for doing so, that foster trust from the public and that gives them ultimate commercial protection. That is a different thing from having an incentive to give tangible privacy protections to individuals. The companies talk a lot about trust. They talk less about meaningful privacy-protective measures. A huge amount of budget is spent on PR and marketing while, when you drill down into privacy policies and how some of these devices work, quite often you find that there is really deeply embedded data collection, which is for commercial gain.

In particular, if you think about the way that technology is going towards ownership of devices diminishing and towards subscription models, software updates and so on, you have a constant relationship with the technology supplier. One smart device I do have is a printer and, for whatever reason, the driver will no longer update or work with my laptop so I have a defunct device—it will not print anymore. The point is that you have a constant data exchange with a company, which is radically different. In many cases, the company seeks to commercialise your data and wants to have an ongoing relationship where they can do that.

It is true that companies want to have a good reputation in this regard and want to foster trust but that is a belief—it is not meaningful privacy protection.

Q19            Kevin Brennan: I don’t want to take too much time but can I, finally, put this to you all? Some of us on the Committee recently had an opportunity to visit Samsung in South Korea. They showed us a video of the connected home of the future, which I thought was a mixture of utopia and dystopia. It made me wonder. The unexpected technology that 20 years ago nobody thought would be ubiquitous—they confirmed this—was the text message, funnily enough. Nobody spotted that. Nobody on “Star Trek” sent a text. Which of these technologies do you think will be ubiquitous in our lives in 20 years? Antony, as a representative of the industry, I will ask you first.

Antony Walker: A very good question. What tends to happen is a sort of blending of the technologies. Rather than trying to pick a technology, I would say the trend is clear that we will see continued innovation in these technologies. Technology companies come from a basis of trying to work out what the potential to the consumer is and then try to deliver products that benefit the consumer.

Kevin Brennan: I am looking for a short answer.

Antony Walker: We will live in a more connected world, and that has huge potential benefits but significant downsides that we also need to address.

Kevin Brennan: I will not force you to say, but does anybody else want offer a thought on that? Leonie, go on.

Dr Tanczer: If I understood your question correctly, it is, which of these functionalities or devices will be the most ubiquitous? I am scared that in 20 years I will watch this back and I will be wrong, but I would say smart speakers. When I say smart speakers, we are all thinking of Google Home and Amazon Echo, but I think it is more the functionality of having one central node in the home that connects to other systems or, say, with my phone, although phones probably will not exist, and we will have something implanted—God knows. But we will have this kind of intelligent master in the house. I think that is the future.

Kevin Brennan: Big Brother in the home?

Dr Tanczer: Yes.

Kevin Brennan: Lulu, any thoughts, or Silkie? You don’t have to.

Dr Shi: I don’t do forecasts, so I will pass.

Kevin Brennan: Very wise.

Silkie Carlo: Not a product but a feature that I worry about is that, with this environmental surveillance that we are likely to have in 20 years, there will be a state surveillance element in many of these things and that that will be normalised. We are already seeing that in the expectation that encrypted messages should have a surveillance element. Private conversations are no longer seen as private in the way that they used to be, and that will work into our homes and workplaces.

Kevin Brennan: Thank you.

Q20            John Nicolson: I love this discussion about interconnected speakers. I play 78 records with a horn, so I feel relatively safe but maybe I am not.

Silkie, I was fascinated by your evidence when you said, “We are on the precipice of a technological revolution.” Precipices do not normally lead to happy places, so I am guessing that you do not think that is a good idea. However, lots of people will say—and let me play devil’s advocate—that in a smart city, the connected tech is used for things such as waste management and road maintenance. So some people would say you are taking this all far too seriously and that this technology is all benign.

Silkie Carlo: I do take privacy seriously because privacy is a fundamental right protected in human rights frameworks. When I talk about the precipice of this technological revolution, I am talking about us renegotiating what privacy means in the modern world. I love technology as well. I am not a Luddite. I absolutely agree that there are beneficial uses of connected technology. This is not a binary discussion. However, it is also the case that throwing the floodgates open to having connected technology everywhere, because it is innovative, because it is new, is not necessarily a good idea. We have to think very seriously about the risks.

Q21            John Nicolson: Obviously, authoritarian regimes love this kind of stuff because it allows them to control their populations more effectively. Looking long term, is there a danger that there will be a blurring of the lines between states that use technology from an authoritarian standpoint and more benign states, with both ending up effectively with the same level of intrusion?

Silkie Carlo: Certainly there are risks about the political environments in which these smart and connected environments are built. For example, one of the leaders is Dubai. As a human rights campaigner, I would feel differently from an average tourist if I were walking round a smart city in Dubai and I knew that I was leaking data, perhaps about my web browsing habits, emails or personal communications. It absolutely matters, but in any environment it is not just about political authoritarianism; it is also about the commercial exploitation that can happen and that is commonplace in this area.

Dr Tanczer: Can I jump into this? Nobody would have thought that Roe v. Wade would be overturned in the US. I certainly would not have. It is an abortion issue. What that does now for women, or anyone able to give birth, is to make them think, “If I have a fertility tracker, what does that mean?” It is not a question of a benign or an authoritarian state. Laws can change, sometimes slowly, but sometimes quickly, and you do not know what environment you will be in. If you are already part of a very vulnerable group—say, an LGBTQ member in the USA right now—you will probably look at this and think, “Who knows if me having been on certain apps where I have outed myself as gay might be used as a prosecution reason for me in the future if the law changes?”

Q22            John Nicolson: We do know what risks are entailed because several of the justices have said that the next group they are going for is LGBT people. We have seen in this country exactly what can happen when you set up a witch hunt for trans people. Several of the justices have said that they are coming after LGBT people over marriage equality next.

Do all of you accept that big tech is now intrusive and threatening? Are any of you blasé or relaxed about it? Or are you all concerned about it? Lulu.

Dr Shi: I don’t think we should be blasé about it. I think we do need to worry about what is going to happen and to think about a regulatory framework. There are two issues with data. I think Kevin Brennan mentioned data breaches. That is one point. Of course, companies do not want a data breach, because it can damage trust in the company. However, data breach is not where companies are voluntarily sharing data; it is other people hacking into their systems and getting data.

But looking at it the other way around, firms may not be super concerned when it comes to sharing data as a source of profit—voluntarily sharing. In that case we have seen that when Human Rights Watch inspected, I think, around 160 education technology products, they found evidence that with 90% of all the products access to data was being granted to third parties or directly given to third parties. Therefore, you can see that there is data exchange for profit purposes, and I think we need to be concerned about that.

Q23            John Nicolson: Several people have mentioned authoritarian regimes. Is there a danger that our moral authority to criticise authoritarian regimes will be lessened if we are doing much the same thing, albeit from a more benign perspective, in the way that we collect data?

Dr Shi: I think it is a good question. I can only echo Leonie when she brough up the legal framework change in the US. I think it is not only about authoritarian companies; it can happen anywhere. It depends on how the law is structured, and the law can change.

Q24            John Nicolson: Leonie, as an amateur individual, what should I be doing to protect myself from some of the dangers inherent in all of this?

Dr Tanczer: I am not a lawyer, but this is a very legalistic answer: it depends on your risk profile.

Q25            John Nicolson: What is my risk profile? Presumably, higher than most because I am an MP.

Dr Tanczer: Definitely. When it comes to domestic abuse, you normally do a risk assessment and you would go for certain questions, which would not be applicable to you. I would normally not give generic advice to anyone, because people are different, but there are certain things that you could definitely look at, such as passport security.

It is a very broad question, but it boils down to: it depends on the individual, and generic advice can backfire. For example, I might be giving Antony advice based on his needs, and someone might be sitting here, for whatever reason, who was part of an intimate partner violence relationship. If I told them to deinstall something, that might trigger something with that person’s perpetrator that was not intended.

Q26            John Nicolson: I am also gay, so when you tell me about inherent risks that is always a concern for me.

Finally, can I come to you, Antony? What do you think, as a Committee, we should be recommending to the Government that they do to keep us all safer?

Antony Walker: An excellent question. The reason why I think this Committee and the inquiry you are undertaking are so interesting is that very often when Parliaments are scrutinising legislation they scrutinise a particular piece of legislation—for example, the online harms Bill, which in itself is a very broad piece. What is interesting about this inquiry is your ability to look across the perspective of a more connected world and think about the risk of harm, the risk of misuse and cyber-security issues either for an individual, for a business or for the state as a whole.

The crucial thing for me, not least because we have a lot of legislation coming through—legislation on competition, data protection and so on—is our ability to look at this. All these pieces of legislation are extraordinarily complex. What we need to do sometimes is step back and understand how these pieces of legislation work together. Do they work together effectively? Or are we sometimes layering on too much complexity, which stops you from getting to the end point that you wanted to get to?

I think you have a very interesting holistic viewpoint, and I would urge you to look at this from that perspective and to start to see where some of the dots—different bits of legislation or administration—need to be joined up, but also to look at where some of the complexity can be taken away to make things work in a more streamlined way.

Q27            Jane Stevenson: I am going back mainly to Dr Shi, because I want to talk more about tech in education and the concerns that you raised in your submission. Can you briefly outline what are the most commonly used applications at the moment in schools? What sort of things are children getting involved with in edtech at the moment?

Dr Shi: There are mainly two types of education technology being used. One type is school management apps. School management includes learning environments, such as Google Classroom and also Canvas—applications like that. On the other side, you have technologies that help people to learn or improve skills. You can think of Duolingo to learn a language. Those are the two common ones.

There was an increase in both types of apps or technologies during covid and the school closure period. Perhaps the more influential ones are the school management apps, because they restructure schooling and have the power to change power relations between users and also between users and the company. If we only have a limited amount of energy and resources, we need to focus on that.

It has been argued that, for example, very large providers, such as Amazon Web Services, at some point will have more power than the Government and the states because they are offering the entire infrastructure—the entire cloud infrastructure, platform architecture—to schools and also to individual smaller edtech companies, so they are in control of rules and structures. Rules will then be directly, or are directly, inscribed in codes—the rules of schools that are using the software.

Q28            Jane Stevenson: Thank you. You have talked about the inability of children to give proper informed consent and also about there being no opt-out for them—they are either in school or they are not. Do you think this is a design issue with tech that has been used? Are your concerns with the design of what is available now, and do you see a way that these things can be designed out so the technology is safer to use?

Dr Shi: Yes, it is a design issue and also a regulatory issue. As mentioned before, simply opting out could be the default. We have changes in pension schemes where we have to actively opt out of something. Here I think it is very similar but maybe opt-out should be the default, and by default not everyone’s data should be included.

There are also of course design issues. The designers of education applications or education technologies are mainly engineers and not so much people with education in their backgrounds. Very often, teachers are blamed because they do not know how to use those tools, but we need to look into having experts who understand something about education sitting in the education technology companies.

Q29            Jane Stevenson: That sounds like a gap in the market for an entrepreneurial teaching body.

Whose responsibility ultimately should it be to ensure that education technology is safe to use? Is it for school governors? Is it parents? Do you feel that parents know about these risks now? Is it a responsibility of the school itself or the individual teachers?

Dr Shi: That is a good question. I think control and power should be distributed equally among stakeholders, but currently we do not see much involvement. Parents and students are not often being asked, so their data is automatically collected. So to that extent I think there should be that involvement.

Again, talking about responsibilities, I can see two places. On the one level, maybe the macro level, there should be a safe regulatory framework that ensures data is being handled safely. On the individual level, individuals can also do something. For example, through education people can be informed about what is happening with their data.

However, I do not think that individuals should be made responsible for everything, so that one can just say, “Oh, it is your fault.” We do see quite a lot of differences within society, and the most vulnerable or minority groups are being disadvantaged. Information distribution is also not equal across society.

Q30            Jane Stevenson: If we stop data storage, data harvesting with edtech, is this going to disincentivise companies? Is that where the money is for them, or is it at the point of sale to the schools? Would it get more expensive?

Dr Shi: That is a very good point. A lot of educational products are free and there is no financial cost to the Government and to them being used by schools, but the cost is actually the burden on the children, because it is paid with their data. Currently the business model of many tech companies is handling data, so data is the currency. We need to look into that. One good way to address this is to have financial resources instead of using data as a currency.

Antony Walker: If there is an environment where we should be able to get the use of technology right, I think it is education. First, that is because it is about children, and we have a responsibility to get it right. Secondly, schools are a controlled environment. We have the potential to set national policy and to work in a way that helps to cascade information down to schools, academies and others to help them do the right thing. My sense is that this is an area where we should be able to get it right, but there are some big questions that we must think about, all the way from a very top-level ethical point to quite a detailed point.

Coming back to the point about data harvesting, it is a mistake to assume that data harvesting in itself is a bad thing. We can use that educational data for all sorts of positive outcomes in terms of helping us to understand how to meet the diverse needs of different types of children, provide more personalised learning and so on. The question we need to focus on is: what is that data being used for? What do we think is acceptable and that we should support, and what do we think is more high risk and that we should therefore mitigate against?

Q31            Jane Stevenson: Is it about anonymising?

Antony Walker: In many instances a lot of that data will be anonymised. It is not an either/or. There are times when you need to understand that that data is about that particular pupil. There will be other times when you want a very large dataset that gives you a sense of what happens with children at this age, how they perform with these kinds of tasks and so on. I think we should not look at it in a binary, good or bad way, but get into what the purposes are that we are very comfortable with and would support, and then what the purposes are that we think would be negative and that we want to push back on.

Dr Shi: I totally agree that we should look into what the purpose is. This is very important, and I have two points. The first is about data and anonymising data. There is a huge effort to do that, especially with health data, NHS data, but colleagues also at the Oxford Internet Institute have shown that it is not totally anonymisable—you can always match back to people. Maybe you cannot tell who the individual person is, but in many cases you can. By having other data, such as health information, and by matching across different datasets you can find out who the person is. Anonymising data in many cases is not enough.

The second point is about the purpose, which is about control of the data. Who has control over the data? What is currently happening is that as soon as data is extracted from people through their behaviour, you lose control over that data and it is monetised by companies. So they are in control and own your data. We need to talk about ownership and control of data.

Q32            Jane Stevenson: I could question you about this for about four hours, but I am sadly not allowed to do so. One of the potential benefits of edtech and online learning is in developing countries. Is this going to become a data war globally about who gets their edtech out in Africa and developing parts of Asia? Is there money in that? Do you think it is going to happen? Should it be up to developed nations to send in our knowledge and help? Do you think there are potential issues?

Dr Shi: Yes, I do think it is up to countries and people in the countries, rather than imposing something. The most important factor is context. It always depends on what kind of context there is—about the structure in a country and whether you can use certain technology or not. Something that may work somewhere else may not work in a different place. I think there is too much of a tendency to say that technology can help us in everything—techno-solutionism. In many cases this can be problematic.

If we just look at the UK, we have a problem in schools with underfunding. Underfunded schools also have the problem that the ratio between teacher and student is suboptimal. We have too many students and too few teachers. The techno-solution here is that we can use technology and surveil students. Teachers can observe students on one screen. It is very neat. We can have a really large classroom with 50 students and you can have a neat dashboard and observe students.

However, there are many issues, and one issue, of course, is data collection. In this case we anticipate that poorer schools may use more technology because poorer schools are more underfunded. Private schools have the money to have one-to-one education, so there is less monitoring going on in well-funded schools. On a broader scale, in society, we can say again that underprivileged people are being more heavily monitored than people who are more well off.

Q33            Jane Stevenson: Potentially, it would stop disadvantaged pupils falling through the gaps, if they are falling behind. It can flag them to a teacher. Do you think the Government are looking at the benefits rather than the risks? Do you think the balance is not quite there yet?

Dr Shi: I would say we need to focus more on the risk. The benefits are always very bright and shiny, and we can say that we have done something, but what are the risks? I would emphasise that.

Q34            Jane Stevenson: If there were a regulation saying that all data collected from children had to be immediately erased and that no one could own it, would you be comfortable with the tech that is coming in? Is it just that storage thing?

Dr Shi: Here I would echo what has been said previously. It is not just, “Let’s not collect or harvest data at all.” As academics we also collect data. We love data. Here the purpose is of course different. What is the purpose? Is it a commercial purpose or not? Who owns the data? Who controls the data? I would not say, “Okay, let’s just stop everything,” but we do need to think carefully about the power relationship between the data collectors and the data points.

Q35            Jane Stevenson: Finally, a slightly different question. Silkie and Lulu, you do not have smart speakers in your homes. Do you have a similar mobile phone to me, and is that listening to you 24 hours a day anyway, or do you turn off settings that enable that?

Silkie Carlo: Yes, I do have a smartphone. Absolutely, there is potential for a lot of data collection and leakage. Unsurprisingly, I am very careful about my privacy settings and Big Brother Watch has organised what we call crypto-parties, where we have taught people how to adjust their privacy settings.

I detected maybe a hint of defeatism in the question that you posed—that since there is a level of intrusive surveillance that we live with already and opacity about where that data goes, we have already lost the fight. I don’t think that is the case at all. I spoke about the precipice. We are looking at massive expansion of ever more intrusive and pervasive technologies that collect data about us, while also considering legislation that removes some of the most basic protections for people to control that data. There is a lot that can be done, not only to preserve the data protections that we have but to build more so that we are better equipped for this change.

Dr Shi: I echo this. I am also very careful about privacy settings. I am aware that I am already leaving huge data evidence through my online behaviour and so on. The second point to add is that, even if you do not have a huge data footprint, it is possible to find out about you anyway, because other people share their data. If I have certain information about you, such as your age group, your gender and maybe your education, I can basically build a proxy of you using data that is collected from other people, so I can create an accurate profile of you even if you are careful. By that, I want to emphasise that everyone should have some responsibility, but we should not be counting only on individuals for responsibility. It needs to be an overarching framework.

Q36            Dr Rupa Huq: I want to come back to Leonie, to some of the stuff you were saying at the beginning about the unintended consequences of some of these devices that were meant to be labour-saving and to make life easier and safer but that have this sinister dimension. You started listing the common ways that perpetrators can use connected devices in controlling behaviours, domestic violence and sexual violence. Can you say a bit more about that? I think you said it is impossible to say how widespread it is, because domestic violence happens behind closed doors, but presumably it is on the rise.

Dr Tanczer: I can give figures. I know people love figures. Different support organisations account for tech abuse, which is a big bucket. I want to stress this, because tech abuse is anything that people account for. It is the use of smartphones, social media applications and smart devices. Some say they have figures between 75%, 85% and, as I said, 100%.

Again, because we all have smartphones, if any abuse happens through the smartphone and other associated technologies, of course there is then a tech element there. We are currently working on a study to extract data from police forces and other organisations, to go through and check what devices are flagged in these records. We are working on this, and it is quite a tedious process currently, but it is on the horizon that we have an exact figure. I think that will then make it easier to say that, hypothetically, in 20% of cases Google Home is used. That is a hypothesis, but I am just say that that will make it easier.

Q37            Dr Rupa Huq: It is psychologically damaging as well, because it messes with your mind. You feel you are going mad.

Dr Tanczer: It is gaslighting, if anybody has watched the play or read the book. You are in your home, which should be a place where you feel comfortable and safe. It is so common nowadays.

If I could jump back on something that was talked about earlier, my biggest fear—I said this earlier—is that we are basically creating an environment where we are telling students, young children, that it is totally fine that their teachers and parents are monitoring them. They give them a smartphone and put stalker or spyware or parental control software on it. Then the children grow up and have their first partner and they wonder, “Why is it not normal for me to know where you are all the time?”, “Why are you suspicious of me installing that software?” or, “Why do I not get your password?”

That is my worry about what society we are creating if we say that it is normal for our kids to be under surveillance and it is okay with my partner too. It facilitates financial abuse and, of course, physical abuse, because these devices can be manipulated, but also sexual abuse—for example, image-based abuse offences. You can have a one-night stand and go home with someone, and they can have that camera installed, but you do not know that. All of that is possible.

Q38            Dr Rupa Huq: Dr Yvonne Yardley from Birmingham gave us evidence and she talked about omnipresence. Is it quite often the man in a conventional relationship that has control over these things—locks, lights, thermostats.

Dr Tanczer: Anyone who is thinking about their home environment—their parents, their brothers, their sisters—there is a gender dimension to tech. Even in my household, there is a certain dimension of that. In a heterosexual relationship, one partner—often the man—is responsible for the decision of what to purchase, how to maintain it and when to dispose of it.

Because domestic abuse is extremely gendered, that then exacerbates the possibilities. You start off in a heterosexual relationship, perhaps with someone who says, “I found this cool Alexa app,” and they install it. You have it in your house, but things can change, and suddenly it is hard to extract yourself from this device because you perhaps do not have the credentials. You are not the legal owner either. You are not the account holder so you cannot just phone up Amazon and ask for it to be removed because your account is not connected with it.

There is this discrepancy in digital literacy as well, around genders. A lot of the people we work with say they are not tech savvy but we all must be tech savvy nowadays. We all have smartphones, so that is the challenge here.

Q39            Dr Rupa Huq: In terms of this asymmetry of power and relations, I did an amendment on buffer zones around abortion clinics. Quite often women who use those are livestreamed on Facebook, or the Metaverse. An abusive partner can do that. This is just another dimension if they can control every single aspect.

Dr Tanczer: Omnipresence is a good description. Nowadays with technology it is hard to extract yourself from a relationship. Previously you may have gone to a refuge, started a new life and moved on. However, as Lulu said, my friends take photographs of me and then I have to make people aware that they should not post where I am. They should not tag me in pictures and so on. That is the challenge.

These things are only going to be exacerbated with IoT, because people now need to think, “If I go into a refuge, is my smartwatch still connected with my device?” Interestingly, people have found that women are often detected in the refuge through their Netflix account because they forget that they are still connected when they log in at the refuge. It is these things that women are not thinking of, and of course they aren’t.

Q40            Dr Rupa Huq: There is the Snapchat app where all the kids, including my son, can see where their friends are. He is always saying, “Oh, so-and-so is in Tuscany now”—they have finished their exams—“and I am here in Acton.” Is this being taken seriously? How have the Government and the law responded? Could you build it into non-molestation or restraining orders that as well as not coming near someone, people also disconnect apps? What could be done from a Government perspective?

Dr Tanczer: We did respond also to the call for evidence on new controlling and coercive behaviour statutory guidance as well, which is a different Committee. There are certainly elements that could be improved in this regard that perhaps go beyond the capabilities of this Committee, which involve the police taking this more seriously.

The problem is—and we hear this repeatedly—where police say to just go offline. You cannot go offline. My life is dependent on being online. Whether you have a public profile or it is your job or public services, you cannot expect that anyone does this nowadays. We need to find a solution where people can remain online and have a presence online, which is necessary for their livelihood. That is one issue. Police need to take this seriously and not consider this as a less horrible thing.

When we think about domestic abuse, we have this image of a beaten woman sitting somewhere cowed. That is what we see on the BBC or when we see an article about this—a woman with bruises somewhere. As we were saying, we also need to think about the hidden things—coercion and control—that are not as visible but are just as haunting and daunting. Tech abuse is one element of that. It is harder to detect, because that requires me to hand in my device. We saw that people cannot be analysed on devices, so they cannot go to prosecution saying that this is happening.

The police are a big issue, but so too is aligning policy. For example, cyber-crime legislation is barely used for domestic abuse conviction. We currently have a project looking at how the Computer Misuse Act is used for these offences. As a researcher, of course, data collection is the core thing, and if I can tell that, in 20% of cases, it is this manufacturer, we can invite the manufacturer to stand to account for that.

Q41            Julie Elliott: This is fascinating. Leonie, do you think regulation on data and product security and so on is failing victims and survivors of tech-enabled abuse?

Dr Tanczer: I know the Bill that is being rolled out, and I think it is great. As part of PETRAS we have been working on helping to support the development of 13 principles, but what they are focusing on primarily is very technical things, such as encryption settings and vulnerability disclosures. What I need to stress is that the average tech abuse perpetrator is not a hacker. They are not doing anything sophisticated on the kernel. What they are doing is using existing features that are there by default, such as camera installation and remote control—I have a smart speaker at home—and that people pay to have at home. That makes this hard to regulate, because it is a dual-use issue.

That then boils down more to the question—I think Silkie and Big Brother Watch could talk more about this—of what we, as a society, are comfortable with and, again, what is normal. Is it normal that I know where my child is all the time? Why would it then not be normal for my partner to know where I am all the time?

It is a very unsatisfying answer, but the Bill is good and basically addresses the low-hanging fruit of the security issue, but it does not necessarily address the type of abuse that is happening through a tech abuse perpetrator, because this is about functionalities rather than encryption levels, vulnerability disclosures and so on.

Q42            Julie Elliott: What you are describing sounds very similar to what we heard when we looked at the online harms Bill, although it is a different context, about things that are legal but harmful for different people in different situations. Your submission says that diversifying products on the market can mitigate tech-enabled abuse. Why do you think the market is currently failing to deliver this outcome, and how could Government address this in practice?

Dr Tanczer: I am so glad you brought this up. This is my pet peeve. One of the comments earlier was about big tech—are we scared of it? Also, can we stop data collection? We cannot, because you cannot make inferences, which is the golden nugget of knowing something, without having collected data. What I think would be important is ensuring interoperability and data portability. That would also break up big tech.

We have done tests with IoT devices, and we published a paper on data portability. We have a legal right in GDPR on data portability, which means that I can take my data out of, for example, a Fitbit provider and bring it into another, similar smartwatch provider. Most likely they will give you the data because the right to access has been very well implemented, but data portability has not because most providers use different standards for the way that they organise data, which means I am locked into that provider.

Our argument is that you can diversify the market by having standards that all manufacturers must follow so you are not locked in with Google, Amazon, Apple and Microsoft, but can go to a smaller provider that might be far more privacy-friendly or have other perks that might be better. That is what we have not addressed yet. Currently it is the big tech corporations that set the technical standards—they do it in edtech. That makes us buy into the whole ecosystem.

You have people that have all the Apple products, and people that are fully sold into the Microsoft environment. That is what we need to break up. That will help people to move, from the domestic abuse perspective, away from certain products to ones that might be more privacy-friendly.

Q43            Julie Elliott: People do tend to stick with one product for all sorts of reasons—mainly for somebody like me it is because you learn your way around it and you feel comfortable using it when you are not particularly techy. How can we make changes and how can Government make changes to break that up—to make that happen in practice?

Dr Tanczer: With data portability and interoperability. The Government in the UK are very active in standards setting, but so are the big tech organisations. I am currently on an IEEE working group, and there are all the big players on that board. They will of course have a certain interest in how these things are standardised. The way we are thinking about tech is so dominated by the way big corporations have told us how things must work. There is a very cool research project called Databox. That would mean that data processing could happen in the household, but that could never be marketable because a big corporation will buy it up or block it. That is where we are at.

How you incentivise is of course—as the Government have already done—with funding for start-ups. The big problem is that now they are basically bought up the minute they could have something interesting, and that means nothing new comes out that is not approved by big tech. The only way forward would be making some form of requirement for data portability to be a mandatory thing.

Q44            Giles Watling: There was a television programme some time ago called “Spooks” and there was all this imagined tech in it. Now we see that it has come to light and is happening. They had this expression when somebody was not available that they were “off the grid”. One of your most chilling pieces of evidence just now was about the normalisation of tracking. Therefore, if you say, “I do not want to be tracked,” the thinking is, “Why? What are you up to?” I find that chilling.

Going back to an earlier point made by Jane Stevenson, I don’t know why we are so scared of these connected devices in our homes when we have these smartphones. I am sure if I shout, “Hey, Siri,” loud enough, half the devices in this room will start listening to me. We are living in that extraordinary world. Are the Government doing enough? I get the advantages, and I get the points you have just been making—I am addressing this to Leonie—but it is a balancing act between what is good and helps humanity, and the chilling side of it. What do we need to do? With legislation which way do we go now? It is a big question.

Dr Tanczer: That is a question for an MP to answer. My immediate reaction would be data portability to diversify the market and interoperability. That would not just address issues like domestic abuse, which, as I said, is a far more nuanced issue. Basically, it also helps to break up certain locked-in dimensions. With regard to domestic abuse and tech specifically, or being tracked, again that is not necessarily something that is a simple technical solution; that is a societal problem.

As everybody can hear I am not from the UK. I am from a German-speaking country. Coming here I was surprised how comfortable people are with certain things that, in Austria, people would go nuts about. I think we need to have a public discussion. We are currently in this environment where people refer to “Black Mirror” or “Spooks” or going to Samsung to see what things in the future look like, which gives us a very hyped perspective of the future. Most of the time, these devices are extremely dumb—they still cannot understand my accent.

In that regard I feel that we should propose a David Attenborough of IT, to raise this idea of what digital literacy should look like and have a public debate about it, because consumers often feel helpless and buy in because that is all they can ask for.

Q45            Giles Watling: From what you are saying, it is education rather than legislation?

Dr Tanczer: No, I also think legislation would be good, especially with regard to data portability. That is in the GDPR but it is not really enforceable.

Q46            Giles Watling: I would like to make the point that we MPs are guided by listening to people such as you. Any other comments on that?

Silkie Carlo: I echo again that to be at this point in time and looking at watering down data protections is an act of madness. We should be looking at building them up rather than emaciating the foundation that we already have in law.

Q47            Giles Watling: Thank you. That is a point well made. In a previous incarnation this Committee looked at a company called Cambridge Analytica. You might recall that it collected the data of some 87 million people and affected the way they voted, and equally affected the way they wanted to purchase things. How do you think these devices in our homes, which are listening to us at all times, can start to influence us in those sorts of ways? Is that possible? Do we know? Antony?

Antony Walker: I would challenge “listening at all times”, because they are listening when they are activated, so they are not sitting there listening. Voice-activated devices are very easy to use and have huge utility—they are literally hands-free, so you can be doing something else. That is one of the things people find so useful about them. There are interesting questions about what happens if they become the default for search. For example, when you search on the web, you will get 10 or 20 answers to your search, whereas it is harder to have that multiplicity of search results when you are responding—

Q48            Giles Watling: You mean you are more vulnerable to misinformation?

Antony Walker: Or simply less choice as a consumer. You are presented with less choice. I know, for example, that that is one of the things that the Competition and Markets Authority is thinking about. It is thinking through what would happen if this became the predominant route through to search.

We have been talking about risks. I think the risks in this discussion can feel overwhelming. The solution to that is to break down the risks into their component parts and then think about what you can do about them.

Coming back to the issues around domestic abuse and so on, these are extraordinarily complex situations about human interactions. In some of these areas the solutions may not be with the tech. The solutions need to be about how we come in to support people in these situations, how law enforcement works in these situations and how law enforcement better understands technology in these situations. Sometimes you do not always have to come at it by layering on another bit of regulation to tech. Sometimes it is about recognising that this is a societal problem and working out how to address it.

Similarly, as new forms of abuse perpetrated online come along, you have the potential to make those activities illegal. Sometimes we only come at it from a technology perspective. Sometimes we need to say that there is a new phenomenon in society and that it is a bad thing, and we need to make it illegal, and then it is clear for everybody where we have drawn the line as a society.

Q49            Giles Watling: So that we just know where we are.

I have one final question, for Silkie: do you think that the ICO is effective in regulating connected tech?

Silkie Carlo: As someone who makes more reports than most people to the ICO, my perception is that it has a massive workload and it could certainly do with more resources, both to investigate data breaches and also to enforce.

Q50            Clive Efford: Thank you for coming to give evidence to us today. Dr Lulu Shi, can I go back to the questions my colleague, Jane Stevenson, was asking you earlier? I picked up something that I found very disturbing, about collecting the data of young people in education. You said that quite often the service is provided free because it is the data that is valuable. What is it that is valuable about that data that would make a big tech company want to provide that free service to education?

Dr Shi: There are multiple perspectives as to how data can be valuable. First—this is maybe less about data than about the locked-in effect—if you use one company’s product, you are very likely to be locked in. Capturing students or children at a very young age is very effective because they will be using this product, and that is then connected to data collection. If you have a long observation of a person, starting from a young age to a later point, this is very valuable because you can then observe how a person is behaving, and then you can also match that with other datasets.

In terms of how it can be used, it can be used for future employers. It can be used for Governments. It can be used for law enforcement. It can also be used, for example, by insurance companies. If you have a very longitudinal dataset, you can have so many data points and use that data to make predictions on how a person is likely to behave.

As we know—perhaps from the A-level fiasco—predictions can be very dangerous. Predictions are made by drawing on your data—for example, your socioeconomic background—and then inferring how you are likely to do in the future. This is basically the opposite of meritocracy in some sense. In this way, data can be used by multiple actors for all different purposes.

Q51            Clive Efford: Can this data of young people be kept for as long as when they go out into the world to become an employee? Might that affect their ability to find a job?

Dr Shi: I cannot say that this is true for every product, although we do observe that it is true for many products, but the data is not being deleted, and then it is up to the company how it is going to handle this data, because currently we do not have a framework that can help us to control what is going on with the data, once the company is in control of it.

Antony Walker: That is one use of the data—

Clive Efford: It is a pretty big use.

Antony Walker: Well, that is one potential use—and you could argue, in some cases, misuse—of the data, around profiling. A huge amount of the data will be valuable in terms of developing a product. If you are trying to understand learning outcomes, and trying to understand the effectiveness of the educational tool that you have built, you are going to need data to come back to work out how the changes to the service have improved the learning outcome for the individual. A lot of this data is going to be necessary to improve the service. Some of the data could be very fundamentally useful for giving feedback to a school—for example, where there might be a need for an intervention from a teacher or where a teacher might benefit.

I come back to the point that we should not simply use the terms “data collection” and “data aggregation” and assume that all of that is for a purpose that has negative connotations or could bring risk. Some of it could, but my point is to focus in on the risk and take a risk-based approach to thinking about the implications of this data being gathered and the extent to which it is appropriate for it to be stored, and about the purposes that it may then be used for, which we might say it should not be used for. For example, you might say that we don’t think it is appropriate that data that was used and gathered around a child’s education at school should be available to a recruitment company when that person is in their 20s. You could say that that is not appropriate.

Q52            Clive Efford: Would you say it is not appropriate?

Antony Walker: I think there are risks, and I think you need to explore those risks and come up with a decision. I have a child going through education. I can see where edtech could be beneficial for her. I can also see outcomes where I am not very comfortable with that.

Q53            Clive Efford: Would you be concerned if the data that was being collected on your child, who is going through education right now, and on the choices they are making at this moment in time as a young adolescent, let’s say, should then affect their life chances in future?

Antony Walker: Yes, and that is my point. Those are the kinds of questions that we need to get into—not whether using and gathering data is good or bad. It is more about what the potential outcomes are and which of those are appropriate and which are not.

Dr Shi: I do agree that there are certainly positive aspects where you can use data for learning, but we do also observe the risk. For example, data can currently be used to detect aggression in children and aggression in schools. What this product does is based on e-mining methodology—so catching sounds. It can detect when someone is shouting or has some sort of aggression in their voice. This is tuned towards the majority of people—people without an accent or who are not on the autism spectrum. We have seen cases of, for example, children on the autism spectrum or from migrant families who are more likely to be detected as aggressive. We need to understand the background, and just relying on this kind of technology flagging this up is not enough.

It can be very misleading because often the data is captured in a massive amount, and there is not enough resource to understand the underlying issues and the reasons behind it. People tend to just believe in data and at the dashboard. People will face long-term consequences, and these consequences are also not distributed equally across society—it is usually the minority groups who suffer from this.

There is an ethnographical study—it is not a quantitative study—that has been done in the States asking questions about how school admission officers use data. There is one super interesting quote that summarises everything: “Be more male, be more white and be richer,” because those are still the data that have the most predictive power.

Q54            Clive Efford: I want to ask about smart cars and micro-transactions. I am not sure which one of you would like to answer this. On smart cars, we seem to be opening up a marketplace for add-ons or additional services that the car can provide. BMW just announced that it can offer additional services connected to its ConnectedDrive Store, including heated front seats at £15 a month, a heated steering wheel at £10 a month, automatic high beams at £10 a month and Driving Assistant Plus active cruise control at £35 per month. You buy your car with all the hardware for all of these attributes attached to the car and then you have to pay for the software to access those services and you seem to be paying in perpetuity. When I bought a car in the past everything that was attached to it was part of the car and I owned it and used it.

Steve Brine: How naive.

Chair: That was in the days of the crankshaft, was it?

Clive Efford: I certainly learned to double declutch, yes. Is this right? Should we be going in this direction?

Antony Walker: We will see whether there is a market for that. It is a relatively new innovation. I do not think we know whether consumers will adopt it.

Q55            Clive Efford: Yes, but there have been accusations that, when cars are sold second-hand and passed on, the technology can then switch off parts of the car and make the new owner repay to gain access to those parts of the car that have been switched off. Again, is this a technology that we should accept when people buy these vehicles? Should we accept that the people who create the car decide, “We are going to charge further down the road again for reconnecting this car to this certain piece of technology”? What is going on here?

Dr Tanczer: That is an ethical or moral question, and also a question around what kind of economy you want. As Silkie said earlier, we are moving away from a PC environment, where a personal, connected device like your phone is yours and your laptop is yours, to a shared environment where a smart speaker is in a shared home and whoever is in there may or may not have to use it and deal with that. As a society, we have not figured out if we are comfortable with this or the leasing aspect of these services.

I know Antony said the market will solve this, but I actually wonder if the market will. If you have recently tried to purchase a car, it is impossible to buy one that does not have software in it. The only way to do that is to buy a second-hand car that is old. In terms of the market, it is not like the consumer purchases this and that means they want it. Looking at BMW, Volvo or whatever, those are all the options that are available. In German we have the saying, “Between plague and cholera”—so you just choose the least evil. Is that the way forward? That is a question for society. Do we want that? If the market feels this is valuable, I am sure they will do it.

Q56            Clive Efford: Does this open up the opportunity for unscrupulous garages and car dealers to access this software and suddenly switch off parts of a car and get people back in for bogus repairs and things like that? Are we not in danger of creating a whole black market in car dealership and repairs and maintenance? Silkie, is this an area that you would be concerned about?

Silkie Carlo: We have looked at connectivity with cars before. Quite plainly, this seems to me a commercial exploitation aspect. It is slightly on the edge of our remit as a privacy organisation, but, certainly, what you are describing is the inclusion of features that are for profit and, as we discussed earlier, changing the nature of ownership. It also requires that there is a connectivity and data exchange element that does not necessarily need to be there. The car is capable of having the feature, but data monitoring and the contract that the individual has determine whether it is accessible to the individual.

I am highly uncomfortable with this in lots of different scenarios with lots of different products, where people are owning them less and the products are owning them. In particular, the privacy aspect needs to be considered. Having a car that collects data on you is an incredibly intrusive thing. It shows everywhere you go and how often. There is a rich dataset there that people should be able to exert control over. Yet now, as Leonie said, pretty much all modern cars essentially have inbuilt tracking, for different reasons. I know that people have contacted us at Big Brother Watch and said that they are uncomfortable with that and that there is no way that they can exert control over it.

Q57            Clive Efford: Do you have any evidence of insurance companies, for instance, getting access to that tracking information and using that to set premiums and determine if somebody should be insured or charge more to insure?

Silkie Carlo: I have not researched that recently. It has been a few years since we looked at this issue, but I very much imagine that there will be evidence. Others might know more.

Antony Walker: There are insurance companies that offer products, particularly to younger drivers, where they have tracking devices and it is a means for those younger drivers to get cheaper car insurance. I think there is quite a lot of data that suggests that those people become better drivers because they are much more cautious. There is a positive outcome there that must be taken into account.

Dr Tanczer: It feeds into that discussion of certain groups, who may not be as affluent, who may have to pay with their privacy for the services. We have seen smart cars being hacked, and there was a case in Australia where a victim was affected by this.

Q58            Clive Efford: A ransom, where they had to pay to get their car?

Dr Tanczer: They were driving and features of the car were remotely switched off. Again, when it comes to tracking, there are opportunities that smart connected cars are offering. As with any technology there are avenues for abuse and misuse in this regard.

Q59            Chair: To follow up on cars—you touched on this—the cars are listening to you and your conversations as well. In my car, if I say something, occasionally it says something back to me, which is rather disturbing. I often use an expletive, and it does not recognise that. Is that something quite concerning—cars that listen to you? Where does that data and conversation go?

Silkie Carlo: That is a good question. The other factor of course is the location tracking. I think it is a safety feature. I forget what it is called, but it is inbuilt into most cars now. It essentially means that your journeys are tracked, which we are far too casual about. That is quite a serious thing.

Q60            Chair: Can the police access that?

Silkie Carlo: I don’t know. Certainly, we are seeing more cases where police access data from all kinds of smart technologies, whether it is water and electricity usage, recording devices, all kinds of technologies. The starting point for that in the UK has not been encouraging. One of the areas that we have worked on is in relation to complainants of rape and sexual offences, where people have routinely had their phones taken and downloaded and then they have been subject to investigation. The way that we have started off with smart technologies being used in a policing context in that case has, frankly, been disturbing.

Q61            Chair: You mean fishing expeditions?

Silkie Carlo: Absolutely, yes.

Q62            Chair: Could the same thing happen with your vehicle usage as well?

Silkie Carlo: Not just vehicles—unless a grip is got on the situation, which is hopefully happening now, and in fact, there is a deadline for the consultation on the Police, Crime, Sentencing and Courts Act today, with regards to digital extraction—but all kinds of areas. The new provisions in that Act are with regards to digital extraction from devices. Certainly, I think the policy has been drafted with phones in mind, but it absolutely could apply to any device.

Q63            Chair: Obviously, there is a counter-argument here that, effectively, the police or the investigative authorities need whatever information they can get. They already use CCTV. As you have mentioned yourself, there is the mobile phone data—the way you turn your mobile phone off and on and so on. We see all that, and I think Giles referenced “Spooks”—that is a constant refrain about his mobile whenever it has come on. If the police are here to protect us and that is their role, surely we should give them all the weapons we can to do so.

Silkie Carlo: Police need evidence. They do not need all information, especially if you think about the amount of information that each of us is now generating. It is a vast amount; it is incredibly detailed about your life. If we were to follow a path whereby if you wanted to make a crime report about anything you were expected to open up your digital life to total scrutiny, that would be such a curious thing. Why should you have to do that? So much of that information would be irrelevant. The extraordinary fact is that that is what has happened, particularly with gendered crimes—rape and sexual offences and domestic abuse. It has completely clogged up the system, and police do not know where to start. That has been—

Q64            Chair: Too much data for them to sift through. That is probably one of the reasons why we have the lower rate for follow-through.

Silkie Carlo: It has absolutely obstructed prosecutions, keeping people in the system, and it has effectively obstructed justice in one of the most important areas where justice must be sought. That has been very painful. I do think we are getting nearer to a breakthrough, but there has been a very painful four or five-year stretch of time where there has been this stasis with police and this awful practice.

If we see that applied to other areas of connected devices where all information is sought, you now have TVs that collect information, let alone home energy management, listening devices, cars and all sorts of other things. There needs to be a much more precise and evidence-led approach taken, rather than a view that all information is valuable in those cases.

Antony Walker: I agree with all of that. I think there is possibly a broader point here. When you are thinking about the remit of your inquiry, maybe one of the questions to ask is how policing and criminal justice keep up with this more connected, digitised world and work effectively when there will be all sorts of new technology—in terms of abusive relationships and so on, where policing needs to catch up with how those kinds of crimes are changing. Also, we need to think about the safeguards and about making sure that the police do not overstep in seeking data in situations where they probably should not be seeking data.

Then there is a huge question about police resourcing. We still have a focus on police being out in the community on the beat, but when so many criminal activities are now taking place online, maybe we should think a bit more about how the police are equipped for policing that environment, both in terms of the safeguards but also of more effective policing and criminal justice outcomes, and better support for victims.

Silkie Carlo: I have one follow-up point. I was thinking about the off-grid comment earlier from “Spooks,” which I loved. This Saturday in central London the Metropolitan police used live facial recognition cameras. This is an example of an area where police use of technology and data collection has completely outstripped legislation, because there is no explicit legislation for the use of live facial recognition. They scanned the faces of over 36,000 members of the public and had not one match. They had one incorrect match and that individual then had to prove their innocence to the police.

I have been on a number of observations of the use of this technology over the past couple of weeks and I have seen this happen. In fact, I saw a French exchange student who was asked for his ID, who he was and where he was going. He was held for about 20 minutes because he was a false match on the system. It is extraordinary. It is something that desperately needs to be examined in Parliament. Frankly, I felt ashamed when I spoke to the boy afterwards—the one that I saw being misidentified. He said, “I know that they have this in China. I had no idea you had it here.” A lot of people don’t know.

The future idea of smart internet and connected technologies and how the police might use them is on a vast scale. I think live facial recognition is on the dystopian end. Let’s not do that end of the scale. Yet it is something that has unfortunately completely evaded parliamentary scrutiny—there has never been a full House of Commons debate about this.

Q65            Chair: Yet your description there shows how it is far from foolproof. You witnessed two false positives during the two incidents you mentioned, and they did not find a single criminal.

Silkie Carlo: They have had some arrests, but at the cost of scanning millions of people’s faces in what is supposed to be the capital of one of the world’s leading democracies. This is the kind of thing that is used in Russia and China. It has not been used in the same way at all across the rest of Europe and is being banned in many areas of the US. It is a prime example, unfortunately, of how technology is outpacing legislation in this country.

Q66            Chair: I presume it is being banned in the US at the state level, rather than the federal level, because you said “areas of the US”.

Silkie Carlo: Yes, but the public pressure has been such that even the companies have started to prevent law enforcement using their technology, like Microsoft and Amazon.

Q67            Chair: I presume that when they are doing that, the legislation is specific, rather than part of a catch-all about connected tech and data. Is it specific to facial recognition, seen as a particular potential harm?

Silkie Carlo: Yes. As Antony says, what is interesting about this inquiry is how broad it is and that it brings in multiple pieces of legislation. Coming back to data protection legislation and the Human Rights Act, here are two foundational pieces of law where, for everything that you are looking at, that lens applies. At the time that this inquiry is taking place there is uncertainty around those legal frameworks. With facial recognition, for example, there is no explicit legislation. Parliament has not even debated it and, by the look of some of your faces, you did not even know this was happening.

Q68            Chair: You are looking at our faces and you recognise no knowledge at all.

Silkie Carlo: The one thing that has been able to apply, which was used successfully in the legal challenge against one use of facial recognition, is the Human Rights Act, and of course data protection legislation applies as well. There are going to be some very serious consequences if we start watering down the foundations, let alone being slow to act on legislating in areas where it is really needed.

Chair: Thank you. Antony Walker, Dr Leonie Tanczer, Dr Lulu Shi and Silkie Carlo, thank you very much for your evidence today. That concludes this session.