Public Administration and Constitutional Affairs Committee
Oral evidence: Work of the UK Statistics Authority, HC 781
Tuesday 23 May 2023
Ordered by the House of Commons to be published on 23 May 2023.
Members present: Mr William Wragg (Chair); Jo Gideon; John McDonnell; Damien Moore; Tom Randall; Lloyd Russell-Moyle; and John Stevenson.
Questions 1 to 42
Witnesses
I: Sir Robert Chote, Chair, UK Statistics Authority; Sir Ian Diamond, National Statistician, Office for National Statistics; and Ed Humpherson, Director General for Regulation, Office for Statistics Regulation.
Examination of witnesses
Witnesses: Sir Robert Chote, Sir Ian Diamond and Ed Humpherson.
Chair: Good morning and welcome to the Public Administration and Constitutional Affairs Committee. Today the Committee is holding an inquiry into the work of the UK Statistics Authority.
This is the first such session that the Committee has held since the appointment of Sir Robert Chote as chair of the UKSA and the reappointment of Professor Sir Ian Diamond for a further term as national statistician. This session comes at an important juncture for the statistical world, with continued debates around the use of statistics in political discourse and discussions on the future of the census currently under way. Could our panel introduce themselves for the record?
Sir Ian Diamond: I am Ian Diamond, and I am national statistician.
Sir Robert Chote: I am Robert Chote, and I am chair of the Statistics Authority.
Ed Humpherson: I am Ed Humpherson, and I am head of the Office for Statistics Regulation.
Q1 Chair: My first question is for Sir Robert. At your pre-appointment hearing, you told us that your goal would be ensuring that the statistical system is, and is seen to be, trusted, transparent, responsive, collaborative and innovative. Now that you have been in post for almost a year, have you managed this?
Sir Robert Chote: If it has been managed, I do not claim the credit for it. First of all, it is a great pleasure to be back here before you for the first time since the pre-appointment hearing. As you say, I have been in post now for coming up to a year. During that time, I have had a chance to meet people working right across the official system on both sides—ONS and OSR. Ian and I have met lots of groups across the Government Statistical Service as well, and we engage with the broader statistical ecosystem.
On those criteria, first impressions are positive. On the trust issue, that is tested relatively formally by a survey on public confidence in official statistics that is conducted independently of the authority by NatCen. The most recent numbers show that public trust in both the ONS and its statistics is high—in the high 80s per cent.—among those people responding to the survey, and that compares well with the trust numbers of other institutions.
That probably reflects a number of things. One thing to say is that it is particularly impressive that those numbers are high—and this is well before my arrival—at a time when awareness of the statistical system is greater and more widespread as a consequence of both the census and all the activity around covid. The fact that many more people are aware of what the statistical system is doing and still have high trust in it is a credit to the leadership and the work of everybody across it.
One important factor on the ONS side is the openness and communication of the numbers and responsiveness to questions around those. That is a key factor in trust, and there are also the casework interventions by the OSR. Some of those are in the public domain, and I am sure we will come to that, but at least as important, if not more important, is the work in the iceberg below the water of the engagement with statistics producers that does not necessitate public intervention.
On transparency, again, it is a very positive story. I am sure we will come to the work that the OSR has been doing in promoting what we call intelligence transparency, which is ensuring that when Ministers and others are using high-profile data and statistical outputs in the public domain, people are able to see where those come from, find them, see what the context is and put greater detail and understanding around that. That transparency agenda is a very important one.
Another issue on transparency is the way in which the ONS communicates. The fact that you can now tune into the “Today” programme and listen to informed statisticians from the ONS talking about the numbers, being very clear and engaging with that—that is a world away from when I was a nipper and an economic reporter back in the 1990s. You rolled up to the CSO press room at 11 o’clock, and the first anybody would see of this was the following morning when you wrote stuff in the newspapers. That has been a huge advance.
Another thing that is important is around the transparency of the organisations themselves; for example, the OSR producing more information in its business planning and report on where it is allocating people to different functions, which has made for a better sense of where the resources are being allocated in case of need.
On responsiveness, the ONS has admirably demonstrated its ability to respond to urgent and emerging policy needs. The covid infection survey was probably the largest and most dramatic example, but there has also been analysis on Ukrainian refugees, the richer data available on the cost of living and issues around the participation of over-50s in the labour market. You can see a lot of areas where that responsiveness comes through.
To be responsive, we have to be open and engaged with people. One thing that I really enjoyed in my first months in the job was an event to mark halfway through the “Statistics for the public good” strategy, which was an opportunity to get a lot of stakeholders at a single event, both in person and online, to discuss where they thought the strategy was performing well and where it had some way to go. That is key to that.
On collaboration, obviously there is a lot within the Government system—Ian is head of the GSS and of the Government Analysis Function—and sharing good practice and data, and making sure that the whole is greater than the sum of the parts are key elements of that. The Data Science Campus is a good example of collaboration with the private sector to bring in data—for example, on financial transactions or mobility—and the covid infection survey is again a good example of engagement with Oxford and so on.
Innovation is covered in quite a lot of the examples I have already given. I have been a consumer of and a stakeholder in ONS for 30 years before coming into this role, and for most of that period “nimble” is not a word that I would have attached to it immediately. I think, however, that you can see that through the covid period and in addressing the other subsequent shocks and needs—where Government need to know things, or where the public need to know things, to understand society—the ONS has stepped up. It is early days, and I don’t claim credit for any of it personally, but I am very impressed with how it has performed on those criteria.
Q2 Chair: Thank you. You told us that you would be looking at how the board is functioning. What have you found, and what changes have you made to how the authority operates?
Sir Robert Chote: It functions very well. The major change is that we have had a change of personnel, so we have three relatively new non-executive directors who have come on board and are settling in very well: Penny Young, who will be familiar to parliamentarians from her work in the Library; Carol Propper, a distinguished economist, with whom Ian and I have both worked in the past, when I was at the IFS, for example; and Jacob Abboud, who brings business technology project delivery experience.
A key question for the board is ensuring that we have a good mix of skills across that. I should pay tribute to the NEDs who left. To be frank, there has been a policy to remove NEDs after serving three years only, which I think is regrettable. I think good practice in the public and in the private sector should be to allow NEDs to develop understanding and capability. We are letting the executive off a bit if we remove people too quickly. I am not sure that is going to carry on, but hopefully not.
Chair: That is relevant.
Sir Robert Chote: I am very pleased with the people I inherited to start with, and with the process—the Cabinet Office ran the NED recruitment and appointment process well.
On the functioning, one of the issues that is raised perennially—you raised it with me at the appointment hearing—is the fact that the authority and the board span both the ONS and the OSR. Some people would say that that is a slightly odd structure, because regulator and producer are under the same umbrella. I think that is actually a positive, as I explained to you beforehand. Chairing a board that spans the two ensures that in the highly unlikely event that ONS tries to give OSR the run-around, I am in a position to stop that. That has not been the case, but having that together is a strength relative to any other system that I can think of off the top of my head.
It is a very collegiate board. The relationship between the executive and non-executive members works well. There is an appropriate balance of challenge and support. We are very lucky that the NEDs we have will have particular areas of expertise that they can engage with the executive on—Sir David Spiegelhalter, for example, on trust and statistical methodology. It is not just about the contributions that he is making during a board meeting; it is about what we can draw upon more generally than that. The board regularly reviews its effectiveness.
As I say, I have not quite been there a year, but they have been there a full year. There is also a review of governance within the ONS, so we will want to think about frequency of meetings, types of meetings and how we do the balance of that. But at the moment, I am very encouraged by what I have seen so far.
Q3 Chair: I am told that your second permanent secretary is due to leave shortly. Are you intending to advertise for a replacement for that role? I don’t know if Ian Diamond has anything to say about that.
Sir Ian Diamond: Thank you very much, Chair. It is great to be here; I am looking forward to it very much. I am not intending to advertise in the short term. We are faced—we cannot hide from the fact—with financial constraints right across the public sector, and we are no different. We believe that, certainly over the next three months, we will restructure just a little bit and continue to deliver, and we will review in three months.
Chair: We will no doubt come on to the topic of finance in due course but, for the moment, thank you very much.
Q4 Damien Moore: This is to all three. Three years ago, the UKSA launched a new strategy aimed at providing high-quality data and analysis, improving lives and building the future. What progress have you made so far?
Sir Ian Diamond: I think we have made very good progress, if I may say so. It is worth remembering that the strategy was written just before the pandemic and published just after the start of the pandemic, so I would not claim that we have done absolutely everything. On the other hand, we have done many, many things that we did not expect to do and done them, I believe, very well and right within the major pillars that we set ourselves.
Let me just expand on that a little, if I may. We have been incredibly agile and radical in our use of different data sources. In 2020, at the beginning of the year, I would not have envisaged a major ONS survey that collected blood and swabs from large numbers of people at the same time as collecting information on their social habits. I would not have expected a weekly survey that took attitudes or a fortnightly survey of a large sample of businesses, which have enabled us, in an agile way, to find out how businesses were responding at that time to the pandemic, and how they are responding now to other issues. I would not have expected the way in which that has been used right across Government and outside.
That agile collection of data has been important. We are using many, many different sources of data. Traditionally, things were done using surveys. We now make much greater use of administrative data and born-digital data, and we work very closely with different providers. For example, we have very good weekly data on consumer spend, which we bring together through partnerships with the financial transactions sector—really radical data.
Secondly, we have been ambitious. We wanted our data to be used, to be useful and to be relevant. I would argue that we have worked very hard to understand the important questions to which one requires answers. It is too easy for a national statistics institute to sit and think, “I wonder what people would like?” and then make up something and throw it metaphorically across a wall and hope somebody catches it. No, that is not the way forward. I would be keen to know that we have engaged right across Government, and outside of Government, to understand the questions to which people wanted answers, and then to ask whether we have that data, or whether we need, in extremis, to collect data.
I would argue that we have been ambitious. The higher profile that we have in the public, as reflected in the data that Robert just gave to you, I think reflects that. I submit that at the beginning of my term, an awful lot of people in our country would have wondered what the Office for National Statistics was. Now, I do not think that that is the case so much at all, which has been very, very good.
Critically as well, we recognise that, increasingly, data should not just be national. We believe fundamentally in local data for local leaders to make local decisions, or to inform national decisions. We have put in a lot of work into subnational and granular data, right down to what the statisticians would call “lower super-output areas”—to most people that means areas with about 400 households up to about 1,000 households, so really small areas—and understanding the data at those levels. That enables you, if you want, either to understand local data, or to build them up into your own geography.
We have worked hard to be inclusive and passionately tried at all times to make sure that every citizen has a voice in our data. We have set up an inclusive data taskforce, and we have Dame Julia Cleverdon chairing for us a kind of challenge panel to make sure that we are building inclusivity into everything.
At the beginning of your question, you mentioned quality. Underlining everything, we believe, is getting the quality right. We have been prepared to revise statistics where we have said, “Okay, we don’t think that, historically, we were doing this as well as we should have been.” At the same time, we have built new panels to make sure that our design is good, that the collection is appropriate and that we can, metaphorically, look you in the eye and say that we are comfortable with the quality of what we produce. I think we have done very well. I will not pretend that we have finished, or that we do not continue to have challenges.
Clearly, we are at a moment in time when technology is getting ever greater. We need to seize the opportunities that technology brings, while at the same time maintaining privacy, maintaining cyber-security, and maintaining, if you like, ethical use of data. We still have challenges around access to data from different sources. This is perennial. It is not the first time that I have said this to you. I hope that it will be the last, but I fear that it will not be, as it remains a challenge.
We bring in some of the best brains in our country to work at the ONS, but they then tend to go elsewhere, so retaining our skills is an incredible challenge. At the same time, we are working to increase the diversity of the people who join us, using things like apprenticeships—personally, that is something that I am passionate about—and looking to build our offices in different environments.
We have expanded our offices to Newport and Southampton. We now have offices in Darlington and in Manchester, where we are able to increase the diversity of our workforce pretty well. I mentioned earlier that we have resource constraints. I recognise the public finances, but clearly we need to work in the future to do everything that we can, but make the case, I would submit, for doing a little more.
Finally, I mentioned inclusivity as a plus. Inclusivity remains a challenge. I am absolutely clear in my mind that once you say, “We are inclusive,” you have lost the game a little bit, because you must constantly work to improve your inclusion, so we continue to do that.
Ed Humpherson: If we go back three years, the OSR as an independent, separate part of the authority was in its relative infancy. In the strategy three years ago, we set out our ambition to mature from infancy. If we were to improve lives and build the future, as the strategy had it, we recognised that we needed to do more than simply assess a series of individual statistics on a rolling basis. We needed to identify ways for the statistical system to improve. We needed to be responsive to emerging concerns, and we needed to be willing to stand up to defend the appropriate use of statistics.
We needed to be strong, independent and effective. In my view, we have achieved those things. We have been strong in the sense that we have not ducked difficult issues. We always seek to clarify the evidence on issues like migration and asylum or the use of employment statistics. We are independent in the sense that we look at issues on their merit, not on which organisation is raising them. If you look at our track record, we will take on issues wherever they come from, be it the UK Government or the Scottish Government, or indeed highlight areas where the ONS itself can improve. I think that we are effective in the sense that we do not just highlight flash points; we want to drive change.
Sir Robert has already mentioned intelligent transparency. That is our response to a problem that we saw cropping up in lots of different places. Rather than play whack-a-mole trying to address the issue of transparency, we created a campaign, we created guidance, and we engaged with the top of Government to create a concept of intelligent transparency that can influence practice across Government rather than in an individual case. In my view, over those three years we have become stronger, we have demonstrated our independence, and we have been effective.
Sir Robert Chote: There is not much that I need to add to that. I would simply say that having inherited the strategy—very much chiming with what Ian said—I think the pillars and objectives of it, the notion that we are aiming to promote statistics for the public good, the radical, ambitious, inclusive, sustainable elements that Ian spoke about, and the characteristics of the system that we started off with, with the questions, look robust and good. We should always keep things under review and come back to them, but there is no point in having unnecessary logo bonfires when you have a good set of criteria by which to guide the institution.
The key thing, as Ian said, is that you are doing a lot of things that you did not know that you would need to be doing back in 2020. In order to ensure that you will continue to be ready for the next unexpected or unknown thing that is going to come around the corner, capacity, capability, vision, and a good approach to people are absolutely central. I think that the infrastructure is in good shape. It is never something to be complacent about, and neither organisation is, but that framing sense of what we are here for and how the bits of the system work together to try to deliver is in good shape.
That is very important in delivering the day-to-day ongoing outputs that people have relied upon for years, and to be able to respond to the thing on which in three months’ time we will suddenly discover there is a really urgent need for data and insight, and we do not have it and we need to get it quickly. As I said, that is one of the things that I have been so impressed by: the way the system has performed in the period immediately leading up to me joining it. It is very important that we keep that.
Q5 Damien Moore: Sir Ian, we are coming to the end of the ONS strategic business plan 2022-23. You have mentioned quite a long list of things that have been going well; you also mentioned some challenges. Is there anything that has gone less well that would not be in that challenge column? What will your focus be over the forthcoming year?
Sir Ian Diamond: To be honest, some of those challenges are what I would say have gone less well. We are not as far on as I would like to be with our access to some data, and I do believe that some data providers do need to spend just a little time moving things forward.
Secondly, I mentioned technology as a challenge. It is a challenge and we have been challenged on a number of occasions regarding being able to access incredibly large datasets and do the matching that is necessary to build that. Nothing has gone badly wrong, if you like, but those things are taking a little longer to push forward than I would have liked. I would like to have moved much further forward with the use of passport checks to measure migration, for example. Now, we are moving there, but it has taken longer than I would personally have liked.
As I said earlier, retaining the very best people is a constant challenge. We do everything that we can, but, sometimes, some of the issues of salary do really play a role for people with the very top digital and data skills.
Q6 Damien Moore: Thank you. Can you tell us a bit more about your plans to develop an integrated data service, and what has been—
Sir Ian Diamond: Yes, that is a really major—this is so exciting as a project. Let me be clear: when I say that we are going to bring enormous amounts of Government and other data together, that is not into a big data lake. The datasets will be held where they are at the moment, but we will be able to access them through catalogues that enable us to link different datasets together.
The proof of concept has already been demonstrated during the pandemic. To give you an example, we were able to understand some of the very sad inequalities in mortality during the pandemic by being able to link death-registration data with census data to understand ethnicity, for example, with other health data to understand comorbidities, and with other survey data to understand other issues.
By doing that, one was able to look at the impact of socioeconomic status, housing status and ethnicity, while controlling for comorbidities and region. That really had an impact on the way that policy could be developed and in our understanding of a really sad and difficult situation.
However, you cannot do that from any one data source, nor from any one data source held within one Department. Therefore, the idea of the integrated data service is to enable data from very many different places to be linked and for us to be able to answer important policy questions that statisticians such as myself could have only dreamt about until recently.
We have been working very hard on it. We are in a beta phase at the moment, which is enabling some work to be done. For example, we have a project that is looking to understand the distinction between the reported speaking of Welsh in the most recent census and the estimates of Welsh speakers that come from some of the Welsh Government surveys. There is quite a gap, and there are various hypotheses that you can make quite easily, but being able to do some linkage and to bring a programme that understands that that is an important policy area for the Welsh Government is something that we are able to do at the moment. It will be, I hope, fully operational during this year.
We are just in the process of applying for Digital Economy Act accreditation, which is an important step. We are confident that we will gain that accreditation, and we are also confident that not only will the integrated data service exist qua itself, but it will be able to link into one or two other trusted research environments in different environments. We work closely with, for example, Health Data Research UK, which also works in the same sphere, to maximise the ability to work together. I see this as being a really exciting, transformational opportunity for the use of data to inform policy for all of our citizens in many areas. We would have liked to have done this in the past, but we simply could not put everything together.
Q7 Damien Moore: Moving on, there is a very limited set of data currently available on the platform. How confident are you that when the time comes, owners across Government will agree to make the dataset available to that platform, and which Departments are particularly important in this context? From your dealings—you have already mentioned Health—are any Departments particularly problematic? Please feel free to name and shame them if there are.
Sir Ian Diamond: Oh no, no!
Chair: Please do. We would enjoy that.
Sir Ian Diamond: I mentioned in my last response that we are just in the process of applying for Digital Economy Act accreditation. That is a critical phase, because central parts of that are your ethical procedures, your procedures for establishing whether the research that is going to be undertaken is in the public interest and, critically, your safety issues and security issues. I know that a lot of Departments say, “When you’ve got that, they’ll be fine,” so Digital Economy Act accreditation is critical.
I would just like to call out HMRC here—not, I stress, in a negative way, but in a truly positive way. Working with HMRC has been a real delight. There are many people in this country who would not necessarily agree, but it has been—
Damien Moore: You heard it here first.
Sir Ian Diamond: It has been absolutely brilliant. We are, for example, able to talk about the number of people on payrolls, bring in data from HMRC and link it together, and it works extremely effectively.
On Health, it does take time. We were able to make a lot of progress during the pandemic, but that was as a result of something called COPI notices, which enabled us to access data quickly when it was of pandemic importance. That has stopped now, and there is work to bring together what used to be called NHSD and NHSE. We work very positively with them. I am expecting that as they get their amalgamation going, we will have very strong links to be able to build—ethically and in a privacy-enhanced way— access to those data, which will be incredibly important for us to understand all kinds of issues around public health. As I say, when we get Digital Economy Act accreditation, I am expecting really positive access to other datasets.
The other area that I think is really super important to call out is that the Department for Education invited me to chair their unit for future skills. That has enabled us to work with them to really link together some really exciting data sets on school performance linked with subsequent employment and subsequent tertiary education.
Again, that will be moving imminently into the integrated data service, but it is an example of what you can do if you can link those data together and enable people to understand what the economic returns to different skills are and what the pathways that people take into different jobs are, so that we can look at, for example, people in particular jobs. Did all accountants come from accountancy degrees or actually, do many accountants come from engineering degrees, for example? I think that understanding those pathways at a truly national level is critically important. I do think, again, that bringing those data together with data from other sources and Departments will be critical in the future.
Damien Moore: Thank you.
Chair: Thank you for that. Let us move on to Tom Randall.
Q8 Tom Randall: I want to ask a couple of questions about the census. The 2021 census was a key deliverable for ONS over the course of UKSA's five-year strategy. To what degree did ONS meet the success criteria defined by the Government in its 2018 White Paper?
Sir Ian Diamond: It exceeded them. I mean, I am a bit of a census expert and a nerd, so I can talk for the next two hours on this. I will try to be brief.
John McDonnell: You did last time.
Sir Ian Diamond: Mr McDonnell has heard me speak on this. I will be very quick. We had a very high response rate. We did a very successful coverage survey, which we do to estimate both under and over-enumeration. We would estimate that we got a little over 97% coverage; we then topped it up. We have brought out a production of regular statistics since then, which have been extremely well received.
Also, for the very first time, we have brought out a tool that enables people to construct their own tables. That has been enormously well used, and we have had a number of other computing interfaces that have enabled people, for example, to look at their own area and at patterns in different areas. We are very pleased with not only our production of the census, but the public response to the use of those data.
To come back to Damien Moore's question about the integrated data service, we were able to get those data into the integrated data service and to be used at the turn of this year. That is four years ahead of anything that has ever been done before, so we are absolutely delighted with the whole of the census arena.
Q9 Tom Randall: Thank you. Speaking about integration, the ONS is responsible for the census in England and Wales, but UKSA has a broader interest in UK-wide statistics. When can we expect to see the results from the Scottish census? How are you working to reconcile that with the data from England and Wales?
Sir Ian Diamond: That is a very good question. As you know, the Scottish census was delayed, by a decision of the Scottish Government, by one year, and was undertaken during 2022. It would be right to say that there were some challenges with initial response. I am part of an international steering group that was set up in the summer of 2022 under the chairpersonship of Professor James Brown from the University of Technology in Sydney. The group met, initially on a weekly basis and now on a slightly less than weekly basis, to steer the subsequent collection through its coverage survey, and the estimation of under-enumeration is being done through a mix of the coverage survey and administrative data.
That is going well, and in the spirit of supportiveness the ONS has seconded one of our best census people, Jon Wroth-Smith, to direct the work in Scotland. He is doing a good job, and he spoke to the board at its last meeting. We have helped in a number of other ways, too. I fully expect the production and publication of the population estimates for Scotland to take place in autumn this calendar year. That will give us estimates for Scotland for census day 2022.
We will then work to use our statistical techniques to roll that forward to mid-’22—census day was March ’22, and then we move forward to June ’22. I very much hope that we will be able to produce UK-wide comparable estimates by the end of this calendar year, but certainly a few weeks after the publication of the Scottish data.
Q10 Tom Randall: Previous censuses provided a snapshot of the UK. Would you say that that is still going to be—
Sir Ian Diamond: Yes.
Q11 Tom Randall: Thank you, Sir Ian. Mr Humpherson, the OSR recently conducted an assessment of the census in Scotland. Would it be fair to say that you have some concerns about the quality of the data that might emerge?
Ed Humpherson: The exercise that we conducted is exactly the same exercise as we conducted for the Northern Ireland census and the England and Wales census. We do an assessment at the interim point, after the collection process is completed and before the production of the population estimates themselves. What are different are the requirements that we set. In all three cases—Northern Ireland, England and Wales, and Scotland—we set requirements at that point, we expect the producer to meet those requirements, and only if those requirements are met do we accredit the resulting statistics.
In the case of Scotland, we tailored those requirements to the context of the Scottish census, and in particular the fact that the original collection process did not meet its expected levels of coverage. We put in place a couple of requirements around how National Records of Scotland, which produced the Scottish census, will demonstrate quality using the alternative methods that Sir Ian has outlined. That is where the requirements focus. I would not say that we have any greater concerns than simply saying, “In this particular situation, we expect the producer to address the expectations we have,” which will then lead to national statistics accreditation.
Q12 Tom Randall: Are you able to say why it did not meet the coverage?
Ed Humpherson: Sir Ian, you might want to come in on this, but essentially National Records of Scotland set some expectations as to what level of response it would get across the population of Scotland, and there was a lower level of response, particularly in some local authority areas. National Records of Scotland then did a number of things to raise awareness. It extended the data collection period and did some follow ups, but even at that point, the level of coverage was lower than its prior expectations. That is the point at which the international steering group, which Sir Ian might want to come in on, has started to advise on ways to enrich the data it has by using different data sources. Sir Ian, do you want to follow up on that?
Sir Ian Diamond: I couldn’t agree more. The overall enumeration was not as high as the Scottish Government had set out because of their challenges. By extending the fieldwork, they got it up to a reasonable level, but not in all local authorities. There is then the question of how you account for under-enumeration. The judgment was that it was not going to be possible to account fully for under-enumeration simply using the coverage survey, which went out after the enumeration and was delayed because the fieldwork was extended.
Some quite innovative methods have been proposed that use a mix of the coverage survey and administrative data to be able to account for that under-enumeration. I am confident—this comes back to my earlier response—that they will get accurate estimates, but those estimates will be accompanied, as they would have been anyway, by estimates of uncertainty, so that we will be able to really have a look. I am confident that there will be population estimates for Scotland that we can be comfortable in using and that will be comparable with those we have for England, Wales and Northern Ireland.
Q13 John Stevenson: Sir Ian, earlier in this session you said that data should be used, useful and relevant. It also needs to be, I am sure you would agree, correct and accurate. We understand that the ONS is currently reviewing some of the data relating to gender identity, to understand the responses, because there appears to be a larger number counted as transgender. Should we be treating this data with caution?
Sir Ian Diamond: Not necessarily with any more caution than we said very clearly in our release on 14 April. This is the first time that we have collected data on this topic, and when you collect data for the first time there are always then questions about how accurate those data are—I remember when we first collected data on ethnicity and on nationality, for instance. It should be added that this is an area that is a relatively rare event, so to speak, so the numbers are relatively small, and therefore, again, one needs to be clear as to the accuracy.
The question, as asked in the census, was subject to extensive qualitative and quantitative testing, including being used in the census dress rehearsal. Subsequently, we have got the results, and we took a judgment to say that we will publish them but continue to work on our analysis. This is the same as we do with every question. We continue to do quality assurance on all our questions. We did, as we said, compare our estimates with those from surveys done by the British Medical Association, and they are broadly in agreement. We now have a programme of work that is going on to really look carefully at—
Q14 John Stevenson: You now have a programme looking at this particular issue.
Sir Ian Diamond: Yes.
Q15 John Stevenson: Was that a result of you making that decision, or was it a result of commentators making an observation to you?
Sir Ian Diamond: No. We said we said very clearly in our initial release that we would be continuing to look at the quality assurance of this. We recognise that others have taken those data and looked at them, but we initially said that we would be continuing our research in this area.
Q16 John Stevenson: Okay. Mr Humpherson, do you have any concerns about this?
Ed Humpherson: Users have raised concerns with us about this; as a result, on 23 April, we wrote to ONS to say that we would do a review of the way that these user concerns have been handled by ONS. We will complete that review in the course of the next couple of months and draw our conclusions at that point rather than now.
Q17 John Stevenson: In the meantime, should the data carry the national statistics designation?
Ed Humpherson: The national statistics designation covers the entire demographic package of the census collection, rather than giving an indication of specific number accuracy. At this point, I do not see any grounds to conclude that the overall census demographic package has not complied with the code of practice.
Q18 John Stevenson: Obviously there is an element of concern about this data in particular. Do you have concerns about any other data?
Ed Humpherson: Not at this stage, no. Of course, statistics are there to be used, and in the process of use, different user communities might identify things that cause them to ask questions. One of the great things about the census is that the underlying data are made very widely available, which can lead people to conduct analysis that has not yet been done and identify new patterns. It could be that people identify new things that raise questions, but at this point, I do not have any of those questions myself.
Sir Ian Diamond: Could I just say that we are always willing to engage with users when they have questions? We respond positively and, I hope, constructively with people when they raise questions.
Q19 John Stevenson: You are just confirming to me that you have no concerns about all the other data in the census.
Ed Humpherson: Yes, I am confirming that.
Q20 Lloyd Russell-Moyle: This particular dataset is only really able to be gathered through a questionnaire-style set of data. Does that make it more difficult because you are not able to compare it and corroborate it against other datasets? Therefore, is it important that you continue to investigate this data and find alternative ways to gather that very interesting information that might be accurate?
Sir Ian Diamond: You are absolutely right. One of the reasons for asking people questions is that sometimes, there is no other way to collect those data. We will, and we will continue, to look at alternative sources where we might get some information—for example, using health data or whatever. We remain completely committed to using every method that we have to make estimates and to understand trends over time, and to making those data as accurate as possible.
Chair: Thank you. Jo Gideon.
Q21 Jo Gideon: Sir Ian, in recent years, ONS has received a substantial injection of funding to deliver key surveys, such as the census and the covid infection survey. How successful have you been in ensuring that that funding delivered new processes and systems that can be used in the production of statistics beyond those two specific surveys?
Sir Ian Diamond: That is right. It is really important that one puts an enormous amount of effort into the technology platforms surrounding the census. That was an incredibly important thing for us. Equally, what you do not want to do is just put them together, use them and then say “That’s great,” and off we go.
We put a lot of work into saying that we wanted to have a transformational change in moving our surveys, particularly our business surveys, online. That is a really big advance because it reduces response burden, it enables more speed in the analysis, and it also reduces cost. We have used the census platforms and we now have 39 of our 77 business surveys online using those platforms. It has been extremely successful.
We have built platforms to deliver the covid infection survey. Our conversations are continuing with DHSC and UKHSA about how, in the future, we can maximise the technologies that we have built. and we will continue to do so. The other thing is that, overall, within the staffing resource across the organisation, people have had incredible opportunities to be able to work on questionnaire development and on different methods of analysis. That stands us in great stead when they move to different roles within the organisation.
Q22 Jo Gideon: UKHSA has committed to delivering 10% in baseline budget savings by 2024-25. You touched on staffing, but how will you go about this?
Sir Ian Diamond: It is worth saying that in year 1 of this spending review period, we have £7.6 million of identifiable savings, which are ongoing, and £1.9 million of one-off savings that we have identified. All in all, we are well on track to make that 10%. How do we do that? Well, there is a number of ways.
There is automation. You mentioned technology, as have I, and we continue to develop technology. I will not go into the details of reproducing analytic pipelines, but they are a fabulous way for us to ensure that we improve accuracy, while, at the same time, reduce the need for different people to work on different areas. That is a big area for us. Secondly, I mentioned the moving of surveys online, which brings efficiencies. We are also using this as an opportunity to rationalise our business surveys.
I have to say that, in the past, the number of business surveys that we do have increased organically. As different topics came in, we would say, “Well, we’ll have a new survey.” We have now said that we want to reduce respondent burden, make sure that we are properly representing what is going on in the economy and to only ask questions once. A major programme of reducing the number of surveys that we are going to do was reported to the board at the last meeting but one and will, again over the next period, bring efficiency savings in big ways.
The other way that I would point to is using increased numbers of administrative data and born-digital data, which reduces the need necessarily to do large surveys. We still have to do large surveys when there are things that you can only ask people, but reducing the need to do large surveys brings efficiencies. I am very comfortable that, by using technology and really rationalising what we do, we will be able to deliver within the envelope that we agreed with the Treasury at the beginning of the spending review.
Q23 Jo Gideon: I am interested in your statement that you will only ask a question once. Does that apply across Departments—for example, if you had asked a question within a health survey that might be appropriate to another Department? That is probably a bad example. Clearly, datasets are discrete to departmental requirements.
Sir Ian Diamond: In the past, one could not necessarily have done without some basic information that you would ask on every question. We still need some basic information, but I would argue that with the ability to link data together—particularly administrative data, where you have the data for everyone—you can reduce the burden on the respondent. That is incredibly important. Certainly, across our entire portfolio of business surveys, I would argue that on a number of occasions we have been asking the same question of the same industries or the same company. That is not helpful.
In order to have a representative sample of what is going on in, for example, British industry, you will always have some of the much bigger companies in your sample. If you put everybody on a list and take 10% of industries, and every company you come out with is an SME, frankly you don’t have a decent, representative sample, so you have to stratify.
Some of the very, very big companies—I don’t need to name them—will be in most of our surveys, so we need to work with them to look at what we asking them, so we are not just saying, “It’s us again, asking you to fill in a questionnaire.” That is going down well with our bigger companies. It reduces the burden, and we can then transfer data across and use it differently. We therefore have efficiencies, better accuracy and happier respondents. That has to be a good place to go.
Q24 Jo Gideon: Absolutely. Mr Humpherson, the OSR’s budget increased at the beginning of your current strategy to support an expansion of the regulatory function. What has that expansion achieved?
Ed Humpherson: The first thing to say is that, although it was a significant expansion in percentage terms, we are very, very small—a mere rounding error in the context of a larger organisation’s budget. We have increased significantly to about 40 staff members.
In my earlier answer to Mr Moore, I said that at a basic level of maturity, we would simply assess a series of statistics on a rolling basis in a planned way. That was what we had been resourced to do, and we continue to do it. We do about 10 assessments a year.
We do about 40 what we call compliance checks, which are faster reviews, and then we do what we call systemic reviews, which look at bigger ticket items. We do about five of those a year. We carry on doing that. What we have added with the extra resource is the ability to be much more dynamic and respond to emerging issues. We may come on to this, but we have had a very significant increase in the number of cases and the use of statistics in public discourse over the past three years; it has between doubled and tripled.
We have also been much more responsive to emerging issues. On things like the A-level and GCSE algorithms, we stepped in when those problems emerged, as we did when some very significant user concerns came up about the population estimates in different cities across the UK. Again, we were able to respond in an agile way without damaging our ability to do that core, ongoing work.
We have also been more systemic, in the sense that, rather than look simply issue by issue, we have been able to stand back and say, “What does this whole system look like?” We now produce a report every year called “The State of the Statistical System”. The extra resources have enabled us to do that. I mentioned the “Intelligent transparency” campaign. We did a couple of reports on the lessons learned from the pandemic—statistics and data in the pandemic. Those are reports and insights that help the whole system.
The final thing, which we were not resourced to do before, is that we are more public, in the sense that we think more about public attitudes, the notion of the public good, statistical literacy and the ways in which statistics go beyond expert discourse and impact on people’s lives. We didn’t have that research capability before. The expansion has allowed all those things to supplement this quiet drumbeat of improvement work that we do through our assessments and compliance checks. We have been able to supplement that in those ways.
Sir Robert Chote: I would just add that the board recently discussed a further modest increase in the OSR budget for its work on economic statistics, which is part of responding to Eurostat’s role not being what it was in the UK context. It is not just spread significantly more widely than Eurostat’s focus on the quality of some of the key economic statistics. That is being piloted at the moment with one particular study. We felt that it was important that OSR was properly resourced to do some more of that across more of the economic data. That is an answer to the same question the next time you ask it.
Chair: With that theme, we seamlessly transition to John McDonnell.
Q25 John McDonnell: A perfect segue. I have got a few economic stats questions for you, Sir Robert, and Sir Ian. Brexit and covid have had huge impacts on the economy. How much do you think official statistics have captured and communicated the impacts of those sorts of phenomena?
Sir Ian Diamond: It has been both an unbelievably transformational and a very, if you like, rocky time for data collectors, with so much going on. We have tried to use many different ways of collecting data. During the pandemic, for example, we rapidly transformed some of our ways of collecting education data and health data to get very accurate estimates of GDP in a way that very few other countries did. We have tried to maintain that agility.
I mentioned earlier, in response to Damien Moore, our fortnightly business insights survey. That has continued. It enables us to find out what is going on and how companies are responding. For example, one of the key issues at the moment is the use of artificial intelligence, and being able to find out which companies, portions of companies or types of companies are using artificial intelligence has been important. We have used our surveys in an agile way. We have also, as I indicated earlier, used a lot of new faster indicators, as we call them. I mentioned HMRC and the use of payroll data, which has been incredibly important to us in understanding what is going on at pace in the labour force. Historically, you might look back three months, but that has not been any use to Government at all in the last few years—we need indicators at pace.
I think it is always important to ask, do those types of indicators give the perfect answer to the perfect question? No, they don’t. It is incredibly important that we are transparent about what those data show. For example, when we produce data on the numbers of people on payrolls, they do not, of course, include the self-employed. If people—for reasons, for example, around IR35—have moved from being self-employed back into the labour force, it will not tell you about the overall labour force, but it does indicate some quick, accurate data under particular conditions and particular definitions. It is right that we have done that.
You mentioned Brexit and the pandemic. We have also had the cost of living, and responding at pace to cost of living has been important not only in making sure that we have accurate monthly inflation figures, as you will have seen this morning, but in responding quite quickly to enable citizens to calculate their own inflation.
Over a million citizens have used our personal inflation calculator. We have also done some special work to look at low-cost items, to try to understand the impact of inflation on people from different socioeconomic backgrounds. I think we have done well. I am not going to pretend in any way that everything has been absolutely perfect at all times, but we have responded very quickly.
We have also responded very quickly to where there have been particular questions. For example, in late autumn of 2021 we identified that there had been a flight from the labour force among the over-50s. The Government was extremely interested in that, but it wanted to know why, and what the conditions would be under which people might be prepared to return to work, so we stood up and provided two surveys.
The first one came with responses in four weeks to help inform the understanding of not only the reasons why the over-50s had left the labour force, but the conditions under which they might be prepared to return. Some of the work from those particular surveys has featured in subsequent Government policy. I think that we have tried to respond at pace to what is a challenging environment over the last few years.
Q26 John McDonnell: Sir Robert, do you want to comment at all?
Sir Robert Chote: The economy is always evolving, and you can never be complacent about the ability of the statistical system to identify those changes. On the one hand, obviously you have the public and users and—having been, for my sins, a forecaster for 10 years—the value of a consistent time series over time, but also responding to the new questions that come up. It is getting that balance right between ensuring that the underpinning architecture is robust and delivering for you, but also being able to say, “Well, actually, now this is a particular question that we need to focus on,” and the over-50s and the cost of living would be good examples.
Q27 John McDonnell: That leads nicely on to the next question. Occasionally, you have been accused of declaring a recession, then later down the track you have had to pull back and erase that. Do you think you have got the balance right between the quality and the timeliness when it comes to economic statistics?
Sir Ian Diamond: That is a really good question. International best practice around GDP—and exactly what the Treasury and the Bank of England want—is for us to give, if you like, the earliest indication that we can. Then, rightly—so it is not as if we are saying, “Oh God, we got that wrong”—as more information comes in, we say, “Well, actually we've now got better information; we need to move this a little bit.”
We published on 4 April a piece that looks at the last number of years. What we show is that over that period of time, we have gone up and down in about equal measure. We have gone from, if you like, plus or minus 0.3%-ish to plus or minus 0.1% or 0.2%, so we have actually got better. You cannot say, “Oh, they'll go up; they'll go down”—it goes in both directions.
I would say that you highlight a very important point. If the economy were growing at 3.8%—it is not, but let us just pretend—and we turn that to 3.6% or 3% or 4%, hardly anybody bats an eyelid. If we say 0.1% and then we go to minus 0.1%, is it really changing? I might argue that it is still flatlining, but because you are so close to that particular place, clearly it has greater attention on it.
In summary, we have evidence that we are getting better, and we have evidence that we do not have a bias—that we are going up and down—but when things are, as they are, very close to zero, it is not surprising that sometimes we are going to go over that particular threshold.
Sir Robert Chote: This is a key point. With the national accounts—again, having been a consumer of them for many years—revisions are a feature, not a bug. When you are trying to measure the size of the economy, you are trying to measure it in three ways simultaneously—the output of the economy, spending in the economy, and expenditure—and you are collecting information on all those things. Some of that information comes in quicker than other bits of the information.
The constant challenge is, as this information flows in, to see how that changes your overall picture. For example, when you are looking at quarterly changes in GDP, it tends to be the output measures that give you the earliest indicator, but when you are looking at income, you do not get anything close to a full page of that until you have self-assessed income tax returns, and you get VAT somewhere between the two of those. If you are looking at an initial estimate that is up or down by 0.1%, it is a very early draft of economic history, which you can reasonably expect to change, sometimes over long periods.
I think, as Ian says, the technology and the methodology around this has improved. The cheering thing is that there is no evidence of bias. Revisions are smaller than they were. I remember turning up to a Treasury Committee meeting 30 years ago, just after Ken Clarke's first Budget. The then official estimate of the downturn of the 1990s recession was that GDP had fallen by in excess of 4% and it had ended in a double dip. When I was at the OBR 20 years later and I was looking back at the same period, it was a 2% decline and no double dip.
Over time, you learn more. You could say, “We’ll just produce one estimate and that’s it,” but that is not the way to deliver an accurate picture. Obviously, as a forecaster, that was occasionally frustrating because you thought you had pinned the tail on the donkey, but you would then discover that the donkey was not where you thought it was. That is because you know more, you understand better, and you learn over time. As I said, revisions are a feature, not a bug.
Q28 John McDonnell: So as a poacher-turned-gamekeeper, is the balance right between quality and timeliness in your view?
Sir Robert Chote: Yes, I think it is. If memory serves me right, there is now one fewer estimate of GDP than there used to be, so there has been a change in that trade-off and I think we were content with that. Obviously, it depends in part on whether you are interested in the quarter-on-quarter changes. My professional recommendation was not to place too much attention on those as to the broader picture. That, I am afraid, does take time to clarify.
There is a very important message, which both the ONS and the OSR have emphasised, about how you talk sensibly about the uncertainty around outturn. When I was at the OBR, we put a lot of emphasis on the uncertainty around the forecasts, partly because of the uncertainty around the output data.
We wanted to just be clear with people about how the process works and the fact that you have a provisional view that changes. That does not mean that the provisional view was wrong or negligent. The forecasts are just based on an information set that gets larger and richer over time and the difficult technical task of getting three different measures of apparently the same thing to mesh together. It is not a straightforward business, but I think that it is conducted very well.
Q29 John McDonnell: Ed, how do you assess the economic statistics? How do you think they meet the views of users?
Ed Humpherson: We think about this as three tiers. The core expectation is the responsiveness: are the ONS responding both to emerging economic phenomena and to emerging user interest? We look at that. As we have already heard, there is a really strong track record there, particularly over the last two or three years, of ONS responding, identifying new issues, reporting on them, finding data sources to put people’s arms around them so they can understand and respond to them.
There are two other things. Upstream of that responsiveness, there is the underlying foundations—the quality. We spend a lot of time—it is rather unglamorous and unheralded—thinking about how good the core business surveys and core consumer surveys are. We do assessments of those. I have talked about our assessments a lot this morning. They are the core of our work. We look at them a lot, and there are times when we raise questions about those foundations and encourage ONS not to neglect effort there, as well as doing those fast-response issues.
Downstream of that, there is the sharing of these insights in the public domain and the communication. I think it is really worth saying—Sir Robert has mentioned it already—that there has been a real transformation in communications. We played our part in that because, in the past, statistics were produced and published at 9.30 in the morning. That meant that they missed the morning media round.
In the pandemic, as a regulatory sandbox, you might say, we allowed the ONS to publish at 7 am. After running that experiment for a year and a half we did a consultation and confirmed that we should allow that flexibility on a permanent basis. That is why we now continually have ONS experts speaking to the statistics—communicating publicly around the statistics—at a time when the audience is at its widest.
The other thing that we look for in the wider communications is the point of uncertainty. Some really important experiments have emerged about how to convey uncertainty. Essentially, the experiments demonstrate, and the ONS is building on these experiments, that the best way to communicate uncertainty is not just with words but with visuals. There are various ways: fan charts, fuzzy graphs, box charts and so on. They are better than long paragraphs that say in prose that something may have some uncertainty attached to it.
Those are the things that we look for. We look upstream at these foundational components of quality, and we look downstream at communications, but at the core we are very encouraged and impressed by the responsiveness that the ONS has shown. Probably five or six years ago I would not have said that.
Sir Robert Chote: One important message for producers is that being up front and talking about uncertainty around numbers actually builds trust; it does not undermine it. One should not be afraid of saying, “Oh gosh, this number could look different in the future.” Actually, if you take people into your confidence, show appropriate humility, and explain how this is going to work, that actually increases their confidence in the underlying robustness of the numbers and the processes rather than undermines it.
Sir Ian Diamond: There is a very good reason we call them estimates: they have some uncertainty associated with them. We work really hard to minimise that uncertainty, but we will not eliminate it.
Ed Humpherson: Just to build on both those remarks, when I talked about these experiments around communicating uncertainty, one of the really consistent findings—this is the Winton Centre at the University of Cambridge, the Economic Statistics Centre of Excellence, the University of Warwick behavioural insights team; they have all done this sort of thing—was that actually conveying uncertainty does not undermine trust and it appears to enhance trust. These are tests with members of the public.
Q30 John McDonnell: People will appreciate the difficulty of arriving at those estimates. A year ago—this is to you, Sir Ian—the ONS set out plans for developing the measures beyond GDP that we were interested in, to try to capture better economic wellbeing. Can you update us on the plans?
Sir Ian Diamond: Absolutely. In my opinion, this is incredibly important. I do not go quite as far as the late American presidential candidate Bobby Kennedy, who once said that GDP measures everything except that which is important. I do not quite go as far as that.
John McDonnell: It was the poetry of his daughter that he referred to.
Sir Ian Diamond: It goes down the line that way, so we are really working very hard. We will this summer start to bring out what we call inclusive income, where alongside economic capital we will bring out our estimates of some other capital.
Human capital is important, and that is going to be a mix of demography and education, so that we are starting to build models that enable us to look at the number of people moving through different educational systems so that we can build estimates of educational capital. We are already putting out a small portfolio of environmental indicators alongside GDP, but our environmental team is working on how we can calculate environmental emissions and produce them.
We are bringing in unpaid household labour estimates in the summer, which I think is incredibly important, particularly at a time when we are seeing many more people, particularly those over 50, in some of the surveys that we did saying that one of the reasons they are not working is because they are what we would call sandwich carers. They have elderly parents and children or grandchildren. We are making those estimates.
You mentioned wellbeing. That, I think, is really super important. We have done a consultation on wellbeing indicators, which we will publish on 6 July. If I give you a sneak preview, it will look at things like physical and mental health, social cohesion and personal finance—those are some of the key issues.
What we will be looking at is a subset of a number of indicators to build into GDP. We are also working with Lord Layard, who, of course, comes from a view that you only need to ask about personal satisfaction, and you can get very good correlations. We are working with him to see how best to bring these into our estimates of GDP.
I gave a speech recently on this. If colleagues were interested, I’d be happy to let the Committee have the text of that. But, in summary, this is moving at pace. It’s one of the most important things we are doing. The production of what we call “inclusive income” in the summer is going to be an important milestone.
Q31 John McDonnell: Terrific. You published your “Beyond GDP” programme last week—or was it two weeks ago? How do you think it is going to be used? How do you envisage it being used now?
Sir Ian Diamond: I very much hope that we will be leading globally. We are working with the United Nations to make sure that other countries are building on this. I very much hope that we will soon have international comparisons—that’s going to be incredibly important—and I very much hope that, over time, it becomes really well used by Government, who look at the economy as well as educational and social wellbeing, and that we are able to understand how people feel as well as their economic status.
I think there is a real potential for Governments to use this in very many ways. Our job is to provide them with the tools to be able to do that. I would very much hope that the media also gets as excited about movements in some of the human and wellbeing capitals as they do about some of the conversations we’ve already had about the economic movements.
Ed Humpherson: I would just add one thing: as these new measures beyond GDP emerge, I cannot wait to assess them and hopefully accredit them.
Sir Ian Diamond: Absolutely.
Q32 Chair: We are coming towards the end of the session. There are some questions to come from Lloyd Russell-Moyle, but let us come briefly back to me. In that spirit of tools that you mentioned, Sir Ian, the current strategy commits to developing a better and richer understanding of user needs as to how those tools might be used. How have you gone about that, particularly thinking of your key users, and how you reach them? How do you balance the users’ different needs—for example, those of policymakers and central Government, and members of the public?
Sir Ian Diamond: I might start with how do we get needs. Wouldn’t it be nice if there was an algorithm that I could just publish—that I could just press a button and it all came out? I don’t know how to do that, so what we have been doing is talking to people, which I think is the right thing to do. We put a lot of effort into talking widely to other Government Departments and to engaging with, for example, the Bank of England and some of our key user communities, such as Robert’s former organisation, the OBR, to really understand their needs.
In addition, though, we have a number of advisory committees of experts—an advisory committee on prices, an advisory committee on population, an advisory committee on research programmes. One thing I did was bring the chairs of all those committees together into what we call an expert users advisory committee. Professor David Hand, a former non-executive director, chairs that committee for us. The committee not only gives us advice but brings in outside speakers. We were privileged to hear recently from Tony Dent from the organisation Better Statistics.
We listen, then, to our expert users, but I do not think we can stick there—actually, not “I do not think” but I absolutely, fundamentally believe that we cannot stick there. That is why we have also introduced what we call the ONS Assembly. One thing we learned from the inclusive data taskforce that I mentioned earlier was the real need to engage with civil society organisations and local communities. So we decided that we should not just stop there with the taskforce, but form the Assembly, which brings together people from across civil society to have conversations with us.
We also engage directly with some civil society organisations, such as the RNIB, which is helping us with how we measure disability and other areas there. That is, we have wide-ranging conversations with a wide group, as well as using all forms of media to enable people to input to us.
How do we prioritise what people want? Clearly, there is a very long list of things. For some of those things, we just have to say that there is not enough demand; for other things, we can say, “Well, if we do things in a slightly different way, we can find some evidence.” At the end of the day, we have a difficult job of prioritising and we share, basically, the broad plan of what we are doing—our business plan—with the board.
As I indicated earlier, where there are special requests that we see as being important, we work with organisations to say, “Okay, let us see how we can provide you with data from the different sources that we have to help a report that you might do.” We are doing that at the moment in a number of areas—children’s rights, for example.
There are a lot of different ways in which we engage, but we cannot provide everything to everybody, because at the end of the day there is a resource constraint on what we can do.
Q33 Chair: On the theme of regulatory perspective, Mr Humpherson, the OSR has produced guidance for producers of statistics on how best to engage with users. Briefly, how well do you think producers are applying that guidance?
Ed Humpherson: It is mixed, of course. In some places, it is applied very well, and in other places, less well—
Q34 Chair: Can you give us areas where it is good and areas where it is bad?
Ed Humpherson: Yes. Generally, I would draw a distinction between engaging with known expert users who are visible and have a voice, and broader outreach to people who may be less easy to identify and access. Typically, producers are familiar and comfortable with the first, and will appropriately invest time in that, but if you look at our assessments over time, you will see that we frequently encourage producers to identify other user groups.
Of course, we are not expecting producers to have enormous statistical advertising campaigns to reach masses of the public—that is an unreasonable expectation. It is more to say, “Where are those pockets of interest in civil society where you might find organisations that are potential users of the statistics and might have a perspective, because the statistics are about their community or interest group, and that might then be able to enrich what the statistics say?” That is what we encourage more of.
Sir Robert Chote: May I add one point? You have argued in the past for the importance of users sharing their plans for development in future, and that gives users a chance to input before the concrete has dried.
Q35 Chair: Short of asking the public to sit an A-level stats paper—which is the extent of my understanding of the subject, I should declare—how do you evaluate whether you are improving public understanding of statistics as you set out in your strategy?
Ed Humpherson: When I give lectures and talks publicly, I have a story about how important it is to stand up for statistics, to speak out against misuse and so on. Invariably, I will get a question, which is, “Isn’t the problem here really that the public have a low level of statistical literacy?” I started to get a little bit frustrated with that being a recurring motif in the Q&A that I had, so we commissioned some work on the subject, essentially looking at the literature—what does the academic literature say about this question?
It turns out that while it is easy to say there is a deficit of statistical literacy, in practice what it actually means differs in different settings. Sometimes people mean there is a deficit of basic, foundational mathematical skill; sometimes they mean there is a deficit of understanding of statistical concepts; and sometimes they mean there is a deficit of critical thinking—the ability to be sceptical and question what is being asked. Given that those things are quite different, you would expect to have different treatments or different responses to them.
On the first—the foundational mathematical skills—there is probably evidence, which is reasonably robust, that there are quite different levels across the UK population. I would accept that. On the statistical knowledge, while it is true that people might not be able to say what a standard deviation is, you might say, “How much does that really matter?” On the critical thinking, I would say there is fairly good evidence that people are quite skilled, particularly if they have a small amount of exposure to advice and guidance. They can be quite sceptical and appropriately challenging of what they are told.
What that all gets us to is to say that, in a way, the problem of thinking about this sort of public literacy is not to say, “The public have a problem: they have a deficit. We need an enormous UK-wide education campaign.” It is more that the people who are responsible for communicating statistics need to bear in mind the things I have just mentioned. They need to identify the audience that they are communicating with. They need to think about the role of visualisations. As we have talked about already, they need to help people to understand uncertainty and understand that these things are estimates.
So what we get down to is to say, “Actually, this is as much a problem of communication as it is of education.” There are, of course, educational activities, which we might come on to, but it is very important not to get trapped into the idea that the public have a deficit and it is all their fault. That is the wrong way to think about it.
Chair: I would concur. Is there anything else to add on that?
Sir Ian Diamond: It is worth adding that I do not think statistics A-levels, which are wonderful, are the answer for everybody; it is much more about data literacy. I am absolutely thrilled that the Cabinet Secretary has chosen, as his one big idea, that the entire civil service will do some data literacy this year.
We are delighted to be working with the Cabinet Office in the preparation of the material for that. I say that not only because I think it is a brilliant move, but because that is the way that we as a nation ought to move, with more and more organisations and areas really pushing the boundaries of data literacy and understanding data. But for those who would love to, as a result of that statistics A-level, I say go for it.
Chair: I shall bear that in mind.
Sir Robert Chote: The only thing I would add is: do not underestimate the appetite for this. That is something that has been demonstrated quite clearly through the covid period, where you had national conversations—not necessarily always perfectly informed—about what the R number was at any given moment. Both among the general public and within the policy world, there is a greater appetite for trying to engage with these sorts of issues, and that is much to be welcomed.
Chair: That is certainly true. Final questions from Lloyd Russell-Moyle, please.
Q36 Lloyd Russell-Moyle: Sir Robert and Ed, you told us last year that you had received a great number of concerns about the use and presentation of statistics by public figures, researchers, Members of Parliament and Ministers. Has that trend of dobbing in people for the misuse of statistics continued?
Ed Humpherson: The trend has not only continued; it has increased. Let’s begin with some facts. Each time an issue is raised with us, we call that a case. Pre-2020, we were opening about 100 cases a year. In 2020-21, it went up to 323, so it pretty much tripled. In 2021-22, it was 241. For the year that we have just completed to March 2023, it was 372, although there were some duplicates within that.
Lloyd Russell-Moyle: So it has levelled out at about 300.
Ed Humpherson: Yes, it is levelling out. I suppose the question then is, what is driving this? Is this evidence of a tripling in the propensity of people to misuse statistics? That is probably not the right way to look at it.
First, the pandemic is clearly a factor. That big inflection point in 2021 was timed with the pandemic. We had a surge of members of the public coming to us with questions and not understanding things, not being able to access things and being concerned about things.
Q37 Lloyd Russell-Moyle: Was it that there was more engagement because people had time, was it that statistics were more relevant to people’s lives—the R rate and so on—or do you not really know the underlying cause?
Ed Humpherson: Of course, we do not really know in every individual case what is driving the person, but based on the content of what people were coming to us with, it was that people were seeing this national and global phenomenon of a pandemic being presented to them through the medium of data and statistics and really wanted to understand it. That is what drove an increase in awareness of statistics and data and, coupled with that, an awareness of the role that we play. Our awareness increased.
I talked earlier about being independent and strong. That is why it is so important that we want people to come to us, because they think their issue or concern will be given a fair hearing. That is the first thing to say—what is driving this is not necessarily an increase in statistical wickedness.
The second reason to think that is that not all cases are about misuse. A big chunk of our cases are about data availability. It is not that something is being misused; it is that somebody is concerned that they have encountered something, and they are not sure what the source is. That is what we call a problem of intelligent transparency. Around 15% to 20% of our cases are about data availability and data accessibility.
Some cases are about data quality, and that is not about how politicians use statistics; it is about the production of statistics. Last year, our largest case, which we had about 170 separate complaints about, was about a weights and measures consultation by the Business Department, because people were concerned about the way data were being collected. That is not really about use; it is about collection.
Having said all of that, there are cases where people raise concerns about political misuse, and here we are strong. We step in without fear. We highlight issues where a claim has been made that is not consistent with the underlying statistics—things like claims by the former Prime Minister on employment, claims about the backlog on asylum, claims by Scottish Ministers about renewable energy and misleading visualisations.
Q38 Lloyd Russell-Moyle: When you do intervene, does it get the response that you want?
Ed Humpherson: As a general rule, yes. Where we identify that there is a gap between what is being claimed and what the statistics say, by and large we find that the person who has made the claim is willing to accept our advice and modify what they say. There have been cases where that has taken some time and taken repeated interventions. At the moment, we are doing a lot of work on the use of data by Ministers in the Home Office, and we are getting good engagement at the official level. I think it is fair to say we are still getting a flow of cases there, but over time we tend to see behaviour modify.
Q39 Lloyd Russell-Moyle: When you highlight these things and you write a public letter, should Ministers officially correct the record, or is a letter back to you, or correcting behaviour going forward, enough? Does there need to be something in the sand that says, “This was a marker that was incorrect, and I have put that on the record”?
Ed Humpherson: Correcting the record, certainly in the context of this place, is a matter for the House. We really welcome it when corrections are made. In April, the Prime Minister made a statement at Prime Minister’s Question Time about there being a record number of people in jobs, and the very next day he corrected that, which is really good practice and we welcome it.
In fact, the way we think about this is not with the gotcha moment of the correction. We want to have a system, a culture, that takes data provenance and data accuracy really seriously. We would set more store by things not happening in the future, so that the misuse stops and the lesson is learned. Otherwise, the risk is that corrections become a tokenistic, performative gesture. It is much more important to establish the principles of good use and get sign-up and adherence to those.
Sir Robert Chote: In the letters that I do publicly, which tend to be ones that are more ministerial or to do with Members of Parliament, I am trying to set out clearly the particular circumstances of the individual case, so if somebody is confused about whatever the substantive issue is, they have it clearly set out for them.
The other thing is setting out what looks like good behaviour and expectations, and encouraging that in future. An important part of that can be ensuring that statisticians working in Departments are consulted by political offices and communications teams and brought in early to those sorts of communications, because they are likely to spot those sorts of issues and have a better sense of what needs to be done.
I guess another thing that cross-cuts with some of this is the issues that arise with people using social media to communicate quantitative information. It is in the nature of a tweet that those things are quite brief—it may be a chart, an individual statement or a couple of sentences—and nuance is not always possible. There is a need to think very carefully when you are having very brief communications on quantitative issues, so that they do not end up being misleading, perhaps inadvertently—it is just that you have written it up or presented it in such a crisp form that somebody else might not interpret what you meant to say correctly. There are a whole lot of issues around ensuring that the statistical professions are consulted. People should think very carefully about saying, “Well, I’m just going to chuck this out there because it looks nice and it broadly has the message right.” That is not quite good enough.
Ed Humpherson: I sometimes think that there is a life cycle to these things. Something begins as a robust piece of analysis, well-explained and well-grounded in sources, but it gets compressed into a more bite-sized format for communication purposes. It gets compressed into a tweet and then it starts to be cycled into the version of a tweet of a single number that gets weaponised. In my experience, when a number that has originally been drawn from official statistics gets used over and over again, it is almost always a sign to me that there will be a problem with it, because it starts to get detached from its context.
Q40 Lloyd Russell-Moyle: Do you provide guidelines to Ministers on how to condense those things into tweets, or is that a hazard that you do not really want to get into?
One of the most important of those is to enable people to get to the source. We understand that these forms of communication are compressed, but if you allow people to click through the compressed version to something behind it, you are going a long way towards being able to establish the trustworthiness of what you are communicating. So on those principles, yes, we share them freely.
Q41 Lloyd Russell-Moyle: Sir Robert, the Committee is currently undertaking an inquiry into the civil service people survey. We were recently told that the Cabinet Office do not consider that to be official data, so therefore they do not comply with your code of practice in its entirety. Do you think that it is the right approach for them not to comply because it is not official data?
Sir Robert Chote: I don’t know how long the people survey has been around. In its origins, I presume it is a management tool first and foremost. The fact that it is subsequently published does not seem to me to mean that you should not aim to do that to good standards. That might have various dimensions to it, for example the time it takes to get released and who it is released to. On that basis, aiming to do things as transparently, clearly and promptly as possible and with equality of access, which would be key elements of what you would be looking for in an official statistic, are desirable, whatever label you happen to put on it.
Sir Ian Diamond: I completely agree. I think it is a very well-constructed and thought-through survey. However, getting those results out promptly seems to me to be incredibly important.
Ed Humpherson: There is a timeliness question here.
Q42 Lloyd Russell-Moyle: So maybe if they did adopt it in its entirety it would focus their minds on some of the timeliness issues that you have raised.
Sir Robert Chote: If you were aiming to meet that best practice, that would include things around the timeliness of release, certainly.
Lloyd Russell-Moyle: That is very useful. Thank you very much.
Chair: I thank our three witnesses this morning for, as ever, a fascinating canter through the world of statistics and their use. We are very grateful. If there is anything further you wish to furnish us with—I know, Sir Ian, that you undertook to let us have a copy of your recent speech—that would be gratefully received by the Committee. For the moment, thank you all very much indeed.