Mike Keoghan and Grant Fitzner.

Q1 The Chair: Welcome to this session of the Economic Affairs Committee, where we are looking into growth and productivity statistical methodology. We are delighted to welcome our witnesses from the ONS, would you like to introduce yourselves?

Mike Keoghan: I am the Director General responsible for economic statistics at the ONS.

Grant Fitzner: I am the chief economist.

Q2 The Chair: Thank you very much for coming. We have a lot of questions to ask about this complex methodological area. Speaking on behalf of myself, and I am sure none of my colleagues, forgive me if I ask very simple questions to try and understand it.

I would like to start with Covid-19 and how the methodology has reflected that. Before we get into that, however, a number of us are interested in how we have managed the impact of Covid-19. That is the first bucket of issues. The second bucket of issues, looking forward, is where we go from here. The third is the more long-term trends in how we take into account the impact of the digital revolution.

Whether our questions will neatly fit into those three buckets, I am not sure, but those are the three thoughts that we have in mind. Let us start with the first one. Can you elaborate for us on the methodology applied to the public sector's contribution to national output during the pandemic, and, given that, how much faith you have in the estimates for GDP?

Mike Keoghan: Would it be helpful to go through the steps of how we do it? Then we can get into some of the detail.

You all know this, but the way we produce GDP is governed by the system of national accounts, which is essentially an international set of guidelines agreed by the UN and countries globally. The current set is the SNA 2008, which is being updated; there will be a new system of national accounts in 2025. The ONS is playing an important role in helping shape that across a number of dimensions.

Countries have a lot of leeway in the SNA in how they implement the standards, but while we were members of the European Union we were essentially subject to the European system of accounts, ESA 10, which provided much more detail about how we would go about doing that. There was some discretion there, but largely that was the system we were operating in. ESA 10 was a regulation and had the force of European law.

For market sector GVA, the arrangements are relatively straightforward. We work out output and remove intermediate consumption, and that gives you GVA that essentially aggregates to the market sector. Measuring non-market services—the public sector—is more difficult. The historical way of doing that was basically to look at a pound in and a pound out, so public expenditure equalled the output of the public sector. Although we still use that in some areas, it is not a particularly helpful way of understanding economic change from the public sector. It does not account for productivity improvements, and as Hume would say, a pound of defence is not the same as a pound of health.

Therefore, since the Atkinson review in 2005, we have been trying to improve the measures that we implement here. Predominantly in health and education we have been moving to output measures, so we count kids in school, kids who have attended school, GP appointments, operations. Therefore, we take an extensive feed of data from NHS England and from the Department for Education to understand what has been happening in that respect.

Output measures with varying degrees of methodology and approaches are pretty common now across developed countries in measuring GDP. They allow us to do useful things such as seeing the impact of strikes on GDP, or what has been happening to school attendance in recent months. It is a more useful way of getting at the output. Of course, during the pandemic, lots of different countries had to pick up different ways of doing that, and Grant can talk to that.

Looking ahead, to set some of the direction there are two things worth flagging. One is that we have established at King's College London, following Charlie Bean's review, an economic research institute, ESCoE, looking at measurement, and the digital economy, environment, well-being and the public sector are part of the research programme there, so there is further work going on.

Because we have left the European Union and we want to make sure that we are consciously managing diversion, comparability and validity for the UK economy, we have also established, under Sir Ian Diamond, a committee called NSCASE, the National Statistician’s Committee for Advice on Standards for Economic Statistics, which is chaired by Martin Weale and will help to guide us as we implement the next SNA and different measures of adjustment and output measurements, so that we can make sure that we are consciously applying the best measures.

Q3 The Chair: That is very good to hear. Can I just bring you back to the second part of my question about how much faith you have in the estimates of GDP? Maybe you could shed some light on that. I am genuinely not trying to put you in difficult position here. We are just trying to find out to what extent you feel the methodology is really helping us to understand that point.

Also, following up on what you have just said, I note that the ONS itself has queried, and I quote, “Are firefighters producing more output if they attend more fires? What constitutes one unit of education output?” Is output, in the terms by which you have defined it, really a good measure of what the public sector is up to in its performance? There are those two points: the macro, and the more specific.

Mike Keoghan: I will ask Grant to comment on the specifics. We have confidence in the estimates. We think they are good estimates. What gives us confidence is that we are applying the best international standards. The outputs that we have correlate quite strongly with other measures of economic performance, so we do not think they are giving the wrong signal. When we get better data in over time we find that, although we revise—everyone revises—our errors tend to be symmetrical. If there was an asymmetry there, we would think that we were missing something or had done something wrong, but we are pretty confident, the way they play out, that they are giving us the best indicators that we have. Grant, do you want to pick up on the quality issues?

Grant Fitzner: There are a couple of factors to take into account. I will say a bit about education and healthcare. The education service was one of the most difficult areas to assess during the pandemic. We were already adjusting for activity, but, of course, when you are producing monthly estimates of GDP, the ONS being one of the few national statistical institutes that currently does that, we needed real-time estimates of how many children were at home, how many hours a week they were actually doing study, whether they were assisted by their parents, and teacher assessments of the overall level of activity and performance.

Initially, we used Teacher Tapp, which is a third-party app-based tool to survey teachers in primary and secondary schools and get some data to feed into that. More recently, we typically use the monthly attendance figures from the Department for Education. If you look at that particular sector, you will find a wide range of different ways of measuring it across OECD or G7 countries, but there has been a bit of a shift by some countries towards an approach similar to our own. Pre pandemic, average attendance rates were around 95%. They are around 90% now, so there has been a reduction in school level attendance, which ultimately means that fewer children are benefiting from educational services.

There are unique challenges in healthcare as well. A key one is that, although the UK spends less as a percentage of GDP on healthcare than other G7 countries, government expenditure on healthcare in the UK is higher than in other G7 countries. Of course, healthcare is mostly delivered by the public sector in the UK—about 85%—but in quite a number of European countries it is mostly delivered by private friendly societies, et cetera. That poses challenges for those countries in measuring the data, because they have to collect it from a range of non-profit or company-based providers, but also it does not feature so much in non-market output because, in some cases at least, there is a market price for it.

That is a long winded way of saying it is complicated and every country is taking a different approach. We are not the only country that takes a direct measurement approach for health and education; a number of other countries also do so, including France, Italy and Australia.

On the quality adjustment point, a point which the Atkinson review talked about in 2005, other countries were also quite interested in following our approach.

An important distinction in how we measure health, education and other public sector output in our monthly and quarterly GDP numbers is that we try to adjust for activity and outcomes where there are appropriate measures. Obviously, there are challenges there. The point about firefighters is one. How you measure police productivity is also quite challenging. However, we factor those into our estimates of public service productivity, and we are able to produce reasonably good-quality adjusted measurements of output and productivity in over half of public services in the UK. As I said, there is some interest from other national statisticians in adopting a similar approach, but it is complicated. There is probably a degree of judgment as to what the appropriate measures are.

Mike also referred to an ONS-OECD joint study that we did relatively recently to get a better understanding of how other countries approach this. We can provide a link to that study, which shows, as I said, that there is a quite a divergence, or range, of different approaches.

The Chair: Baroness Kramer may want to follow up on some of those points.

Q4 Baroness Kramer: I will try. I have to tell you that I am struggling with this issue. I wish I had listened more in statistics class.

Can you elaborate on the issue of the quest for quality improvements in the public sector output estimates? Presumably a child who learns well is very different from a child who went through the motions. I just do not know how you begin to capture that element, or is that just not possible? Do politicians, and other people who use your data, understand what the limitations are?

Grant Fitzner: We do not take quality adjustments into account for education output as measured in the GDP, but we do for our public service productivity estimates. There are different ways of measuring outcomes, although there is quite an established economic literature about measuring how much improvement, or value-added, you see in children's performance over time from the interventions, different teachers, different school environments and different learning curricula. The Department for Education collects detailed information on pupil outcomes. This is an area where there is quite good data available to try to come up with those sorts of estimates.

Baroness Kramer: Are you going in the direction of a value-added strategy?

Grant Fitzner: Where the appropriate data is available, that is quite a sensible approach to take.

Mike Keoghan: Through ESCoE and the ONS, we have been producing work on human capital accounting, which is better than it sounds. It is basically equivalent to the capital stock measures and is about trying to get a sense of the quality so that we get some measure of education, learning and progression and understand, therefore, what is going into production and output.

Baroness Kramer: What would be the main challenges and limitations of trying to work in that arena?

Grant Fitzner: I think we can measure school performance quite well.

Baroness Kramer: I do not want to limit it to education.

Grant Fitzner: No. Higher education performance is a bit more challenging. Obviously, you have a wide range of providers, the data is perhaps not as consistent and not as coherent, and there is the more philosophical question of what a good education is. To be honest, people would have different preferences and judgments as to where the relative weights of different factors are taken into account.

Quality adjustment is an area that to some degree is data driven, but also to some degree is judgment driven, which is why we tend to do preliminary research through the Economic Statistics Centre of Excellence and other academic institutions. We also generally engage with stakeholders and users before we introduce changes.

One of the things that has been quite helpful is that, pre pandemic, there were considerable delays in accessing education and health data from government departments, and there was a real impetus during the pandemic to get those in a more timely way. For example, we now have a regular monthly data series on GP appointments, which we did not have pre pandemic. Likewise, we are already accessing data on vaccinations and test and trace on a monthly basis in order to feed those into our estimates of monthly GDP.

There have been improvements in the quality and the level of data that is available on public sector activity, and that can only continue to improve.

Mike Keoghan: To pick up on the specific point, we have a matched administrative database called the Longitudinal Educational Outcomes dataset, which essentially has pretty much everyone under the age of 35, 37 in it. It links their education record with their national insurance, their pay record, so you can work out the economic return.

I am afraid that we cannot talk about all the things that Grant talked about in relation to the value of education and more contemplative life, but we can talk about the more material returns to the education system.

The Chair: Very good.

Q5 Lord Davies of Brixton: I should have looked up what your remit is, but presumably you are providing a public service that is manifested in the form of the Government, and the Government have a vested interest in statistical outcomes. Has there been any engagement from the Government directly in the methodology you use? One methodology might produce a better answer than another. Has this been an issue? Can you take us into the locked room?

Mike Keoghan: The office is a government department, but it is independent of Ministers, and we have an independent board that essentially secures the independence and credibility of the statistics. We have, of course, talked to our users to find out what they are interested in, but ultimately the decisions about how we produce our statistics and communicate them are for the UKSA board and the National Statistician.

Lord Davies of Brixton: Just to be clear, there has been no discussion of the methodology that is used, but there has been discussion about what figures they want.

Mike Keoghan: No. We have discussed methodology.

Grant Fitzner: We have discussed data sources and methods with analysts and statisticians in the relevant government departments to make sure that we have the right data and have understood what it is measuring, to quality assure it, and to make sure that, if we are introducing changes, they are aware that they are coming. What we typically do not do is share statistical outputs before publication.

Mike Keoghan: Ultimately, it is for the office to decide what methods and sources it will implement.

Lord Davies of Brixton: Is that something you feel strongly about and would never call out?

Mike Keoghan: Essentially, it is how the legislation is set up by the Government. There is clearly a value in having an independent national stats office, and that is what we work within.

Q6 Lord King of Lothbury: My very first boss was Richard Stone. He would, I think, have been very supportive of all the work that the ONS has done to be at the frontiers of improvement in the quality of statistics for the next SNA. He also did quite a lot of work on social indicators, the reason being that people tend to use your estimates of GDP for very different purposes. One set of people in the press want a measure of short-run economic activity and how much is changing from one quarter to the next. Clearly, there must be very large standard areas in those estimates, but they are never reflected in the commentary in the press. A very different approach would be to measure well-being.

What, in your view, are the big challenges you face in trying to get people to understand what your figures and published data do and do not mean?

Grant Fitzner: GDP, or gross national product as it used to be, has the benefit of having been around for about 70 years and being well known and understood by journalists and others. But, as you said, people tend to use it to explain things that it was never really designed to explain.

For several years now, the ONS has been one of the leading national statistical institutes that have been saying that GDP is very useful for particular purposes, but it is not a one-size-fits-all statistical measure for everything under the sun, including well-being. As you know, the ONS, is one of the first countries to produce regular statistical measures of national well-being. We have a beyond GDP agenda, which, as Mike mentioned, is looking at human capital. We have been doing some really interesting and world-leading work on measuring natural capital for some years now. Part of the discussion about the next steps for the national accounts is to what extent we start to try to incorporate some of that into our measures of output. Do we take into account, for example, depreciation of assets, natural depletion of resources, into our measures of GDP?

Alongside our quarterly GDP measures, we have also been publishing measures on our climate change insights and our well-being quarterly analysis. We are trying to provide users with a broader range of measures to draw upon, pointing them to the appropriate measures and basically saying, “Don’t use GDP inappropriately. It was designed to measure output and activity. It was never designed to be measure of well-being”.

Mike Keoghan: Sir Ian Diamond, in his speech to the Royal Society in early March, spoke about some of these issues. In fact, he paid tribute to Stone’s work in this area and set out a forward agenda for how the ONS is trying to deal with some of the issues that Grant raises. We have a very active and purposeful programme of work in that respect.

The other area that we try to do is public communication. We spend quite a lot of effort as an office communicating, as far as we can, what the results mean, what they are, how to interpret them and how to deal with some of the uncertainties around them, in broadcasts through to blog posts and a podcast series. We will talk to anyone about what the data means, because we think it is really important for our mission that people have a really good background understanding of what statistics actually tell you.

Lord King of Lothbury: As you explained, GDP was a 20th century concept. If you go back to the data for previous centuries, it is always about agricultural output. Do you think GDP will be a 21st century concept, given the big changes in the structure of the economy? I remember going to a Midlands manufacturer and asking him, “How are exports doing this year?” He said, “Oh, terrific. Last year, there were 18 tonnes. This year, it is 22 tonnes”. I thought, “This is what the OS wants to hear, because there is no ambiguity about the measure of output”. That is true now of virtually everyone in the economy and makes it incredibly hard to know how to measure output, despite all the best efforts. Do you think GDP will evolve into a different concept?

Grant Fitzner: Will we still be using it in 50 years’ time? We will be using something to measure activity and output, but it may not be called GDP. Indeed, part of the international discussions about the next system of national accounts is whether we move from gross to net, whether we take into account depreciation of assets and potentially depletion of resources, whether we move to a net domestic product measure. At the moment, rainforest wildfires in Brazil, or finite resources being mined and lost forever do not feature in the national accounts. We are not capturing some of those elements. For statisticians, there is quite a lively debate about where we draw the boundary of what we measure and what we do not in national accounts. Also, other more appropriate measures, such as the ones you talked about on well-being, will be increasingly used alongside whatever measure of output we have in the future.

Mike Keoghan: Even within the production boundary, it is lovely to have measurements in tonnes, but obviously in an environment where it is intangible, or goods are consumed free, it struggles as a measure to deal with some of those issues. We have work ongoing to expand our understanding of that and to try to put some numbers on it.

Lord Turnbull: Just for clarification, ONS is a non-ministerial department. Then you used the term “board”.

Mike Keoghan: Yes.

Lord Turnbull: That is not the board of the ONS.

Mike Keoghan: It is the UK Statistics Authority.

Lord Turnbull: That is right. So that is now Sir Robert Chote. It was Sir David Norgrove. Thank you.

Q7 Lord Blackwell: One of the areas the media focuses on is international comparisons of GDP and productivity. I would like to understand what some of the things you have been talking about have meant for comparisons during the Covid pandemic, but also, perhaps even more importantly, for comparing trends going forward. To put it simply, as I understand it, although the UK, as you have explained, measures public sector output, attempting to look at the real output measures of things delivered, many other countries measure it in terms of expenditure, either input or output. During the pandemic, when fewer children were going to school, we showed a drop in GDP. If France or Italy or some other country was continuing to spend, they would not have shown a drop. How significant was that in the comparisons?

Going forward, is this a continuing problem in these international comparisons? If you imagine a situation where real output, as the way you would measure it, is flat in two countries, but real expenditure is rising and productivity is falling, we would measure that as a continuing decline in GDP. Other countries would measure that as a continuing rise in GDP, or we might be rising and they would be rising more.

Mike Keoghan: That is a great set of questions. We have done some work on this, because it is a really important issue. We did some work with the OECD about a year ago looking at the ways in which different countries adopted different methods. In particular, we looked at public administration, education and health as the three big buckets. We found that basically when it comes to public administration, most countries use the same method of pound in, pound out, so there is not really much difference methodologically in terms of public administrative outcomes.

When it came to education, again, there was quite a lot of use of the output measure—the counting of kids in school across different national statistical institutes. That work suggested that what really made the difference was individual ad hoc assumptions that individual NSIs had made about the efficacy of online learning. There is a difference there, but it is not particularly systematic or methodological.

The big one was on health. Lots of different measures were used. Again, lots of output measures were used and different adjustments made, but what the OECD landed on was that fundamentally what drove the differences had less to do with the methodologies chosen and more to do with the severity of the pandemic in a particular country. The overall assessment of that work is that there are differences, but what you observed in the outputs was not really being driven by particular methodological differences; it was more to do with a particular ad hoc adjustment or it was a function of the pandemic itself. Grant was in it.

Grant Fitzner: It was challenging, particularly during the pandemic in 2020, to explain why we saw the largest peak to trough fall in output that year compared to other G7 countries, but we then saw the largest rebound in 2021 at 7.6%. It is fair to say that those differences in measurement have largely washed out of the numbers. For example, on headline GDP, the UK is the only G7 country that has not yet recovered to its pre-pandemic levels, at least based on the quarterly numbers; on the monthly GDP numbers, we are a little above. If you exclude health, education and other public sector output and look only at market sector output, we are still slightly below our pre-pandemic level.

Where we are now is that it does not really make that much difference either way, but it certainly did in 2020, and it made the job of anyone who wanted to compare the UK economic performance with other countries more challenging. However, we think that we are actually measuring output more accurately. If kids are at home watching Netflix, they are not necessarily getting an educational service, and if other countries were not capturing that, they were just using expenditure as a way of measuring education output. They were not accurately measuring what was actually going on. As I said earlier, there is some interest and some movement by other national statistical institutes more in this direction, as indeed there is growing interest in potentially quality adjusting output measures as well.

So even though the direction of travel may look a little slow, over the next five to 10 years I think we will see more countries adopting a similar approach to the one we have taken.

Lord Blackwell: If they do not, and if they continue to look at expenditure, am I right that they will be more likely to show bigger contributions to GDP because they are not, unless productivity is improving?

Grant Fitzner: If they spend more, that could be reflected as an increase in output, but let us not forget that public service productivity is a very live issue these days. If you are not accurately measuring what you are producing in your education and health services, it is almost impossible to accurately measure how productive and efficient they are. The fact is that we have, through the direct measuring approach, better measures of what we are actually getting for those health and education services. For our productivity measurements, we also go to the trouble of trying to adjust for changes over time and quality. That means that we can have a more intelligent debate about the quality of our public services than countries that do not take that approach.

Lord Blackwell: You both suggested an area of debate about what contribution was made by home learning during that period. Presumably, parents spending time doing parental teaching is not captured in GDP, even though it would be part of well-being.

Grant Fitzner: That is right. We had to adjust for the parental contribution to child education.

Q8 Lord Davies of Brixton: You point out that education, and healthcare in particular, is largely free at the point of use, but there are significant private sectors. Does the output of private sector provision make sense? I know the pricing will probably be very different, but do the figures make sense between the relatively small private sector and the public sector?

Grant Fitzner: A good example would probably be something like childcare provision, where there is a mix of providers, which means that there is a wide range of data sources to draw on. Adult social care is another area where there are some market prices that you can use and there is more of a mixture of public and private providers. The short answer is that it depends for public service and sector by sector. I do not think there is one particular approach that applies to all.

Lord Davies of Brixton: Do you have a specific exercise that makes the comparison to make sure that the two sets of figures make sense, one with the other?

Grant Fitzner: Based on the last time I looked at these, I would say broadly yes, but I would be happy to come back with a more detailed response on the approach we take in different areas of public sector provision. As I said, it varies.

Mike Keoghan: Just to make sure that we get the right response, is your question about making sure that we capture the private provision?

Lord Davies of Brixton: No, it is that you could use a different system for private provision and public provision, but are the figures consistent with each other?

Grant Fitzner: Exactly. We can answer that.

Lord King of Lothbury: Can I just give you an example to see if I understand how big these potential difficulties are?

A couple of months ago, the House of Commons Library produced a paper on NHS statistics showing that in January this year compared with January 2020, there were fewer treatments but there were somewhere between 15% and 20% more doctors and 15% to 20% more nurses. In your methodology, does that equate to a 15% fall in productivity? It would not be surprising, having gone through a pandemic and changing the way you are forced to work, that there would be a big fall in productivity. Your estimates are more accurate than others, but is that the right order of magnitude? If it is, with health being around 10% of GDP our measurement system will provide a 1.5% to 2% difference in productivity growth, as opposed to a country that simply said, “There’s no change in productivity growth”.

Grant Fitzner: We publish annual health counts and annual measures of health care public service productivity, and you are correct: there has been an increase in inputs, not just labour inputs, but also in government spending; if you think about test and trace and vaccinations et cetera, there was a significant increase in healthcare spending during 2021. That is the latest financial year for which we published estimates. In that year, we saw a very large quality adjusted fall in healthcare public service productivity of 25.6%.

Lord King of Lothbury: Could you send that? I apologise for not knowing in advance that you had done that, but that is very helpful. So this is quite big when it comes to the sort of differences in international comparisons that people like to make.

Grant Fitzner: That is right.

Lord King of Lothbury: It is not a criticism of the health service, given that we have been through a pandemic, but it is just that we have a much more honest reflection of the true cost of the pandemic.

The Chair: Coming back to Lord Blackwell’s point about comparability—actually, I will stray into Lord Layard’s point, and maybe Lord Layard will want to come in on this—given what you have said, these figures really are not that comparable in terms of our performance versus that of our peers. Is that right?

Grant Fitzner: Some other countries do use the direct measurement approach for healthcare. I have not looked at how they fared during the pandemic, but I would expect that countries that are measuring similar things to what we would measure would also have seen large falls, although to some degree it depends on the intensity of the pandemic, which varied from one country to another.

The Chair: Is it possible for you to do some research and look into that for us, or is that asking too much?

Grant Fitzner: We could compare the headline estimates. It might take a little bit longer to identify what is under the bonnet that is contributing to those differences, but certainly we could give you a summary of what those figures look like across the different countries.

Baroness Kramer: Did I understand you to say earlier that, regardless of what the adjustments might be to put things on a more even plane and give a more granular comparison, the overwhelming determinant of what happened was the severity of Covid in that particular country and it swamps the rest?

Grant Fitzner: That was for 2021, right in the middle of the first lockdown, so you would expect to see fairly dramatic changes. An important caveat is that we have not yet published estimates for 2021-22. You would expect there to have been some recovery in output and productivity in the year that followed. That was the OECD finding.

Baroness Kramer: I am just trying to work out whether we are talking about an absolutely key, critical, significant piece of information or whether it was something that was useful to understand the period.

Grant Fitzner: That was output as of March 2021, the first full year of the pandemic, so in some ways it is not surprising that those numbers are quite dramatic.

The Chair: It would be great if you could do that bit of work. It would be fantastically useful.

Grant Fitzner: Yes.

Lord Blackwell: To follow up on Lord King's point, if you calculate that the increased expense and reduced number of treatments produced a 25% fall in productivity, how can you be sure that the extra expenditure was not actually producing results that have not yet been measured, or that the treatments were not more effective because of the higher expenditure?

Grant Fitzner: That is a good question. One of the very live issues around measuring public services productivity is that if you go from expenditure to activity, the next logical step is to try to measure outcomes. But, of course, that is harder, and it includes the issue of what the appropriate point is to measure the outcomes.

Going back to Mike's point about LEO, the longitudinal database for students, do you measure them when they graduate from high school, or when they complete their higher education, if they do so, or when they are 20, 30, 35? At what point do you say what most accurately reflects the impact of their schooling and education? Similarly, there is the issue of what is a sensible timescale for health outcomes. Also, from a purely statistical view, you do not necessarily want to introduce a method that although on the face of it is superior—in principle, outcomes should be better than activity—could also introduce greater revisions into our estimates, which is not necessarily a good thing.

Lord Blackwell: There could be an argument that since the effectiveness of the treatments is so unknown at this stage, you might be better using expenditure as the measure of value than quantitative numbers of treatments.

Grant Fitzner: Except that if we have fewer dental services, less elective treatment, et cetera, that is an actual reduction in health services.

Q9 Lord Verjee: Given that the pandemic was unprecedented, dynamic and very fast moving, with the benefit of hindsight is there anything you would have changed in your approach to the release of the GDP data during the pandemic? Allied to that, would you change your approach in the future if there was another black swan-type event? Can one prepare for an unprecedented, unknown black swan-type event?

Grant Fitzner: With the benefit of hindsight we are all wiser, of course.

Going a little bit beyond the particular topic that we are discussing, I co-ordinated the pandemic response for the ONS in the first few months of the pandemic. What was immediately apparent was that we did not have the data to answer many of the questions we were getting from government colleagues: “Are people socially distancing?” “What is the impact on the economy?” “What are people doing?” “How are they responding?” That is the very reason why we set up the Business Insights and Condition Survey, a Household Survey, and the Covid Infection Survey. We also started using more real-time indicators, such as Google mobility data to try to identify whether people were socially distancing, and, where we had access to financial transactions data, whether people were spending, to help to inform the Government's pandemic response.

Most of that data we did not have pre pandemic. Those surveys proved to be so useful that we will continue to keep the business and household surveys post pandemic, because they enable us to answer a range of other relevant questions as well. What has the impact of leaving the EU been on trade? How are businesses and households responding to the cost of living crisis, et cetera? It would have been great if we could have had those right at the start of the pandemic, but we got them up and running relatively quickly. If you go to the ONS website, you will see that there is a wide range of real-time indicators, typically daily and weekly measures, that we now use to help inform what is going on. They are not a replacement for official statistics, but because they are more timely and quite often more granular they are a really useful supplement to our main official statistics.

It would have been great if we could have had all of those at the start of the pandemic, but we have them now and we will continue to draw on them.

Mike Keoghan: I was chief economist at BEIS at the start of the pandemic, and the collective view of government analysts is that the office had a really good pandemic. Early on in the first lockdown we were losing 1% of GDP every day. The fact that we had the monthly GDP estimate, which we had confidence in, and these additional surveys were rolled out gave us the ability to respond and give Ministers really good advice about what was actually happening. It was a very strong response.

Lord Verjee: What would you do differently if we hit another black swan-type event?

Grant Fitzner: Certainly in terms of the business impact, we would certainly be looking at the high-frequency financial transactions data. We would also be using a household survey to find out how individuals and households were responding to that crisis.

Mike Keoghan: We have subsequently built capability on big data, effectively; digital data. As Grant says, we now have Visa data, so we can look at transactions. We also have expenditure data from scanners, so we will have a much better granular understanding of expenditure and what consumers and households are doing because of some of the investments that were laid down during the pandemic. The BICS has been useful for the cost of living crisis, as Grant said, but also for looking at energy costs and identifying which sectors are most exposed, right down to quite narrow information about their tariffs and what the point of maximum exposure may be when they come off tariffs.

The pandemic has left quite a legacy of additional capability, but of course the whole point about a black swan event is that you do not quite know what the next one is going to look like.

Q10 The Chair: I would like to pick up on how the pandemic has changed our workforce. There is a raging discussion taking place about the impact of working from home and productivity. You may have published on this, so forgive me if you have, but how are you measuring the productivity of people who are working from home two, three or more days a week? Secondly, if you are collecting that data, what impact is that having? Can you discern that as yet?

Grant Fitzner: Through our household and business surveys, we periodically survey the extent to which workers are working from home or hybrid working, with some time in the office and some at home. After that initial spike during the first lockdown, it seems that we have settled into a new kind of equilibrium where a larger proportion of the workforce are working at home, or in most cases hybrid working, than before.

There is a growing body of economic literature about the productivity impacts of that. I do not think there is a settled view, because it very much depends on what question you are asking. Is it good for individual workers? They seem to like it, but it can have some potential impacts on their career and access to mentoring and to senior people in an organisation. Is it good for business? One of the things that we noticed early in the pandemic when we started surveying people is that there was much more positive support from workers towards working from home or hybrid working than there was from business. Over time, that gap between what businesses were telling us and what individuals were telling us seems to have narrowed, and home or hybrid working has become the new normal. You only have to go into the city to see how many people are going to offices on a Friday to see that there has been a change in practice.

What does that mean for productivity? It means that people are spending less time commuting, so one of the questions is: what are they doing in the time they would otherwise have been commuting to work? Are they working, or are they engaging in leisure activities? Most of the research today suggests that they are spending a bit more time working and a bit more time on other activities. That has probably given business a little bit of a productivity bonus, or certainly an output bonus, but it has also given individual workers more discretion over their time.

Q11 Lord Londesborough: Has there been any study on the actual output of workers from home versus workers in the workplace? There is a raging debate in business about the actual productivity as opposed to saving two hours of commuting time. Does working in isolation without your colleagues make you more productive or less productive? Has any study been conducted on that?

Mike Keoghan: The best study I have seen on this is by Nick Bloom at Stanford. He used a very similar outline to the one that Grant set out, which was partly contextual, depending on the sector or particular business, and a division of the gains between business and the individual. I have not seen any macro-level national accounts-type analysis done yet. The question for us would be: if you wanted to do that, how might you do that? Presumably, we would carry out a management survey.

Grant Fitzner: I would say that the implications for the economy are a bit broader than just the productivity question. One is clearly the impact on transport services, where there is less demand. You are also seeing people relocating, moving to outer suburbs or away from the major cities, which has certainly had an impact on housing and people's location patterns. Interestingly, one of the major real-time indicator measures is called the Pret Index, which notes where people are spending across the Pret shops. Apart from airports, we saw a big fall in inner-city Prets but a significant increase in some of the outer suburban ones, so there has been a change in the pattern of consumption, there has been a change in where people choose to live.

It has not been noticed as much in the literature, but, in the past, employers would hire people within commuting distance of their offices or factories. Increasingly now, you can hire anyone across the UK. It really does not matter where they are. In some cases, companies can hire, or choose to hire, anyone globally who has the relevant set of skills. From an employer perspective, that potentially opens up a lot of skilled workers who were not previously able to work for you, and it gives individual workers a greater choice of jobs and employers a greater choice of people to hire. Over the longer term, that should have a productivity impact in matching skills to the appropriate roles.

The Chair: So net-net it is a bonus working from home. Sorry, I am putting you a bit on the spot here.

Grant Fitzner: Certainly the majority of ONS staff would probably agree with that.

The Chair: Is the data suggesting that it is heading that way?

Grant Fitzner: It is not an unalloyed benefit. There are some downsides as well and we need to acknowledge those.

Lord Londesborough: Surely it depends on the sector.

Grant Fitzner: Yes.

Lord Londesborough: Indeed, there are a lot of sectors where working from home would be disastrous.

Q12 Lord Rooker: What percentage of ONS staff are working from home at the moment?

Grant Fitzner: I would say most are engaged in hybrid working; they spend some of the time in the office and some of the time working from home.

Lord Rooker: This can be a bit confusing. Do you measure what the increase in asset performance of people's homes becoming the business or the employer's offshoot would be, as opposed to the millions of pounds worth of empty office space that we have lying around the country, particularly in this part of London? In other words, the offices are empty for a long time. The Home Office now has three departments in the building, something like that, and it is built for only one. Nobody has knocked anything down as far as I can tell at the moment or sold anything off. Do you measure that, because there has obviously been a big increase in asset performance of people's homes becoming an offshoot of business? Is it a measure worth doing? I do not know.

Grant Fitzner: That is one of the impacts of this change in working practice which I think will take a little bit longer to play out. We really do not know where that is going to settle in terms of how many offices there are or where they are located. Certainly, in the US, more offices are being located in outer suburban and smaller towns and commuting centres rather than in the CBD. This will no doubt have some impact on property prices and valuations, and we will probably end up with a different mix of offices than we have now.

One question that is not yet clear is what the fate will be of offices that are effectively second or third tier and may be empty in the future? Will they just stay empty for years or will they be repurposed for residential housing, for example? Those sort of questions are really not yet clear, but that will obviously have an impact for the longer-term capital productivity of the United Kingdom and, indeed, of other countries. We do not have all the answers yet. It is not entirely clear where we will end up.

Q13 Lord Layard: Could I go back to the international comparison issue? Most of the discussion so far has been about the comparison of trends, but I would like to ask you about the comparison of levels. We are constantly told that we are X% less productive than the US or whatever, but if the calculations are done in different methodologies in the two countries, I do not see that a remark like that can have any meaning. Have there been attempts, just on a selective basis, to do comparisons between countries using the same methodology?

Grant Fitzner: Yes, there have. One of the publications we put out last year was an international comparison of performance across the G7 countries for 2021. We have not published 2022 figures, because some G7 countries are still finalising their numbers for last year. On an output per hour basis, UK output levels, or relative performance, was below France, Germany and the US but above Canada and Italy. However, there are some important caveats. It depends which method you use. We used the component method based on OECD data sources, because it is more comparable than the headline statistics published by national statistical institutes. It also depends on whether you adjust that for purchasing power parity or just use nominal exchange rates.

In theory, it ought to be relatively straightforward. In practice, it very much depends on what methods you choose to use as to what outcomes you get. But it is fair to say that, whatever method you use, you would certainly find the UK’s level of output per worker, or output per hour, is significantly below that of the US.

Mike Keoghan: As somebody who has used those data for most of his career, the way Grant says things expresses it quite well. There are lots of different methodological differences or ways of doing it, but you always end up with the same ranking, the same league table. Perhaps what you learn very early on as a productivity analyst of some of these things is that you do not overinterpret one or two year changes in the ranking, because all that changes is a PPP change, or a different deflation in Germany, and then you have to start rewriting your story. Over five percentage points is when it gets interesting, so unless you are seeing quite substantial, significant changes in the level when it is indexed, below that there is just too much going on.

Grant Fitzner: Taking a longer term view, the broad story on relative performance is that, up to the global financial crisis, the UK looked like it was making some progress in catching up with some of the leading countries like the US and Germany. Since then, it has fallen a little behind. It has not shown the same pace of growth in output and productivity as some of those other G7 countries.

Q14 Lord Layard: Could I follow this up with a slightly bigger issue? People are using these measures of productivity, or GDP per head, as measures of national success, but we know that is wrong, and you have given some examples from crime and so on. One could go through endless things that are important to people which are not reflected in job satisfaction, family satisfaction and so on. Health is perhaps the most obvious example. It is completely incredible to measure national success by the output of the healthcare system as opposed to by the health of the people. We are so way off GDP as any kind of measure of national success. It exists for some completely different purpose to understanding the operation of the economy. In light of that, what is your view on well-being as a measure of national success?

Grant Fitzner: One of the lessons from the whole GDP debate, which has been running for some time, is that it is probably best in principle to avoid using any single measure to try to capture something as broad as well-being, or indeed economic performance.

Going back to the point about those relative rankings that Mike referred to, you can quibble about the numbers, and you can certainly quibble about the method, but the reality is that if one country is producing, say, 20% more per worker, or per hour, than another, they probably have a higher standard of living. They can afford higher rates of tax, they can afford to pay for better public services, and they can probably afford to have a better standard of living—other things being equal—than countries that are significantly below.

So it does have an impact. One of the things that we have seen since the global financial crisis is relatively weak productivity performance in the UK and relatively weak earnings growth, which impacts people's living standards. Paul Krugman said, “Productivity isn’t everything, but in the long run it is almost everything”. There is certainly the case in terms of the prosperity of citizens of a nation.

Mike Keoghan: On the well-being point, Sir Ian Diamond said this in his speech on the future of economic statistics, but fundamentally we agree with Kuznets and Keynes that GDP is not a measure of welfare and well-being; it is a measure of GDP. We are trying to make it as good a measure of GDP as we can, and there is an awful lot of work on that, but trying to put alongside it other measures that try to talk to welfare, well-being, happiness of the nation.

Alongside the quarter GDP we now produce environmental measures such as carbon dioxide and greenhouse gas emissions measures. We also produce some well-being measures from our well-being surveys to try to place this number, which attracts a lot of the attention, and to look at some other measures. We also have quite a long programme—I think you have talked to Sir Ian Diamond about this—about what we are doing on well-being. We have well-being dashboards. Using the surveys, we are trying to produce different kinds of measures, just to make sure the conversation on this is more balanced than it has been.

Q15 Lord Turnbull: I want to follow up the first of Lord Layard’s questions which was: what is the impact on two similar economies as a result of their statistical authorities having a different suite of methodologies? Gradually all the professional work you are doing is trying to narrow that gap.

The other way of looking at it is that you have two economies whose statistical authorities use the same methodologies, whether it is public sector output or private sector output, but it is simply that the structure of those two economies is different. Let us say that one group might be Germany, Korea, Japan that is big on manufacturing and construction. At the other extreme is the UK, which has a large, but not excessively large, public sector output but also a very large private services sector where measurement problems are particularly difficult—the financial services; professional services; the whole business of film and gaming, which is much underestimated as a pastime; entertainment; music and the arts generally, where productivity is quite difficult to measure. Is there an effect simply because we have a more services-based economy compared with the people who have a more manufactured goods-based economy?

Grant Fitzner: On the methodological point, we have been talking a lot about non-market output, or public sector/public services output, where there are obvious and significant differences between countries. But for many years, we were part of the European Statistical System and Eurostat set quite specific and detailed guidance around national accounts, and indeed about other statistical measures, to ensure a degree of coherence and consistency to enable comparisons across the various EU member states. It is, therefore, fair to say that we did not have a lot of discretion in most areas under the European Statistical System and the way in which we produce most of our economic statistics is still broadly in line with that system.

For that reason, you could compare our consumer prices index, say, with the consumer prices indices in the 27 EU member states and be confident that they are done on a broadly comparable basis, which is not the case with the US, Canadian or Australian measures of inflation, for example. Certainly in comparing us with European countries there is still a high level of methodological coherence and consistency.

Your point about the different structures of economy is absolutely right and people often forget that when they are making those international comparisons. The UK economy has quite a different dynamic to the German economy, for example, and that is particularly reflected at times of major terms-of-trade shocks or in what goes on in China. China’s level of demands can have a bigger impact on Germany as an exporter of capital goods than it might, for example, have on the UK economy. Those are definitely factors that one would need to take into account to do a proper comparison of different economies.

Mike Keoghan: The best work I have seen on the structural component, which is largely done by NIESR, generally says that it is not really a mix issue for the UK. There is generally lower per-hour productivity across the board in the sectors. There is some mix, but it is more that there are just pervasive lower levels of productivity than we would find in the States or Germany. There are different reasons for that; for example, in the US, there is a big domestic home market so they get big gains in non-tradeable services that we do not have. Generally, most of the work I have seen on this suggests that it is not really structural in the sectoral sense.

Lord Turnbull: Is there no sense that because services are more difficult to measure you miss more of them?

Mike Keoghan: Most G7 countries look the same. If we are missing them, then so are the Germans and the Americans.

Q16 Lord Griffiths of Fforestfach: I apologise for being late. You may also have touched on my question before I was here, so you must forgive me for that. Should our interest in the potential for divergence between real and nominal public output data extend beyond global pandemics, or does it all come out in the wash?

Grant Fitzner: There were some significant challenges, particularly in 2020, in adjusting the gross or nominal measures of output expenditure income into real chain volume measures. Certainly, if you look at our GDP deflator in 2020, it is highly volatile and sees some pretty extreme movements but that is not unique to the UK. Other counties also saw large movements in that deflator. More recently, it has been more broadly in line with what you would expect, although not at quite the same rate as, for example, the consumer prices index, but it certainly looks a lot more sensible.

One factor to take into account when you are trying to adjust, and certainly if you are looking at the GDP implied deflator and wondering why it is not the same as the consumer prices index, is that the UK is a net importer of energy and, when energy prices increase, we get a terms-of-trade shock, which affects the domestic deflator in a different way than you would see reflected in the consumer prices index, in which those higher energy prices flow through to businesses and consumers. This has also been a factor in explaining the difference between our GDP deflator and headline consumer inflation.

Yes, 2020 was quite a wild year, and the level of what my national accounts colleagues call “alignment adjustments and statistical discrepancies” was off the scale for a couple of quarters, but it has now come back to more sensible levels.

Lord Griffiths of Fforestfach: If you were looking today at the measure of inflation and the GDP deflator, consumer prices index and the retail prices index, what comments would you have on it, because they do diverge? For example, I do not want to get political, but in the doctors’ dispute they are using the retail prices index, which is roughly 2% above the CPI figure. If you take that back so many years, you are getting a big figure, and the 35% becomes 18%. In order to avoid those errors, what advice would you give us in thinking about inflation? You started with the GDP deflator, which is a great example.

Grant Fitzner: We have two headline measures of consumer price inflation, both of which are official statistics. One is the consumer prices index, which is produced on a comparable basis with European countries, and the other is a measure which includes housing costs, CPIH. It is a broader measure and a bit more representative of actual broad-based consumer price inflation.

We still produce the retail prices index, but it is not an official statistic and we do not recommend that people use it. It has a number of methodological flaws which means it tends to provide an inflated measure of inflation. Of course, if I were a trade union official, I would probably use whichever measure gave me the highest number, but we do not recommend its use.

Lord Griffiths of Fforestfach: If you were looking at the cost of living impact on people earning the lowest 25% of incomes in the country, would you not think the retail prices index may be a better indication of the cost of living crisis?

Grant Fitzner: No, I would not. Alongside those two headline measures of consumer price inflation, we also publish a household costs index. At the moment, it is annual, but we will soon be publishing that as a quarterly output or bulletin. It was directly designed to address the cost of living of the average household in a way that perhaps our headline consumer inflation measures do not do perfectly.

Mike Keoghan: We can do it by decile.

Grant Fitzner: Yes, we can, and that would also provide breakdown for a range of demographic groups including retired versus non-retired households at different levels of the income distribution, et cetera. This means that if you are, for example, a think tank or a charity that is interested in a particular subgroup of the population, there would be a household costs index measure that would tell you how that particular group is faring based on their own basket of expenditure on goods and services. We will soon be producing a quarterly bulletin which will be a better measure.

Alongside that, we have also been publishing regular breakdowns by decile—the 10% income bands of our CPI and CPIH measures—which shows, as you said, that over the past year or so, the cost of living facing a typical person on low income has been higher than that of people on higher incomes.

Q17 Baroness Liddell of Coatdyke: I am interested in the interface between the statistics that are produced and the impact on the policymaker going much wider than Covid. What sort of changes could be introduced into public sector data production, if any, that might be there to help policymakers? Are any of the G7 countries better at doing that than we are? Also, you mentioned Eurostat; where do we sit now with Eurostat?

Grant Fitzner: We do not sit with Eurostat at the moment because we are outside of the European Statistical System.

Baroness Liddell of Coatdyke: Do you get any feedback from it? Is there an exchange in views?

Grant Fitzner: We have certainly approached Eurostat about agreeing a service-level agreement for a regular exchange of information, catchups, et cetera, but at the moment we are entirely outside of that system and we do not have any input into Eurostat guidance.

Baroness Liddell of Coatdyke: Is that a problem?

Grant Fitzner: It may change in the future but that is where we are at the moment.

Mike Keoghan: We still have some engagement with Eurostat because the measurements of gross national income form part of the withdrawal agreement. Therefore, we continue to work with Eurostat to make sure it is content with the way we produce that particular statistic because it relates to a number of transfer payments coming out of the agreement.

Grant Fitzner: I am sorry; I should have mentioned that. We have some level of engagement, but it is around that requirement. We would hope to have a more co-operative and closer working relationship with Eurostat in the future. That is subject to a number of factors that are outside of the remit of the ONS, but we will certainly look favourably in that direction.

One other implication for the ONS of not being part of that group any longer is that we have more latitude in terms of which statistical guidance we adopt and which we do not. For that reason, we have set up an advisory committee to the National Statistician to provide us with independent and transparent advice on statistical guidance. We have also probably become more actively engaged with other players in the international field, including the OECD, the IMF, the UN, et cetera, and are very active in those forums in helping to shape future statistical guidance as well.

Q18 Baroness Liddell of Coatdyke: In looking at the future of public sector data production, are there any changes coming along that can help people like us, who are policymakers and policy analysts? Where does that fit in with the G7? Who in the G7 does it best? What is the one you are most jealous of?

Grant Fitzner: It depends what statistics you are talking about. For some areas, certainly the Dutch and the Canadians are considered as being ahead of the pack and in some areas—the Australians as well. It depends on the particular measure. We are actively working with them on a range of issues. I have talked about natural capital. We are also doing quite a bit of interesting work on the digital economy and one of my deputy directors is chair of the OECD committee on measuring the digital economy; we have been very active in shaping that discussion as well. We talk to our colleagues all the time about a wide range of matters. We helped set up, with the UN, a network of economic statisticians, which we have been closely involved in running and in presenting, including, for example, on the “Beyond GDP” agenda. We are active internationally on a range of forums, both at the bilateral level with other national statistical institutes and at the international level.

Mike Keoghan: On the production point, we are also keen to make sure that we get the data out in formats that people can use and can use in terms of policy developments, in advising Ministers and in shaping public discourse. There are three things worth raising. We now produce a variety of dashboards. The most recent is the one around COP 26 which is a climate dashboard that puts all the Government’s climate information in one place so that analysts are able to access it in a consistent, clean format. It is also a retail front end, where anyone who is interested can see what the data says.

We have a programme called the integrated data service, which is quite an ambitious programme that is trying to ensure that lots of different data sets from government can be put together so we can free up the data, the admin data in particular, for analysts and academics to start linking things together and generating insights which could not be done individually. A good example is the LEO database which, although it precedes the IDS, gives that ability to put education records with Pay As You Earn records so you can see the impact on earnings from particular courses. That is the kind of thing we are trying to encourage. We are trying to make big progress in that.

Another issue is around better use of digital tools like application programming interfaces, APIs, so that people do not necessarily have to go through the ONS website; they can directly access the data themselves, pull it and update whatever modelling or programmes they want to run with our data so that they can provide the fastest insight to Ministers.

Q19 Lord Rooker: I was going to ask you about other sectors for which statistical methodology might generate discrepancies but, as you clearly said in answer to Lord Turnbull, if we are missing out things, they are as well. I therefore have to assume that there are no sectors that you think might generate discrepancies with European or G7 counterparts at some point in the future, on the basis that we are all missing the same things out.

Following Lord Layard’s questions, and I suppose it would come under well-being, for several years, Michael Marmot has used the ONS statistics for some of his reports. They clearly show that, since the year 2010, life expectancy has stalled in this country for the first time in 120 years. I do not know how you could measure the effect of that other than the business of funeral services because people are dying earlier than would be expected. I do not know whether that counts in your productivity or output. I have a Question about this next week, but how did the change occur given your charts of life expectancy are so clear? It is nothing to do with the pandemic, although I accept that there is pandemic effect. But 2010 stalled in the first time for 120 years, so something has happened that would have changed business. Do you take any account of that or measure it at all?

Mike Keoghan: We do not on the economic side. Coming back to the first point that if we are missing it, everyone else is missing it; it does not mean we cannot try harder to capture the sectors, so we do a lot of work on that. Could you say a bit more about the question because obviously we do not capture life expectancy in terms of the economic statistics?

Lord Rooker: I accept that, but you publish tables on life expectancy, and the fact is it has stalled, as your own ONS charts have shown, since 2010, which means people are dying earlier than expected. The Government have just announced they are changing pensions policy because people are dying earlier. Is there a measurable effect on the funeral industry? People have to sometimes wait three or four weeks to get a funeral, which did not used to be the case. Is that something that you actually measure in terms of output?

Lord Turnbull: Are you talking about dying earlier or dying earlier than expected? Because we may still be living longer but not as much as we used to.

Lord Rooker: Life expectancy has stalled for the first time in 120 years, since 2010. It has stalled; it is not going up anymore. The graphs from THE ONS clearly show it has stalled, as Michael Marmot has repeatedly said. The Government actuary and others were working on these things affecting pensions, but it is quite clear that people are dying earlier than would have been expected if we had not stalled. Therefore, I just wondered if there is an effect on business because there must be on the funeral industry, for example.

Mike Keoghan: We do not have that data with us, but it would probably sit within other business services, so we can see whether there is any impact on trend output at that level of category.

Q20 Lord Layard: If you adopt a well-being approach, then what you think of as being the measure of success is the number of well-being life years, which means, if people live longer, the individual life is a better life lived than if you live shorter. This is the exact analogy to the way in which the whole health service is meant to be operating. It is meant to be operating to maximise the number of quality-adjusted life years, which is the same concept. Really, it is the only serious alternative measure in town to the GDP per head, when we are thinking about the success of our arrangements.

Mike Keoghan: In national accounts terms, the other way will be within human capital accounts. The same kind of questions could feature there, but I am not an expert on it so we would have to look at how that issue is dealt with in that context.

Lord Davies of Brixton: I have to say something here. My pet peeve, when someone is talking about life expectancy, is that they always need to be clear about whether they are talking about period life expectancy or cohort life expectancy, because they are very different things. I think Lord Rooker was talking about cohort life expectancy, which is declining: people are not living as long as they would previously have been expected to live. It is a big issue and there has been that slowdown. Equally important—and it may be a project for next year—are the differences between different demographic groups in their life experience. But this is not the stuff that you are doing.

Q21 Lord Londesborough: I would like us to focus on the digital economy for a few minutes, particularly the measurement gap between the economic benefits delivered by digital services and the monetary value measured which, as I understand it, for most free services, has zero impact for GDP purposes. We now have a myriad of free apps, for example, including social media, that are replacing paid-for services and goods—such as cameras, radio, maps, encyclopaedias, books, newspapers and matchmaking services.

What are the challenges for the ONS in getting to grips with this area of economic activity, much of which appears to be zero-rated? Specifically, what changes have been made in the last 10 to 20 years in measuring the digital economy in relation to GDP? You mentioned there is a 2025 global national accounts release; what has been adjusted to reflect the digital economy and what is being considered going forward?

Mike Keoghan: I will pick up on what we have done and then maybe Grant can talk to the SNA. I completely agree with the challenge. It was set out in the Bean review of the economics statistics of the ONS that the office was not really at the races on this. There has been a lot of work done. The main element of that work has been through ESCoE looking at the IT and telecom sectors and trying to identify better deflators for the quantity of their outputs given that the prices have been falling. I think Diane Coyle was very much involved in that.

Linked to that, applying double deflation within national accounts tries to identify where the value from the digital economy is actually manifesting itself. The example I would give is, for a long period, technological improvements in automotive were being ascribed to automotive whereas it is actually the IT on board that has been generating the value. Better understanding, better measurement and better allocation of where the value is being created is needed. We have further work going on with the new ESCoE.

Of course, to some extent, although those apps are free, in some cases the business providing them is generating revenue and output, which is captured in different parts of the national accounts. Even if the consumers are not paying for it directly, maybe advertisers or whatever are and so it will be featuring in that respect.

We are also looking at new technological developments. We have recently asked businesses about chatbots, ChatGPT: have you heard of them, have you used them and what do you use them for? We should have data on that imminently. There is a lot of hype, but is it actually being used in businesses? If it is pervasive, we have to think about how those technologies get incorporated. There is work being done, but there is a lot more work to do and obviously action at the global level.

Grant Fitzner: Picking up on what Mike said, there are a few things on the agenda, although their final shape and to what extent they are incorporated in future statistical guidance internationally are still under active discussion. The first is around platforms, or what we call digital intermediation—for example, Airbnb, Uber, et cetera—which brings users and providers together and provides a platform through which you can book a journey or somewhere to stay. We are trying to capture what value those platforms add, so not the total cost that you pay but the amount that they contribute because, of course, most of that goes to the provider. Those platforms capture some and it needs to be reflected in the national accounts; we need to find a way of accurately capturing it.

The second is a broader suite of what we are calling digital supply-use tables. The OECD has been the lead in developing these. The challenge for statistical agencies is to come up with something with sufficient information to give it a sense of where the digital economy is impacting real-world statistics, but also not to create something that is so burdensome that very few countries will collect it.

The third element, where we have done some really interesting work for Europe with the UN and the European Commission, is around how you value data and datasets. Going back to Mike’s earlier point about intangible capital, obviously brand premiums are one form of capital but another is the datasets that a lot of companies now have, including lots of very detailed information about individuals and businesses. How do you put a valuation on that and reflect it in economic measurement? We are also questioning what the impact of AI will be, which we think will potentially be significant on workers, companies and productivity over the next five to 10 years. We are starting to create some benchmark surveys to capture that.

Finally, probably the hardest is what you called the app economy. People went from buying a map or a camera to so-called “free applications” now. Of course, they are not really free because what consumers are doing is providing the data—often quite personal and disclosive data—to companies, which they then use to sell ads and various other means. This is coming back to the point about the production boundary. If somebody is creating content on Instagram or Twitter for free, for those platforms, should we be capturing that as an economic output? If so, where does it sit? Likewise, how do we capture the value that those platforms—Facebook being another good example—are providing for individuals? It is not in the national accounts at the moment, but there is an argument that at least some of the value that is being created should be captured.

Lord Londesborough: I am sure this is an international phenomenon, not just for the UK. Could it be argued that there is quite a large swathe of the digital economy that is not being captured in GDP? Therefore, in the year 2023, we are actually understating economic well-being compared to, for example, 1953, when there were very few free services. It strikes me that there is a sort of “mind the gap” or what I think some economists are calling the consumer surplus—the difference between what consumers are prepared to pay and what they actually pay. In many cases, it is nothing, even if they are trading privacy. It is very difficult to measure that in economic terms.

Grant Fitzner: Most statisticians and economists would agree with you that we are not properly capturing that, but trying to quantify a so-called free service and incorporate it into your economic statistics is really quite challenging. There are a range of methods that are being proposed, but they came up with different estimates. It is fair to say that there is not yet an international consensus on how you would go about doing that. We certainly have been engaging with some of the leading academics internationally, but this is not a settled area yet.

Mike Keoghan: Nicholas Crafts has written on this in respect of not free goods, but new goods, because when a good is created that never existed before, it generates a consumer surplus that people had not anticipated. Again, we find it hard to incorporate that into measurement.

Q22 Lord Verjee: You mentioned AI, chatbots and ChatGPT. How fast is the ONS moving in these areas? It is a bit of a tongue-in-cheek question, but would you recommend that the committee should have a chatbot here sitting beside you? How far are we from that?

Mike Keoghan: We have a pilot running in the office at the moment looking at exactly how it would be used. A number of staff are using them. From our first blush using it, we have found that it is good at briefing. If you want a short brief on an issue, it will produce one. It is not always 100% right, but are all briefs always perfectly accurate? You do need to be an informed customer of it, but it is pretty good at producing briefs. It is quite good at code. If you want to spin up a dashboard or something, it can provide that. I will be honest: I used it in generating part of the brief today. It is a very useful tool, but you do have to use if carefully as it is not always correct.

Grant Fitzner: It does not do maths and logic particularly well, which is a fundamental potential flaw in some of the work that we do. The danger, of course, is you can end up with very plausible sounding information that is actually false, which is not ideal. These are early days, who knows where we will get to in the next five or 10 years, given the quite rapid pace of change.

What is interesting is that there are literally thousands of new services and applications that are being developed. There is a very high rate of innovation. Most of those will probably fail and will not be successful business models, but some will be. I particularly enjoy that, for the scientific or academic journals, there are now some tools that provide a summary of what they say, provide a critique of that research and basically make that information more accessible.

This is outside of the scope of the ONS but certainly of relevance to your committee: we are not far off the position where children at school will have their own little chatbot, like a personal tutor. The Khan Academy is already developing a service along those lines. You can interact with it, it will give you exam questions and it will give you summaries of the relevant material that you are studying in an interactive way. The potential of that to complement existing schooling is potentially quite powerful.

Mike Keoghan: It is also very good at accessing data from the ONS website, which is not easy.

Q23 Lord Blackwell: I come back to the difficulty of measurement in this digital economy and how that might impact on time series trends in productivity. It has been mentioned a number of times that productivity growth over the last 10 years or so has been slower than historically. There are obviously a number of factors at play in that since the financial crisis, but it has also been a period when there has been significant growth in the digitisation of the economy. There is the impact that you just discussed with Lord Londesborough of many services now being provided that have the same value to consumers but a much lower cost and therefore a lower monetary value in terms of GDP. That reduces GDP and productivity even though consumers are, arguably, no better off.

There is also the growth in online shopping, which has been particularly prevalent in the UK. We all used to spend significant amounts of our free time going to shops, loading up carts and trawling around different shops to find a purchase. That use of our time has now been superseded. Not only is it a much more efficient use of our time, but we now employ lots of people—coming up my drive every day and everyone else’s drive—dropping off parcels where previously it was my time, going in the car to the shop, and these are generally quite low-paid activities. I suspect we have brought a lot of people into the economy on relatively low-paid jobs, or people who would not otherwise have been working have taken them up. All that reduces the average employment value per worker and reduces productivity. Therefore, something which has actually been a benefit to society in terms of well-being would show up as a reduction in per-worker productivity.

Mike Keoghan: Picking up on the point about lower cost, that is not necessarily the case because you hope that the way you deflate picks it up, so you are getting more of the output for less cost. That is part of the reason for the work that we were doing with ESCoE on double deflation and the role of ICT; we are trying to make sure that we are properly accounting for the dramatic improvements in, in effect, the productivity of that sector.

As digitisation and structural change imply, these are all big, live issues. I would say to the committee that we know that and we know we have a lot to do on it. We have some big work programmes going on. It is not made easy by the fact that it is changing in real time. The validity of what we are trying to capture is always shifting, but we are committed, globally and in our own production, trying to capture these trends properly. We have big ambitious research programmes with Turing, ESCoE and King’s to try to make sure that we are keeping up with this.

Grant Fitzner: Your point about e-commerce and the effect it is having on wholesale and retail trade is really interesting. One way you could look at the pandemic—and this is probably true of wars and recessions—is that it accelerated the pace of technical change. Literally, within two weeks of lockdown, 5,000 ONS employees were all working from home with laptops. In fact, most of them were in the first week. That was a rapid change of practice. Likewise, consumers moved massively online to e-commerce because lots of stores were closed. Both the level of working from home and the level of adoption of e-commerce versus physical stores has obviously fallen significantly since 2020, but it is still significantly higher than it was pre-pandemic.

The productivity implications and the implications for business of that are still being played out. We have seen a big expansion of warehouses, but clearly there is less demand for retail space. Is that going to end up as a system that is more or less productive or a better or worse use of capital than what we had before? It is hard to say. To some degree, consumers are paying for greater convenience and being able to access a greater range of goods in a more timely way than they could before. I suspect they are not going to go back to what we had pre-pandemic, but that convenience comes with a cost.

Lord Blackwell: My point was not to criticise your numbers because it is incredibly difficult to measure these things. It was more to suggest whether we should be appropriately cautious in looking at trends in productivity using these imperfect measures now to compare to the past, when we are seeing the impact of the digitisation on the economy and are not clear how that is reflected in productivity.

Grant Fitzner: We should be cautious because those structural changes are still playing out and it is not entirely clear what the end-point of them will be. We also now have another potential structural change in the productivity of AIs, chatbots, et cetera, which could have a big impact, particularly on office-based workers, over the next couple of years. It is very much early days, where the implications for business, profitability, productivity and employment are as yet unclear.

As a note of caution, I suspect many of the members of the committee are old enough to remember the previous headlines about computers taking your jobs, et cetera. We are in an environment where the unemployment rate is under 4% and lots of businesses are saying they are having trouble recruiting people—so we have not seen that yet. That does not mean that this time might be different, but previous occasions where there has been a bit of computer/digital panic have not played out in the ways that people expected.

The Chair: Just to be clear on this and pick up on Lord Blackwell’s point, there is a very interesting article that Baroness Hogg has written about AI and productivity. She asks the question, “Why hasn’t the extraordinary third industrial revolution boosted productivity in the way that the first and second did?” What I am hearing from you is it is because a large part of that may just simply be that, at the moment—I emphasise “at the moment”—we do not have the ability to analyse, or even to extract the data, to understand productivity in the digital age. Is that a rough summation of where we are?

Mike Keoghan: When you have structural change in the economy, there will always be new things that are difficult to measure. The movement into services threw up similar issues. How do you measure the value of services? What are the quality adjustments on services? It is not like there was a golden age when everything was measured. Digitisation and zero marginal cost pricing makes things more difficult, as with a service, you are able to transact. There is always an element of uncertainty about measurement.

If you look at the canonical papers on this like Paul David’s work, there is an element of lag; there is always an element of how the existing capital stock, skills and organisational technology are able to adopt the new tech and yield the productivity improvements. We have not yet seen that fully play through because we are still, in many ways, operating pre-digital ways of doing business.

Grant Fitzner: It is hard, but I have a point about the digital economy. The difference is that as more services are delivered digitally, the pace of change is likely to increase and the ability to capture that in some ways will become easier, because there is a lot more data out there than there used to be. We saw that in 2020 during the pandemic. Within literally a matter of weeks, we had our fingertips on incredibly detailed, granular, daily or weekly measures across a whole range of things that we did not have literally a month before. The pace of change may be picking up, but also our ability to respond to that quickly is probably stronger than it was in the past.

Mike Keoghan: That is a really important point, as the digital economy throws off data as a by-product and that is brilliant for us.

Q24 Lord King of Lothbury: Can I try to push back a little on this belief that the digital economy has totally changed our ability to measure output? If you take the organisations operating digital activities, which they may provide free to consumers, they are still paying wages, they have profits and you can measure their value added, at least in nominal terms. There is a difficult question about the deflator, but it is not missing from GDP. It is there. The more interesting questions seem to me ones that, in a sense, have always been big questions for the national accounts.

Can I give you a very simple example to try to illustrate it and get your reaction to it? When people started producing national accounts data, it was agreed as a convention that we would not try to measure the value of work done at home. It was impossible to do that, so it was just left out on the grounds that everyone worked roughly the same number of hours a week and that we do the same work at home in washing up, cleaning the home, et cetera, so we were just not going to measure that.

The challenge of digital technology seems to me, in part, that it is eroding the sense of that assumption. Lord Blackwell referred to the fact that we do not have to spend half our leisure time tramping around the town, going to shops and hauling the stuff back. We can pay for a service that will deliver it, so we have more leisure time. Those changes are left out. The thing that seems to be missing in terms of valuing the use of smartphones, say, is that, before and after smartphones, the work itself is unchanged. People go to work for eight hours a day, they do a piece of work that you can measure easily—it is piece work, such as the number of calls on a call centre or the number of bits that you can make—and you are allowed five minutes every hour to stop work, so that people do not go mad. That is the same before the smartphone and afterwards. The difference is that, before the smartphone, people just sat around gossiping; with the smartphone, they book holidays, they organise things. They are doing activities for those five minutes every hour, which means that they do not have to spend a lot of leisure time doing those activities, which they would have had to do before smartphones.

It is not that the digital technology, at one level, has made it more difficult to measure output; it is just that the convention we adopted of what output actually is and what we are going to measure has now changed. All these things are reopening questions about the assumptions that were always made in the system of national accounts. Is that a right way of thinking about the two dimensions of digital—the one that you can measure, because the activities are in value added, and the bits that you cannot, because they encroach on an assumption that we used to make, the meaning of which has changed now?

Mike Keoghan: Yes, that is a really helpful way of thinking about it. As we said earlier, the value of the service will be captured at some point if it is in the private sector and we are able to capture that. The point about moving from the household into the production boundary and what that looks like is something we are exploring in the context of the SNA. We are also looking at rebooting the Time Use Survey to try to understand what is going on. It is a really helpful way for us to consider it.

Grant Fitzner: I would not say we cannot measure it; I would say it is hard to measure it. We want to move away from a Time Use Survey, which we collect every couple of years, to an ongoing or rolling survey, which will better capture how people are spending their time, exactly as you have just described. Whether we then ascribe some value to that is the challenge. It is not so much knowing what people are spending their time on but how you quantify that and whether you should incorporate that into our economic statistics.

The Chair: Have any of your peers, in the means that they use in their methodology and in gathering statistics, answered that question?

Grant Fitzner: People have tried different methods.

Mike Keoghan: We can ask the question and we can reply to the committee on that.

Grant Fitzner: It is challenging because individuals are now both producers of value—such as creating content free for platforms—and consumers as well. Putting some valuations around those is much more challenging than working out what contributions Facebook or Uber make because, as you said, there are workers and wages, revenue and profit; you can measure those just like in any other business

Lord Londesborough: In the example that Lord Blackwell was giving, as I understand it, in e-commerce, Amazon shopping, that data is being captured because it is not a challenge locationally. If you take another example such as Wikipedia, it has effectively substituted Encyclopaedia Britannica and sales and jobs have disappeared as a result. There is no output to measure that. It raises another issue, the borderless issue. My knowledge on Wikipedia is quite thin but, as I understand it, the vast majority of the Wikipedia staff are not in the UK, so while the consumption is happening here, it does not leave a trace as far as GDP is concerned. It has actually removed an industry, perhaps for good—no door-to-door sales or paying £2,000 for Encyclopaedia Britannica. I would have thought that digital was also throwing the challenge of, in some cases, big tech being borderless and actually a lot of the labour output being offshore, as far as the UK is concerned.

Lord King of Lothbury: It should not be in GDP because then you have a higher consumption from buying encyclopaedia services and you are importing those services. GDP is C+I+G+(X-M). That means that, if C and M go up, it cancels out. So the fact that these people are working abroad should not enter our GDP. It should be expenditure in consumption that is offset by an import.

Lord Londesborough: In this case, there is no expenditure on behalf of the consumer, who has effectively moved from an encyclopaedia to Wikipedia.

Lord King of Lothbury: In that case, if we are not paying for it, we are not paying for the imports either. Either way, M should offset C.

Mike Keoghan: In a sense, it is worse because actually most of the labour for Wikipedia is free, because it is from volunteers.

Lord Londesborough: I think you said that you do not measure volunteering.

Lord Blackwell: I just wanted to clarify the point I was making about e-commerce and online shopping. As Lord King was saying, it is not just that I have more spare time. It is the fact that, for the same end-product, which is my fridge filled with food, there was no labour content measured because I was doing it in my free time, and there is now labour content measured, which is lots of people driving vans around. Therefore, productivity has gone down for the same input.

Lord King of Lothbury: Is it a batting average effect because you have more lower-wage people? What were these people doing before? If they were unemployed and doing nothing, you have an increase in GDP for the same number of people.

Mike Keoghan: It may also depend on the technology of the distributor. If they are operating a robot warehouse that is highly capital intensive, so it may be that those lower productivity drivers are offset by a super-efficient warehouse.

The Chair: This is such an interesting debate. You talk about the digital economy, and I totally understand why you do but, in a sense, the entire economy is digital. Therefore, coming back to the point I made earlier, when we get a better grasp of this point, it sounds like it is going to have a very profound impact on not only our understanding of productivity but the point that Lord Layard was making earlier about trends and absolute level and performance.

Tying it all up with the question I started with in terms of confidence in the statistics, you are obviously confident in what we have now and the methodology that you think is best, but it strikes me from this conversation that you are both very much aware that there is an enormous gap out there in our understanding and knowledge, which may dramatically change our analysis of productivity. Is that fair or am I putting thoughts and words into your minds?

Grant Fitzner: Some more public literacy about productivity could only be a good thing. It is probably not the sort of conversation you would have with the average taxi driver these days, but maybe there will be a time when that is the first thing people want to talk about.

I will just go back to the digital point, because it is really important one. It is one of the reasons why it is taking quite some time to get a degree of international consensus on how we capture this. One way of measuring it is, as you say, everything that is digitally delivered is part of the digital economy. If you to go back 80 or 100 years, you had the emergence of the electricity economy and now almost everything is delivered via electricity, replacing steam, but is it a meaningful measure to say that 80%, 90% or 95% of things are delivered by electricity? Not really. If we are getting to the point where the vast majority of goods and services are delivered digitally, is that a useful statistic? It is probably not; it is just the general direction of travel.

We need a more granular and targeted way of capturing what is changing and where the value added is from this move to more digitalised delivery of goods and services that really tells us what has changed, what is different and where those impacts are, rather than some very broad measure, which ends up basically capturing everything and therefore is not of any particular value.

Mike Keoghan: I think it has always been our burden in the ONS that we are having to constantly catch up with a super-dynamic economy. It is no different now than the switch to services, or predecessors having to deal with other structural changes. We know that that is what we have to do and we have the plans to do it, but the economy is really dynamic and we just have to try to keep up.

The Chair: We have taken up a lot of your time. Thank you very much. That was an extremely interesting discussion, and we are really grateful. If you could come back on the point we discussed earlier, that would be very helpful and it would be great to hear any other thoughts you have on that final exchange on the measurement of digital. Thank you very much.