Written evidence from Dr Mark Courtney (GOS 16)

 

Public Administration and Constitutional Affairs Committee

Governance of Statistics inquiry

 

Executive Summary

I.  Introduction

I have had extensive involvement since 2010 with the UK Statistics Authority in the debate about consumer price statistics – personally, as an author, and as a member of statistical committees of the ONS and Royal Statistical Society.

II.  The governance failure in the case of consumer price statistics

  1. The House of Lords Economic Affairs Committee has identified UKSA failings in refusing to correct known faults with measuring clothing inflation and in a lack of engagement with users.  But the failures of governance in consumer price inflation statistics are longstanding and extend beyond the UKSA and ONS.
  2. There are also features of the UK system that worked well and prevented the degree of government manipulation of consumer price statistics that occurred in other countries, particularly the constitution of the RPI Advisory Committe1947-94 and the well-developed index-linked gilts market, with provisions in the prospectus that protected investors.
  3. Prior to around 1985, the UK was the world leader in government statistics.  The alleged weaknesses of the RPI were known about and discounted when the RPI was developed, and the ONS was able to resist Treasury pressure in 1998 to follow the USA in changing its formulation to give a lower measured inflation rate.
  4. But from 2003 the ONS effectively ceded control over consumer price statistics to the Treasury.  The June 2010 decision to change the uprating formula was taken without consulting the National Statistician and the ONS immediately started to fall into line.
  5. In 2010 and 2011 the governance arrangements for the UKSA worked mostly as intended: the ONS was induced to reinstate the RPI to the front page of its bulletins and, as directed by UKSA, undertook high-quality research into clothing price inflation and the formula effect.  Both RPI and CPI were confirmed as National Statistics. However, a major change to the calculation of deflators in the National Income accounts went unexamined.
  6. From 2012, under a new Chair of the UKSA and with the replacement of all the key members of the ONS Prices Division, the system failed comprehensively:
  1. Having been thwarted in changing the RPI, UKSA then:
  1. From 2016 onwards there was some limited improvement:

III.  Lessons for Governance of UKSA

  1. For statistics to be trusted, practical independence from government influence in their production and deployment is essential.  The biggest blows to trust in consumer price statistics came when successive Chancellors of the Exchequer by-passed supposedly independent arrangements in 2003 and 2010.
  2. But accountability is equally important.  Elected governments have to take account of public opinion and daily parliamentary scrutiny, but an independent institution, particularly if it has a claim to technical expertise, has the temptation of thinking that it can act entirely as it sees fit.   This was the case with UKSA from 2012 inwards.
  3. Transparency is also vital.  The requirements regarding openness in the Code of Practice for Scientific Advisory Committees should be followed as a minimum.
  4. There needs to be clarity of roles: both the Chair of UKSA and the National Statistician have conflicting responsibilities and there has been no effective champion for high-quality, independent statistical expertise to be applied within ONS.
  5. Good government also needs to be applied to the choice of which statistics to produce.  In the case of consumer price statistics one might think that the existence of two consumer price indices is itself the major problem, since it allows index shopping.  However:

 

I.  Introduction:  my involvement with UKSA/ONS on consumer price inflation indices

  1. I was Head of Economics and Deputy Director in the Regulatory Impact Unit/Better Regulation Executive in the Cabinet Office from 1999 to 2007.  After retirement, I wrote several articles on the relative merits of the CPI and RPI, which, starting in April 2011, were published on the Social Science Research Network and in the peer-reviewed Statistical Journal of the IAOS. In addition, I was a member of the ad hoc Consumer Price Indices Technical Group set up by the Office for National Statistics, which met in 2011 and 2012 and I was a co-author of the Royal Statistical Society’s response to the ONS consultation on improving the RPI, October 2012. My review paper, Consumer Price Indices in the UK, November 2014, is the most comprehensive published exposition of the subject:  I presented it to a meeting of the RPI-CPI User Group in December 2014, and to the first meeting of the ONS’ newly set-up Advisory Panel on Consumer Prices–Technical in January 2016. (A revised version, updated in August 2016, is published on the UK Statistics Authority web-site[1]).

II.  Nature of the governance failure in the case of consumer price statistics

  1. The House of Lords Economic Affairs Committee has identified UK Statistics Authority failings in refusing to correct known faults with measuring clothing inflation and in a lack of engagement with users.  But the failures of governance in consumer price inflation statistics are deeper and more longstanding and have not been confined to UKSA and ONS.

Good governance prior to 1994

  1. Prior to around 1985, the UK was the world leader in government statistics.  The RPI Advisory Committee, reporting to the relevant Minster (the Secretary of State for Employment and his predecessors until 1989, the Chancellor of the Exchequer thereafter) was set up in 1947 and reconvened fourteen times until 1994, with technical committees working on some designated issues in between.  The reports of the RPI Advisory Committee and of any technical committee that reported to them, were published as Command Papers.  In practice, the RPI Advisory Committee was a decision-making body, since the Minister did not make any major changes to the RPI except on their advice.
  2. The RPI Advisory Committee was a “mixed” committee, chaired by a senior Civil Servant, with representatives from employers, retailers, trade unions and consumer bodies, senior statisticians from across government and distinguished academic statisticians, including, until 1975, Professor Sir Roy Allen, the pre-eminent statistician of his generation.
  3. In view of recent controversies, it needs to be emphasised that, by the 1940s, the statistical properties of the aggregation formulae used in the RPI and later in the CPI had been established and were fully taken into account in the development of the RPI.  Although there have since been some minor clarifications and some interesting empirical studies, Index Numbers in Theory and Practice, Sir Roy Allen’s 1975 graduate-level textbook, remains the most comprehensive treatment of the subject.  Had the ONS read it at the time, some of the outright errors that they made in 2010 might have been avoided.

1997-2010: loss of expertise and capitulation to political pressure

  1. The RPI advisory committee was never convened after 1994.  This meant that some significant changes, such as a comprehensive revision and broadening of sampling procedure in 1996 and the construction of the UK’s Harmonised Index of Consumer Prices (HICP) in 1997 were made without the benefit of outside expertise or public scrutiny.
  2. This was a time of heightened interest in consumer price indices.  The shift to inflation-rate targeting for monetary policy and the increasing level of index-linked social security payments meant that governments were looking for ways to produce lower inflation-rate measurements.   In 1995 the US Senate Finance Committee set up an Advisory Commission – the Boskin Commission – which produced its highly influential final report in December 1996. One of its recommendations was that the US CPI should switch from using only an arithmetic aggregation formula, as had hitherto been almost universal international practice, to making use of the geometric mean formula, lowering its measured rate of inflation, as later calculations revealed, by 0.2 percentage points.  Interestingly, the Commission based its recommendation on the one statistically respectable argument for using the geometric mean (Jevons) formula, the so-called Economic Approach.  An index calculated using the geometric mean will always underestimate the rise in the cost of a fixed basket of goods:  however, if our aim is not to measure changes in the general price level but to show the percentage increase in income needed to leave consumers equally well-off after inflation, then, on very particular assumptions about consumer behaviour, the underestimation inherent in the geometric formula might be a good approximation of how consumers could maintain a fixed level of welfare while not being fully compensated for a rise in the general price level.
  3. This argument had been anticipated by the RPI Advisory Committee in its methodological report in 1986, where it reaffirmed that the RPI was an index of the overall change in prices and not a “cost of living” index designed to measure changes in the cost of maintaining a particular level of consumer satisfaction. This is an important distinction that underlies much of the argument that took place in the UK after 2010.
  4. When, in 1998, the Treasury put pressure on the ONS to follow the Americans in introducing the geometric mean formula into the RPI, the ONS retained sufficient expertise and self-confidence to refuse.  Also in 1998, the Director of the Consumer Prices and General Inflation Division published an article explaining why the UK’s new HICP – which used the geometric mean formula in accordance with Eurostat requirements – differed by much more from the domestic, RPI measure than was the case in the other EU countries.
  5. In December 2003, the Government changed the monetary target for the Bank of England from RPIX to the HICP, simultaneously lowering the target from 2.5% p.a. to 2% p.a. as partial compensation for the lower average level of the HICP.  The reasons given for the change were plausible but represented at most a minor improvement.  In retrospect, the significant change was that the National Statistician – who had played no part in the decision – agreed to renaming the HICP the Consumer Prices Index, CPI, a name which was bound to cause confusion with the RPI and suggested that it was being supplanted as the “headline” index, even though the Government had reaffirmed the continued use of the RPI for uprating purposes.
  6. Coincidentally, at the beginning of 2010, after criticism by Eurostat and others of the implausibly steep decline in CPI clothing inflation since 1997 (RPI clothing inflation also fell, but not as steeply), the ONS attempted to solve the problem by introducing a new sampling protocol for clothing prices which greatly relaxed the normal “matched model” criterion.  This was done without piloting the new procedures, and by the middle of 2010 is was already obvious that the relaxation had gone too far: both CPI and RPI clothing inflation had increased, but so had the dispersion of measured clothing price changes, reducing accuracy, and – an algebraic consequence of more dispersed prices – the formula effect difference between RPI and CPI clothing inflation had roughly doubled.
  7. The inauguration of the new government after the May 2010 general election proved the opportune political moment to make the switch to using the CPI to uprate social security payments and public pensions, transferring an extra £6 billion in a full year to the Exchequer. The National Statistician was not consulted about the change, but the ONS immediately showed itself willing to fall into line, removing the RPI from the front page of its bulletins.

2010-2011:  The UKSA as a regulatory body functions well on consumer price statistics but poorly on National Income Accounts

  1. Sir Michael Scholar, the Chair of UKSA, was well aware of the intense political interest in the change in uprating index, and took the view that it was the not the role of the UKSA to comment on the uses to which the CPI and RPI were put but to calculate both indices to a high standard and make them available to users.  The ONS was induced to reinstate the RPI to the front page of its bulletins and, as required by UKSA, undertook to describe the differences between the RPI and CPI, and it also initiated two major research programmes:
  1. Rather tardily, the ONS in October 2011 produced the required paper describing, in neutral terms, the differences between the CPI and RPI, and on 31 January 2012 the UK Statistics Authority confirmed both CPI and RPI as National Statistics.
  2. One development that escaped UKSA scrutiny was that from 2011 onwards the ONS, in an unannounced and unpublicised change, switched from RPI price series to CPI price series for calculating the deflators used in the National Income Accounts to estimate real household final consumption expenditure.  Dividing observed monetary expenditure by a lower deflator yields a higher level of real expenditure, and a look at the unrevised data shows that in the three years after 2010 the implied deflator for real household final consumption expenditure fell (relative to either price index) by approximately the full 0.9 percentage point formula effect difference between the RPI and CPI.  This means that the growth rate of real household final consumption expenditure was overestimated by about 0.9% per annum, compared to the previous method of calculation. Since final consumption expenditure makes up about 60% of GDP, this means that, real GDP growth would have been about 0.5% per year lower if had been calculated in the same way from 2011 onwards as it had been previously.   This was a really major change that went entirely without scrutiny.  It might not have affected billions of pounds of contracts in the way that a change of consumer price index did (although it increased the UK contribution to the EU Budget), but overestimating real economic growth by 0.5% per year has serious implications for macroeconomic assessment.[2]

2012-2016 Failure of UKSA Governance of Consumer Price Statistics

Internal decision to change the RPI

  1. Early 2012 saw a complete change of personnel.  Sir Andrew Dilnot’s appointment as the next Chair of UKSA was confirmed on 12 December 2011, and he took up post on 2 April 2012.  Sir Andrew was a distinguished academic economist, who had made his name as Director of the Institute for Fiscal Studies from 1991 to 2002, before returning to academia.  He had outspoken opinions of his own on statistics, one of which concerned the Carli aggregation formula, which he viewed as illegitimate.  There was also at this time a complete change of the key personnel dealing with consumer prices in the ONS.  In January 2012, a new Head of Prices Division was in post and in March 2012 a new Head of Prices Development working under him. A significant development was that in March 2012 the ONS hired Erwin Diewert, Professor of Economics at the University of British Columbia, to advise them on changes to the RPI: Professor Diewert was known as the world’s foremost advocate of the use of the Jevons formula, so the likely direction of travel was clear. Finally, in July the Statistician who had been responsible for research on consumer prices within Prices Division and was running the clothing price pilot project left the ONS.
  2. At a meeting of the ad hoc Consumer Price Indices Technical Group in April it was already clear to me that the ONS was no longer interested in the reduction of the formula-effect gap between the RPI and CPI that could be brought about by improved clothing price sampling.  This would, at best, reduce the formula-effect gap from 0.9 percentage points to its pre-2010 level of around 0.5 percentage points.  Instead, they wanted to eliminate it almost entirely, and their favoured method for doing so would be to replace the Carli formula throughout the RPI with the Jevons formula: this would turn the RPI into the CPI apart from some differences in coverage, changing it from being a fixed-basket cost-of-goods index into a constant-utility index like the CPI.
  3. In June 2012 the ONS indicated that there would be a public consultation from 8 October to 30 November 2012 on changes to the formula used in the RPI.  The President of the Royal Statistical Society welcomed the principle of a consultation but protested about the narrow scope that was being proposed, focusing only on the RPI and excluding the other significant factors affecting the performance of consumer price indices. “We had hoped,” she wrote, “that the research programme being run by the ONS would be allowed to run a full course examining all these factors and enabling them and their interaction to be assessed before changes were considered.” Professor Isham also pointed out that the consultation period was much shorter than the 12 weeks which was normally required by government guidance on public consultations and would be too short to allow the many affected organisations to gather the views of their members on such an important issue.

Unprofessional content and handling of the ONS consultation to change the RPI

  1. The ONS pressed on regardless.  The inexperienced and understaffed Prices Division faced a formidable challenge.  By early October they would have to produce a consultation document with detailed proposals and supporting arguments for changing the RPI and clear them with the “official” majority (UKSA, ONS, Treasury and Bank of England) on the Consumer Prices Advisory Committee, CPAC.  Moreover, the high-quality work that the Research Division was producing on the Economic Approach, was worse than useless.  It showed, not unexpectedly, that the Carli formula used in the RPI was a better estimator of a fixed-basket index than was the Jevons formula used in the CPI.  But it also showed that, even if one took a constant-utility index as a target, as recommended in the Economic Approach, there was nothing to choose between the Carli and Jevons: not good enough to ditch the Carli.
  2. This was where Professor Diewert was able to help.  In a footnote to his report for the ONS he admitted that, “In retrospect it was probably a mistake to include this material on the economic approach to elementary indexes in the Consumer Price Index Manual.” (He had been a principal contributor to this 2004 UN technical manual).  This provided cover for the consultation document to dismiss the whole Economic Approach from consideration as being too indeterminate to be useful.  Since the Economic Approach had been the basis of the Government’s defence of the switch from RPI to CPI for uprating and had been the reason cited by the USA and Australia when they introduced the Jevons formula into the calculation of their consumer price indices, this was a novel conclusion that could equally well have been read as a reason to retain the RPI.
  3. Having dismissed the Economic Approach, the consultation document was left short of reasons to replace the Carli formula.  It relied on two:
  4. Although few people understood all the nuances, it was clear that the ONS was claiming that the Carli had an upwards bias only because of price “bouncing” in its chained version. And there was extreme scepticism that any resulting upwards chain drift was empirically important.  The ONS was challenged to provide estimates of the degree of chain drift in the RPI by calculating comparable direct and chained versions over several years, which a previous member of the ONS Prices Development team had said would take them only about three days to do.  This the ONS refused to do and has refused to do ever since, because, as the Head of Prices Development told me, “the results would be misinterpreted.”
  5. The consultation drew an exceptionally large response, which was overwhelmingly for no change in the formulas used in the RPI. 332 responses favoured no change; and 20 agreed with replacing the Carli formula throughout or just for clothing, although not necessarily with the Jevons formula used in the CPI.  Of the responses with a statistical content, which included from the Royal Statistical Society and expert statisticians, 44 favoured no change, and 9 agreed with some replacement of the Carli, although not necessarily with the Jevons.  It was also not just a question of counting numbers.  The ONS made the responses containing statistical comment available to Denise Osborn, Professor of Econometrics at Manchester University, for evaluation. Her conclusion was: “The consultation responses made available to me provide a number of coherent arguments in support of retention of Carli aggregation at the elementary level within [the RPI], at least until further work is undertaken within ONS.”

UKSA forced to keep RPI, but removes its National Statistic designation

  1. The indications at the time and later were that UKSA would have been prepared to ignore the consultation response and Professor Osborn’s advice and press ahead with its favoured option, replacing the Carli formula in the RPI by the Jevons formula.  However, the Treasury and the Bank of England had been lobbied by gilts investors and market-makers, who convinced them that retrospectively worsening the terms of index-linked gilts by reducing the level of the RPI in this way would compromise the credibility of the UK Government and lead to an increased risk premium on all future gilts issues.  The legal requirement for the Chancellor to approve any change of RPI formula in a way that was materially detrimental to holders of pre-2003 index-linked gilts gave him a veto, and Dilnot was persuaded by the Treasury that the National Statistician should not make such a recommendation.
  2. On 10 January 2013 the ONS issued a statement by the National Statistician that a new RPIJ using a superior, geometric aggregation formula would be published from March 2013 and that, because of the “value to users in maintaining the continuity of the existing RPI’s long time series without major change” the RPI would continue, using the same elementary aggregation formulae.  There was no mention of only routine updating (although this had, in fact, been agreed two days earlier in the last ever meeting of CPAC).  On the same date, UKSA announced that it would be undertaking a re-assessment of the RPI as to whether it still qualified as a National Statistic.
  3. On 14 March 2013, UKSA duly published a short assessment, based solely on material supplied to it by the ONS, including the responses to the consultation carried out by the ONS on improving the RPI. Contrary to its usual practice, UKSA did not conduct its own written consultation or consult outside experts. It concluded that National Statistics designation should be withdrawn from the RPI and its derivative indices.  In contravention of the UKSA’s Criteria for not awarding the National Statistics designation, the assessment of the RPI did not set out steps which, when completed, would allow the designation to be awarded.  This would, indeed, have been difficult, since the operative reason for the withdrawal of the National Statistic designation was the self-imposed decision by the National Statistician to carry out only routine updating (“freeze” was the UKSA’s word), which, on a rather strained interpretation, was judged to conflict with Principle 4, practice 5 of the Code of Practice for Official Statistics which was:

“Seek to achieve continuous improvement in statistical processes by, for example, undertaking regular reviews or releasing statistical work in progress such as experimental statistics.”

UKSA freezes RPI and makes it hard to access

  1. Having decided to treat the RPI as a “legacy” measure, the ONS demoted it to the back of its consumer price statistical bulletins.  From 2017, once the restoration of National Statistic status to CPIH had removed the last need for equal treatment for RPI, it was demoted to a footnote link in the bulletins and omitted from any press notices and the ONS news blog .  The RPI data series continued to be calculated and available on its website, which was helpful for statisticians but not for the general user.  Before 2010 the ONS had developed and produced a Personal Inflation Calculator, based on RPI prices and accessible from the homepage of ONS’s website, which allowed individuals to obtain an estimate of their own personal inflation rate by inputting information about their own spending habits.  Access from the home page was removed in 2013, making it difficult to find; and it was discontinued in 2017: the lack of data on owner-occupiers’ housing expenditure in the CPI and CPIH series had made it impossible to replace it with a CPI or CPIH-based measure.
  2. The clothing price pilot research was would up at the end of 2012 without its results ever having been assessed or published.  Analysis of a large clothing dataset that had been purchased in 2012 continued, and produced an article in the ONS’ internal Survey Methodology Bulletin which found that there was very little price substitution between clothing brands, a result which, on the face of it, would support the use of the Carli rather than the Jevons for aggregation in this area, contradicting the then-current ONS assertion that the discrepancy between RPI and CPI clothing inflation was attributable to the inappropriate use of the Carli formula in the RPI.

UKSA departs from scientific standards in advocating CPIH and denigrating RPI

  1. Having lost the statistical debate, the ONS decided, quite understandably, not to try to defend the decision to remove National Statistic designation from the RPI.  Although there were intermittent high-level statements from the ONS that the RPI was not a good statistic, there was never any reference to why, or to any ONS publication that made the statistical case.
  2. Despite its desire for a quiet life, the ONS was subject to two reviews affecting its production of economic statistics: the Smith review of the governance of price statistics which reported in February 2014 and the Johnson review of consumer price statistics which eventually reported in January 2015.  The Johnson review provided little guidance on the development of consumer price statistics and was followed by a UKSA consultation Measuring Consumer Prices: the options for change, running from 15 June to 15 September 2015.  There were also reviews and consultations on measuring owner-occupier housing inflation.  More recently, on 8 March 2018, the ONS took the initiative with a paper, Shortcomings of the Retail Prices Index as a measure of inflation, in anticipation of the imminent DWP decision on Protecting Private Sector Pension Schemes.  (In the event, the DWP White Paper published on 19 March ruled out the provision of a power for employers or trustees to change scheme rules in order to apply inflation increases using CPI instead of RPI).
  3. I shall not comment on the debate on measuring owner-occupier housing inflation, since others are better qualified than I am to do so.  The Johnson review was very wide-ranging, but the chapter of most interest in a discussion of UKSA governance is Chapter 10, The use of unweighted averages in consumer price statistics.  Both there and in the short March 2108 “Shortcomings” paper the ONS had to break its silence in defending its contention that the RPI was a poor inflation index. (The Johnson review had an ONS Secretariat and was heavily dependent on ONS research support).  Both Chapter 10 in the Johnson review and the 2018 “Shortcomings” paper did little more than recycle the arguments used in 2012 (including, in the Johnson review, one that the ONS’ February 2013 response had admitted was flawed) or else relied on illustrative or made-up examples, and therefore it would be tedious for me to repeat my discussion of where those arguments demonstrate a misleading departure from standard statistical theory.  I merely note:
  4. Chapter 10 of the Johnson review has a rather more balanced review of the issues surrounding the choice of elementary aggregate index than did the ONS consultation paper but repeats its condemnation of the Carli and says that the one new piece of evidence since then supports the view that chain drift is a serious problem for the Carli index.  Despite its long gestation, the paper cited in support, which used a subset of CPI data, was published by the ONS only on 22 December 2014, so that there was no time for an independent review of it before the Johnson review was published on 8 January 2015.  Moreover, it was published online without the full dataset which would have allowed the reader to see which classes of goods were being referred to: this was made available only later in response to a specific request.  Once that was available, subsequent analysis on the Royal Statistical Society’s Statsusernet website revealed that the ONS paper had been deliberately constructed as a made-up example to show relatively high chain drift using the Carli.
  5. Recall the established statistical theory that chain drift in the Carli is likely to be a problem only if chaining takes place where prices are likely to “bounce” over the chaining period, as with monthly chaining at a disaggregated level, but will be small and quite likely negative where price inflation is relatively smooth as with annual chaining at a more aggregated level. In this case:
  6. The differences between direct and chained inflation rates were presented at the class level to give the impression that the Carli and therefore by association the RPI, was upward biased, but no attempt was made to calculated chain drift in either the RPI or CPI.  Gareth Jones had previously, in April 2013, used published ONS data on the monthly formula effect differences between RPI and CPI to conclude that chain drift in the RPI exceeded that in the CPI by an annual average of no more than 0.02 percentage points, a result that the ONS has never disputed but which was not mentioned in the Johnson review. After a careful review of the new paper, Gareth Jones concluded that it did not provide evidence of serious chain drift that would alter his previous estimate, and this conclusion has also not been disputed by the ONS.

Two examples of limited improvements to governance after 2016

(i) Operation of the Advisory Panels on Consumer Prices.

  1. In accordance with the Smith review recommendations, the two new Advisory Panels on Consumer Prices were convened in November 2015 and began operating in January 2016.  My direct experience with them was at the January 2016 meeting of the APCP-Technical.  At the joint inaugural meeting, two of the independent members had questioned why I had not been asked to serve on the technical panel, since I was one of the most prominent statistical commentators on consumer price indices.  In response, the ONS invited me to present my paper to the panel at its January 2016 meeting.  Most of the issues in my paper were familiar to experts in the area, so I took the rest of the paper as read and concentrated my presentation on the fairly new interest in fashion goods and how taking account of them tilted the balance of probabilities towards use of the Carli rather than the Jevons.  The presentation was sympathetically received by the independent members, who agreed that fashion goods did need to be factored in.  In a prepared intervention, one of them explained that the hard mathematics of the indices was well established – it was all in his 2008 book – and there was no single best measure, it all depended on one’s target and the data available.  Apart from one intervention, the ONS members were silent during the discussion of my paper, although they came alive during the subsequent discussion of clothing prices.
  2. In deference to the independent members, the minuted conclusion recorded that the Chair “noted that panel members found the mathematical properties of the elementary aggregate formulae finely balanced,” but that, “The evidence based on the clothing shows that the Carli formula is less suitable than the Jevons and Dutot formulae.”
  3. The ONS chairman had made it clear in his summing up that, having listened to my paper, he regarded the subject as closed, and the recognition by the APCP-Technical of the statistical legitimacy of the Carli index made not the slightest difference to the official UKSA line that the RPI was inherently flawed and did not have the capacity to become a good index.  The debate about the clothing index continued, but was driven from the APCP-Stakeholder, where there was an independent Chair, even though the independent members were still in a minority.

(ii) UKSA assessment of CPIH

  1. In 2014 serious data problems with the rental series used for calculating the rental-equivalence measure of owner-occupier housing costs caused UKSA to withdraw National Statistic status from CPIH, which the ONS regarded as its “headline” consumer price index.  The subsequent UKSA review of CPIH under its new Head of Assessment lasted until, in July 2017, the improvements to the quality of data and engagement with users on the different measures of owner-occupiers housing were judged to be sufficient to restore National Statistic status.  Even then, the letter confirming the re-designation was carefully worded to indicate that the ONS had established a ‘use case’ for the ONS only as a macroeconomic measure within a wider family of measures. In my experience there was a perceptible increase in ONS engagement with users at this time but with a sense of going through the motions.  There was clearly a limit to what a one-off assessment could do to affect the culture within ONS and, given the range of statistics produced, limited opportunities for follow-up.

III Lessons for the Governance of UKSA

  1. For statistics to be trusted, practical independence from government influence in their production and deployment is essential.  The biggest blows to trust in consumer price statistics came when successive Chancellors of the Exchequer by-passed supposedly independent arrangements in 2003 and 2010 by choosing to use existing indices for novel purposes.
  2. But accountability is equally important.  Elected governments have to take account of public opinion and daily parliamentary scrutiny, but an independent institution, particularly if it has a claim to technical expertise, has the temptation of thinking that it can act entirely as it sees fit. This was the case with UKSA from 2012 onwards.
  3. A third essential requirement – and a prerequisite for accountability – is transparency.  For the UKSA this involves, in the first place, openness and engagement with external statisticians.  The existence and strength of the Royal Statistical Society has been an asset to the development of UK government statistics and in the past there has been fruitful co-operation, but, to work well, a desire for openness has to come from within UKSA.  In the second place, there needs to be engagement with the users of statistics, something that does not come naturally to the ONS statisticians even when their actions and decisions will have large financial and human consequences – and their remoteness in Newport makes this even more challenging.  Lessons in engagement with both experts and users could be taken from the Food Standards Agency, which has put maintaining consumer trust in food regulation as a central objective.  This is demonstrated in practice by making most Board and committee meetings open to the public and by having scientific advisory committees with strong independent membership that follow all the requirements in the Code of Practice for Scientific Advisory Committees– something that the APCP-Technical is very far from doing.
  4. The multiple roles played by both the Chair of the UKSA and the National Statistician have often been remarked upon.  The UKSA is structured to try to separate its role in the production of statistics from its role in the regulation of statistics but both activities are under the chairman of the UKSA and the dangers of his involvement in both roles were demonstrated in January 2013.  The National Statistician has even more hats, being Permanent Secretary of the ONS, Head of the Government Statistical Service, and, as National Statistician, statistical adviser to the Government.  Leaving on one side the governance of Statisticians in other departments, there is clearly a problem in the ONS when statisticians feel that they are being asked to act unprofessionally and have no one outside their line of management to turn to for support.  I was particularly aware of this before and after the publication of the Johnson review, where some of the statisticians were uneasy about the advocacy role that they were being asked to play. Again, the FSA provides a good example, where a Chief Scientific Adviser and a transparent Science Council are there to ensure that objective science from both external and internal sources informs their activities.
  5. Particularly on high-profile issues, it is necessary to ensure good governance not only in producing trustworthy statistics, but also in choosing which statistics to produce to meet the needs of users.  In the case of consumer price statistics, it is tempting to think that the existence of two consumer price indices is itself the major problem, since it allows index shopping.  However:

 

February 2019

[1] Mark Courtney, Consumer Price Indices in the UK, available at https://www.statisticsauthority.gov.uk/wp-content/uploads/2016/11/Consumer-Price-indices-in-the-UK.pdf

[2] This change to the calculation of GDP has attracted so little attention that, even though my own expertise is not in national accounts, my January 2015 contribution is worth a reference.  It is in the final, non-technical section, “Macroeconomics” in: Mark Courtney, The Justification for the Retail Prices Index (January 9, 2015), available at SSRN: https://ssrn.com/abstract=2547506 .  The ONS has now revised all the back series for the National Accounts, re-writing economic history.  American research on the effects of a similar change in the US shows that increasing the measured annual real rate of growth in this way leads to implausibly low estimates of the standards of living in earlier periods.