Written evidence from Mr Thomas King – RIS0020

The opportunity to reflect on the subject coverage of investigative inquiries by House of Lords Select Committees is welcome. I am a social statistician and member the Royal Statistical Society and a member of their Social Statistics Section committee. Statistical evidence bears on much that is important and uncertain about public policy by analysing social data abstracted to a national perspective. Interpretation and criticism of statistics have broad relevance to investigative inquiries and this is a personal response focused on the scrutiny of social statistics.

Summary

Recent investigative inquiries by ad hoc Select Committees of the House of Lords highlight the need for well-designed social statistics. Important recommendations concern:

•        the priority of comprehensive UK social statistics about demographic change;

•        the neglect of other parts of the population caused by threshold targets such as progression to university;

•        the demand for comprehensive official statistics on matters of political importance such as inequality;

•        locally relevant and comprehensible social data for purposes ranging from careers advice to public service delivery.

Social statistics is a theme in these investigative inquiries; no secessional Select Committee is scrutinising social science in general, or social statistics in particular. This specifically implies:

•        skills and infrastructure issues in social science are rarely scrutinised;

•        expertise from related inquiries is not carried forward to the extent it could be;

•        inquiry timeliness, topic salience and incisiveness are restricted.

Evidence-informed policy is a government commitment recently bolstered by the expectation that local policy innovation should be appropriately evaluated. Widely acknowledged market failure in undergraduate quantitative social sciences skills (for which effectiveness of policy response might be scrutinised) suggests the availability and quality of such evidence cannot be taken for granted. Investigative inquiries into policy reliant on social statistics should more prominently scrutinise:

•        the appropriateness of social conceptualisations and analytical assumptions used in policy;

•        the sufficiency and comprehensibility of social data used in informing all stakeholders;

•        the capability of responsible commissioners in modelling and evaluating their policy.

Introduction

1              Noting the 'Jellicoe review' received evidence about the selection of inquiry topics[1], both their timeliness and significance, this submission reviews the topics covered by the recent increase in ad hoc committees in respect of social statistics, and the implications for effective scrutiny by other select committees. Generally topics of social science lend themselves to the investigative inquiries of Lords Select Committees as they cut across departments: some general aims of policy that rely on social constructs which are surprisingly hard to pin down e.g. 'life chances' or 'health'. Achievement of targeted policy by classifying social deprivation and household structure can be taken for granted in policy development and demands appropriate scrutiny.

Official Statistics

2              When statistics became independent under the Statistics and Registration Act (2007), residual ministerial oversight was formally moved to the Cabinet Office. The Lords Liaison Committee at that time considered a proposal that a Joint Committee of both Houses be established to scrutinise statistics[2]. One of the arguments at the time was the considerable expertise of members in the Lords. However, the committee responsible for scrutinising the other work of the Cabinet Office (then PASC, now PACAC) has that responsibility. Yet they also expect that other Select Committees call statisticians when, for example, issues of statistical definitions bear on departmental policy, rather than having a monopoly.

3              Specifically the cross cutting focus of Lords inquiries and the expertise of members should be brought to bear on matters of statistical definitions where these bear on policy, and they should engage with ONS statisticians as necessary. An example of this need is seen with the classification of students in international migration[3], where the importance of internationally consistent definitions was overlooked (and contested) by inquiries in the Commons. Similarly where committees inquire and find a lack of statistics to understand society or support policy, they should make relevant recommendations. While PASC recommended compendia to be compiled on the statistical base for referendums on Scottish independence and leaving the European Union, other topics cut across departmental responsibilities. Timely Lords inquiries on cross cutting topics should critically examine the sufficiency of official statistics, for example those on inequality and ageing are only recently being reviewed by ONS.

Specific ad hoc committee inquiries

4              Political polling and forecasting (or projecting) demographic change are core social statistics activities, so it is notable that these two are among the inquiries since 2012. The former noted limited written submissions and the latter expressed surprise that such forecasts relating population change to public service provision were not routinely made. Social mobility research embraces all of the issues of measurement, coverage and structure in longitudinal studies and that inquiry impressed by its depth and focus. Again coverage of the whole population in planning, which suitable analyses immediately support, is rather rarer than the analyses of problem groups (e.g. NEETs) and spending (e.g. on HE).

5              The long term sustainability of the NHS is driven on the demand side by changes in our population and chronic disease morbidity, and the level of care provided in society by family and community structures, which vary geographically. On the supply side, changes in the skills mix of the workforce and the demographic changes in workforce composition are instrumental and are outcomes of social mobility. Reconciling coverage of the whole population and the assumptions about the complexity of the process builds on both the demographic change and social mobility inquiries.

Themes

6              Themes in the nature of the ad hoc inquiries supported since the 2012 move to more ad hoc committees offer an insight into gaps in coverage of scrutiny offered by other select committees. While international relations has been identified as deserving a sessional committee, there is a further theme in social statistics which points to a longstanding limitation. Social statistics comprises methodology for the collection of data and inferences in social science, including opinion polling, social surveys and social measurement. These bear particularly on changes of the composition our society requiring studies which have validity in coverage and constructs, as well as appropriate precision.

7              Recently announced ad hoc inquiries into intergenerational fairness, the rural economy and regenerating coastal and seaside towns[4] also have this commonality. And to some extent, some past inquiries of ad hoc select committees have added the social science dimension onto more technical or economic uses of data, including the affordability of childcare, and financial exclusion. The concluding inquiry on social and ethical aspects of artificial intelligence can see that how data should be shared and ought to be used transcends the legal obligation of the owner of the data originally collected with consent for a specific administrative purpose. While these inquiries reflect the agility of the system in taking up such topics, they also highlight that such gaps often appear between the departmental responsibilities of Commons committees.

8              Although the Science and Technology committees in both Houses nominally include social science in their purview, such scrutiny is very rare. For example, the ethics of social research receives much less scrutiny than comparable epidemiological work despite similar practicalities. Notwithstanding ongoing work on Research Integrity, Algorithmic Accountability, Early Intervention, Screen Time and Social Media in the Commons, the most recent report is from Census and Social Science in 2013[5], and the corresponding Lords committee has nothing on social science in this period. While POST does consider it a focus, social science is more likely to feature as an inquiry topic in the sense of Science and Technology Studies (STS), i.e. sociocultural examination of scientific practices, than social science per se.

9              As well as not making best use of the House’s expertise, it is possible that some issues are being left too late to have useful influence, or overlooked entirely. Some other concerns arise from the use of ad hoc committees to be exclusive in addressing topics of social statistics:

•        securing suitable submissions critiquing the quality of evidence used to develop policy;

•        timeliness of inquiries in responding to cross cutting issues;

•        committee depth and continuity of experience;

•        and ensuring scrutiny of provision for skills, resources and capability.

Aspects of these issues raising concern are described in the remainder of this submission.

Expertise

10              Sometimes committees have been surprised by evidence provided describing the nature of planning procedures, the data available and the complexity of the process. In other cases the members of committees are evidently learning about the topic during their inquiry, and while this public accountability of experts is essential to the process, expertise on committees is sometimes less evident than the reputation of the House. Where ad hoc committees follow on aspects of their inquiry from a previous committee, they should retain some expertise by including members from the first inquiry on the second. This will be most relevant for technical aspects which feature in several ad hoc inquiries, as identified with social statistics. Building up such expertise will also facilitate accountability for the provision of statistical information on topics such as that on social inequality which is currently developing.

Skills

11              Skills in respect of STEM are the subject of periodic inquiries of the Science and Technology Select Committees; and these have recently been complemented by topical inquiries about Digital Skills. Meanwhile skills in social science disciplines have not attracted such scrutiny despite substantial investments in training, particularly for advanced quantitative skills. Research methods training was identified as needing the establishment of a national centre by the ESRC[6], and provision of quantitative training for undergraduates so poor that it was dubbed ‘market failure’.[7] Although committees have from time to time expressed concern about the lack of a chief social scientist in government, skills have experienced a lack of scrutiny which might have expedited interventions. It is also notable that economics teaching has been criticised for its uncritical treatment of model assumptions and that psychology has been struggling with replication for many years and only recently attracted parliamentary attention (about research integrity). Statistical skills in policy will become more pressing as departments respond to the new expectation to reserve budget to evaluate innovations which was announced by the Treasury at the recent What Works Centres anniversary.[8]

Data resources

12              While the future of census did attract one (Commons) inquiry in relation to social science, the concerns about household composition were not followed up, but they remain difficult to derive in an administrative data alternative. Other resources in social science include an archive of both data and questions, some centrally collected and others by external deposits, which offers a template for sharing data which other disciplines should be encouraged to match. However these large investments carry certain risks and represent prioritisation of resources which have attracted no scrutiny, despite the cancellation of the ‘Life Study’ birth cohort and consequent international review. These resources have a wider international context of comparability with other studies around the world, linkage of such large studies into administrative data, new epigenetic possibilities, and a similar cancellation in the USA. Meanwhile some understanding of policy requires long standing sources of data planned many years ago, which cannot be substituted by other sources.

Capability

13              Quantitative skills are important in applications to social policy in commissioning and conducting evaluations, as well as synthesising existing evidence. More cross cutting inquiries in the House of Lords should scrutinise such substantive issues, statistical definitions, skill and capacity, and research data resources. Ballooning quantities of data from administrative systems and online transactions all relate to individuals, and increasingly this data collection is designed through targeted advertising etc. Development of statistical methodology can often fall between stools, too applied for the mathematicians and too abstract for the substantive areas, but in social science this experience is particularly acute. So utilisation of data and sceptical consideration of the assumptions in statistical models to support policy development is more limited than it should be.

Conclusion

14              Scrutiny of social science is comparatively overlooked in investigative inquiries by sessional committees of both Houses but it often has a cross cutting character. Social statistics is apparent for its importance in public policy both in availability of official data and the assumptions made in policy planning. Ad hoc investigative Select Committee inquiries may not be sufficiently effective or efficient in ensuring the use of social statistics in policy and are likely to overlook skills and resources issues entirely. Capability in using social statistics in policy, ranging across planning, commissioning, evaluating and reviewing evidence is becoming ever more important. The current inquiry should reflect on the efficacy of present arrangements for how social science is scrutinised by the Select Committees of the House of Lords.