Written evidence from the Quality Assurance Agency for Higher Education (QHE 34)

Executive Summary

1                         The Quality Assurance Agency for Higher Education (QAA) is the independent
body that safeguards the public interest in quality and standards in UK higher education. QAA plays a key part in maintaining the world-class reputation of UK higher education.
We believe that an independent body providing quality assurance is an essential part of a sustainable, integrated, internationally respected higher education quality assurance system.

2                         The UK Quality Code for Higher Education (Quality Code) sets out a framework for academic quality and standards, provides cohesion, and underpins the international reputation of the brand of UK higher education. It contains sector-wide expectations for students' academic experiences, including learning, teaching and assessment. The Quality Code has an essential role to play in future arrangements for quality assurance.

3                         QAA fully endorses the need for the quality system to evolve and innovate, and has listened carefully to students and higher education providers, but we are concerned that there could be unintended consequences arising from the funding bodies' proposals in the Quality Assessment Review (QAR).

4                         QAA can lead reform which will provide confidence to students, employers and taxpayers about the quality of our higher education. Some of our thinking is set out in our response[1] to the funding bodies' QAR consultation document. We can design and implement a flexible form of independent external review, tailored to the needs of individual providers. This would build on the strengths of the funding bodies’ proposals without losing those elements of the current system we consider critical.

5                         We believe all students deserve excellent teaching and an excellent academic experience.  We have offered our expertise to the Department for Business, Innovation and Skills as they design the proposed Teaching Excellence Framework (TEF). QAA's aim is to ensure it is delivered efficiently and well, with results students will value.

6                         The approaches for quality assessment and teaching excellence must be developed together to ensure a coherent system. We can deliver this integration.

7                         However it is reformed, the UK system of quality assurance should retain core principles of co-ownership and co-regulation. It should not reside in a state-established regulator, nor be premised on self-regulation. Rather, it should be an approach which is based on an active partnership between the higher education sector, including students,
and government. QAA stands ready to play its part in a revised quality assurance system and TEF.

8                         This evidence provides a short explanation of QAA's role. It also answers
questions 1-5 posed by the Committee.

The role of QAA

9                         QAA is the UK's independent higher education quality body. We are a charity, and our governance structure is designed to balance the interests of students, providers, funders, the professions and the four nations.

10                     We cover all parts of the UK, whilst at the same time meeting the higher education needs of each devolved nation. We review UK higher education wherever and however it is delivered – from universities and further education colleges, to private and not-for-profit colleges (often known as alternative providers), as well as UK higher education delivered internationally (transnational education). The Quality Code includes a set of standards for providers to meet, known as 'expectations'. It has been developed by QAA over a number of years, as a collaborative enterprise with over 3,300 contributions from the sector and students.

11                     We contract teams of independent academics and students to conduct our reviews, which means the judgements they reach are peer-to-peer. In 2014-15, we conducted 337 review activities and will complete over 360 in 2015-16.

12                     QAA's work internationally protects the world-class reputation and brand of UK higher education, for example through our reviews of UK transnational education, through formal strategic agreements with quality bodies in 14 other countries, in meeting European and wider international standards for higher education quality assurance, and our inclusion on the European Quality Assurance Register for Higher Education (EQAR).[2]

13                     QAA has proven itself capable of adapting to evolving government and sector requirements for quality assurance, and leading the necessary change to meet those requirements. For example, the current approach to quality review was developed in close consultation with the funding bodies and the sector. QAA has also been agile in responding to government requests for quality reviews of alternative providers.

Question 1: What issues with quality assessment in higher education was the Higher Education Funding Council for England's (HEFCE) Quality Assurance Review seeking to address?

14                     The drivers for the QAR were stated when the review was launched on 7 October 2014 by the three funding bodies.[3] They were to develop 'risk-based, proportionate, affordable and low burden' approaches built on 'established strength and good practice',
and to 'demonstrate value for taxpayers and students'.

15                     Subsequent consultation documents have focused on the effectiveness of cyclical, external quality review, the UK Quality Code, degree classifications and standards, and the role of external examiners in quality assessment. The drivers have thus changed and broadened during the course of the review.

Question 2: Will the proposed changes to the quality assurance process in universities, as outlined by HEFCE in its consultation, improve quality in higher education?

16                     We believe some of the proposals could have unintended consequences that will damage the international standing of UK higher education, are contrary to international best practice, break the UK consensus and introduce an unnecessary divide in the sector by omitting provision that does not receive public funding.

17                     The proposals to replace independent external peer review with HEFCE's own accountability reviews would risk the international credibility of our higher education sector. England, Northern Ireland and Wales would not be compliant with international expectations as set out in the European Standards and Guidelines for Quality Assurance (ESG).[4]
This matters for UK universities and colleges offering courses overseas, because other countries want to know that international standards are being met. The funding councils' proposals have caused some international concern.[5]

18                     The proposed merging of the roles of funder, regulator and quality assurer creates a potential conflict of interest and is contrary to good practice. The separation of these roles was a specific recommendation in two recent reports on regulation by the Higher
Education Commission (HEC) (2013)[6] and Universities UK (2015).[7] The HEC said that
'QAA… needs independence (not least from funding allocations) for its legitimacy', and UUK explained, 'There remain valid reasons for the oversight of academic quality to be performed through a separate body… not least given the importance of co-regulation and
co-ownership in this area.'

19                     The proposals include the transfer to HEFCE of responsibility for the Framework for Higher Education Qualifications and minimum standards for quality, currently contained in the Quality Code. This has been opposed in sector responses to the consultation, notably by Universities Wales,[8] as it breaks down the UK consensus and shifts control over academic standards to a government body. Scotland has chosen not to participate in the QAR process. 

20                     The funding bodies' proposals introduce a false divide as they only deal with those providers accessing public funding; an incoherent approach which students and the public will not understand. QAA has reviewed alternative providers since 2011, and more recent reforms have ensured they now have to meet the same standards as long-established institutions.

21                     We support the aims of the funders' proposals for more risk-based and proportionate approaches to quality assurance, and better use of information and data about student outcomes. We believe, however, that these aims can be achieved without the threat to the sustainability of the quality of UK higher education and to our world-class reputation. There are other ways of designing a better quality assurance framework. Some of our thinking is set out in our response to the funding councils' consultation document. We can design and implement a more tailored approach to quality assurance based on a flexible form of independent external review, and combining metrics with institutional context to inform improvements for students and the sector.

22                     We agree with those calling for a move away from rigid cycles of external review and a greater emphasis on improving the quality of higher education. The think tank Million+ explains that 'new providers may be relatively inexperienced in quality assurance, and are likely to require a greater degree of oversight and support than those that are more mature in order to ensure that the interests of students and the reputation of UK higher education are protected. It is sensible that those that have been part of the sector for many years, and that are able to demonstrate strong track records of success, experience a quality assurance regime that is proportionate to their deep experience in higher education'.[9]

23                    We can deliver a unified system where all providers demonstrate the same high quality in order to be part of the UK higher education system. Students and employers want to know that qualifications have value and longevity. We also note the recommendations[10] of the Competition and Markets Authority for a system which is 'competitively neutral' so that ‘different types of providers have the opportunity to be bound by the same rules.’

24                     In its response[11] to the QAR consultation, the National Union of Students referred to external review as enabling students 'to fundamentally influence quality enhancement policy and practice'. We would retain meaningful student engagement in quality assurance and improvement, something which QAA has led internationally; an academic paper comparing different countries' approaches stated in 2014 that 'in the UK, the engagement of students in external review is extraordinary'.[12]

25                     The processes of self-assessment and expert peer review provide value for institutional leaders as levers for change, and for students and taxpayers as an external verification of a provider's claims. It is unclear, based on current evidence, whether the QAR proposals will deliver efficiencies, reduced burden on providers or value for money. In its response to the funders' consultation, the Russell Group of research-led universities said: 'Ultimately, there is a risk the resulting system could be more expensive to universities, students and the government'.[13]

26                     The QAR must also now be considered in the wider context of other policy developments, particularly integration with the TEF. History has shown that the system works best when the sector cooperates and works together.

Question 3: What should be the objectives of a Teaching Excellence Framework (TEF)?

27                     The TEF offers the opportunity to focus specifically on improving the academic experiences of every student. To do this, it should cover all types of providers to support the experiences of all students, no matter where, what and how they study, or how that study is funded. It means that the TEF will need to be highly flexible and adaptable, and able to recognise excellent teaching and learning in all its forms, from part-time distance learning to full-time postgraduate research degrees. The close and ongoing involvement of the sector in its development will also be important.

28                     A TEF constructed in this way should aim to:

• drive up improvement (by identifying excellence) and deter poor practice
• ensure excellent teaching is encouraged and rewarded, with high quality teaching and learning held in the same esteem as excellent research
• focus on improving outcomes for students
• encourage the development of skills and attributes that are valued by employers.

29                     A further objective should be that it forms part of one coherent quality system, taking account of the UK-wide context. 

1. How should a TEF benefit students? Academics? Universities?

30                     Students should be the primary beneficiaries of the TEF and it should be designed with their needs at its core. In broad terms, a TEF that helps providers evaluate and continuously improve students' academic experiences will, of course, benefit students.
The assessments made should help existing students push for improvements during the course of their studies, should provide meaningful information for prospective students making their choices and should help graduates by providing reassurance to employers.

31                     Academics, particularly those with a passion for teaching, will welcome the opportunities for professional recognition and career progression that a TEF could deliver.

32                     Universities (and importantly, colleges and alternative providers) that are recognised for their excellence will receive reputational boosts which will directly affect their marketing and recruitment. Those further behind will be identified as needing additional support and that should be made available to them, if the system as a whole supports enhancement.

33                     A well designed TEF will enhance the overall reputation of UK higher education, bringing benefits not only to the sector and students, but to the UK economy as a whole.
It could also be a world-leading development, which other countries will look to as they develop their own systems.

34                     Good teaching should, of course, lead to highly employable graduates; innovative thinkers who contribute much to society, culture and the economy.

2. What are the institutional behaviours a TEF should drive? How can a system be designed to avoid unintended consequences?

35                     A TEF should encourage and reward institutions that strive for continuous improvement, and aspire to and deliver the best possible outcomes for students. This will take different forms depending on the purpose, student body and culture of the provider.

36                     Data about the variety of students' experiences and outcomes is clearly invaluable for evaluation of providers, but care must be taken with which measures are used. One of the potential consequences of the TEF is that the metrics chosen skew outcomes in favour of particular types of institutions. The causal link between teaching quality and student outcomes is also complicated, with other factors such as 'social capital' having an impact. Contextualising the data to take account of the characteristics and operating environment of an institution will help avoid this. For example, higher education in further education colleges has a distinctive nature, often draws a greater number of students from a widening participation background and may have a stronger vocational element.

37                     QAA supports in principle the idea that the TEF could include assessment of how universities and colleges support and retain students from less advantaged backgrounds. This is particularly important as research[14] shows this group achieves particularly highly if supported to complete their courses. Indeed, the TEF could also look at how international students are supported; another group that benefits from tailored support.

38                     The system should be designed to ensure that it does not encourage short-term behaviours, such as recruiting temporary staff in an attempt to boost ratings before dropping back to previous staffing levels. This sort of 'gaming' of the system does not give students clarity or empower them to make informed choices.

3. How should the effectiveness of the TEF be judged?

39                     We have offered our advice to BIS in the early design stage of the TEF. Measures of success could include:

• clear differentiation of TEF outcomes and what they mean
• increase in student satisfaction and/or perception of value for money for students on 'excellent' courses
• clear and increasing public understanding of the TEF and what its categories mean
• significant and rising numbers of students using TEF outcomes to help make their choices
• those providers identified as excellent in some subjects, gaining 'excellence' increasingly in others
• evidence of widespread engagement in enhancement of learning and teaching.

40                     To offer maximum benefit to students, without causing undue burden to providers or the taxpayer, we believe the TEF should be fully integrated with other regulatory and quality assurance systems.

Question 4: How should the proposed Teaching Excellence Framework and new quality assurance regime fit together?

41                     A single, integrated system is essential. It will be easier to implement, easier for providers to manage, and easier for students, employers and the wider public to understand.

42                     In a more tailored and flexible approach to quality assurance – combining metrics with institutional context to inform improvements for students and the sector – a TEF would be integrated through:

• similar assessment approaches with peer review and student involvement at their core
• common metrics and institutional context
• TEF criteria drawn from the standards used in the broader framework (that is, the Quality Code chapter on Learning, Teaching and Assessment)
• conclusions from each process informing the other
• a common framework for publicly reporting the outcomes from each.

43                     The TEF should be built on a robust, independent and external quality assurance framework that guarantees a high quality baseline for all providers, with excellence sitting above it.

Question 5: What do you think will be the main challenges in implementing a

Teaching Excellence Framework?

44                     A key challenge to the TEF is that, in order to achieve its objectives, the sector and students need to be engaged in its development, and currently there is no widespread agreement about what a framework will look like and achieve. Although the proposals are currently for England only, there might also be further opportunities to engage more widely across the UK.

45                     The timetable for the introduction of the TEF inevitably presents challenges. It is ambitious, but rightly so. We note the Secretary of State's comments in evidence to this Committee that 2017 is a 'workable' deadline but that the priority is to 'get it right'.[15] We agree with this assessment.

46                     There are also practical considerations to ensure universities and colleges are able to plan and coordinate their quality systems, setting of fees, and information for prospective students for the next academic year and beyond.

47                     The Minister for Universities and Science has said that the TEF will 'be an important signal to students of where quality resides, discipline by discipline, institution by institution'.[16] At the discipline level, the Joint Academic Coding System (JACS)[17] could be used. Institutions will already have provided data against these categories for other purposes, and consistency across the sector will allow for clear comparisons and judgements across a diverse range of higher education provision.

48                     It will be important to ensure a clear public understanding that, whilst the TEF will recognise excellence, the UK's external quality assurance framework sets a high quality baseline for all higher education providers, whether or not they are assessed as excellent. This will help avoid any potential reputational risk to the UK brand and individual institutions. The TEF must have clear, differentiated judgements, and the outcomes must be easily understood by students and their families, and employers, both in UK and internationally.

October 2015

© The Quality Assurance Agency for Higher Education 2015

Registered charity numbers 1062746 and SC037786

www.qaa.ac.uk

7