Written evidence from National Union of Students (QHE 41)

 

Introduction

  1. The National Union of Students (NUS) is a confederation of more than 600 students’ unions, representing more than 95 per cent of all higher and further education unions in the UK. It is a voluntary membership organisation that makes a real difference to the lives of students and its member students’ unions. Through our member students’ unions, NUS represents the interests of more than seven million students.

 

Quality assessment and HEFCE’s Quality Assurance review

  1. The issues that the Higher Education Funding Council for England’s (HEFCE) Quality Assurance review sought to address (on behalf of the funding councils for Wales, Northern Ireland and England) are outlined in their consultation document.
     
  2. For NUS, the issues raised by students’ unions that we hope to be addressed by any quality review are:
    1. Ensuring that student voice and engagement in the assurance and enhancement of their education is not just a tick-box exercise, and reflects a genuine partnership between the student collective and the institution, paying particular attention to ensuring that these systems are accessible and involve a diversity of students.
    2. That the quality of education is being assessed, which goes beyond assuring the quality of the processes in place.
    3. That the system focuses appropriate scrutiny and assurances to students across the whole sector, and that no matter where a student studies they can be assured that they will be part of an excellent educational experience.

 

Improving quality in Higher Education: HEFCE’s proposed changes to the quality assurance process in universities

  1. NUS has been actively involved in the Quality Assessment review, including participating in the steering group and formally responding to the consultation.
     
  2. As the discussions and intentions of the Government have developed over the past few months, NUS has become increasingly concerned about how the different policy decisions will inter-relate with each other to the detriment of students. However, we reserved significant judgement for the final detailed stage.
     
  3. In particular, NUS and students’ unions are concerned about the ability for students’ ideas, concerns and opinions to be genuinely taken account of, the sole focus on outcomes measures across all the new proposals– rather than a commitment to engage students in meaningful conversations about their teaching quality and the need for robust mechanisms to be in place for when things go wrong at course, as well as institutional level.
     
  4. All discussions within the confines of the HEFCE consultation have been predicated on there being a high bar of entry to the publicly funded system. The initial quality assessment must be high, to give confidence in a risk-based, low burden system thereafter. As such, the current discussions led by the Minister for Universities and Science regarding lowering the barriers to entering the sector are of great concern to NUS.
     
  5. NUS is greatly concerned that this combination of policies will create a regulatory environment that not only fails to prioritise students’ education, but puts it as a secondary to the creation of a market-based quality system. Such a system not only acknowledges institutions and courses might fail, but actively encourages them to, as part of an ideology that such market failure will, over-all drive up quality through competition.
     
  6. Whilst this ideology is one that NUS opposes in principle, it is also in practice that a system that encourages wholesale institutional failure is intolerable for students. What is forgotten in this boom and bust ideology, is that it is people’s lives that are being played with. Vast amounts of time, money and effort are invested by students, and despite the belief that students can easily move between courses and institutions, it has been proven time and time again, that this is not the case. Credit transfer is a huge barrier, but we know that students from low socio-economic backgrounds, women, mature, part-time and BME students are all more likely to be less mobile and therefore less able to change their location of study. In addition, at the current time, and in the proposals, the protections provided to students, both as preventative measures, and as measures for when course fail are completely inadequate.
     
  7. Ultimately, quality is determined by institutional response to the market. The market creates unhelpful pressures and distortions on university provision, which undermine quality. Quality assurance can only go so far – the only way to ensure institutions have the freedom to innovate and improve is to remove the market forces that produce perverse outcomes.

 

The main challenges in implementing a Teaching Excellence Framework (TEF)

Timescales

  1. NUS has no confidence in the timescales for rolling out the TEF. The timescales appear to show a lack of understanding on the part of government as to the complexities of measuring quality in higher education, and perhaps more worryingly, suggest that the motivations of government in implementing the TEF are more geared towards rushing through a mechanism for raising tuition fees and opening the sector up to more competition from private providers rather than focusing on teaching excellence.
     
  2. This view is supported by the fact that, rather than allow sufficient time for consultation and fine tuning the mechanisms for implementation within institutions, the Government aim to pursue a “soft” version of the TEF first, in order to allow existing data to be used at an aggregate level as a heuristic for excellence.

 

Measuring quality effectively and appropriately, particularly in relation to employment outcomes

  1. We are not confident in the use of employment outcomes as a measure of quality as we do not believe that these tell us very much about the quality of learning. The economic return on a degree is affected by many things that are outside of the control of the higher education sector and, indeed, the individual student. Put simply, the direct link between education and income is challengeable, and that, hitherto, the empirical evidence that is used to explain the link is highly dubious.

 

  1. The background of students clearly has a large effect on their employment prospects, and this holds true regardless of degree classification, subject, or institutional reputation. Ultimately, if you’re from a more privileged background with the right social capital and networks, you are far more likely to end up in a good job. Measures of economic output from degrees are heavily distorted by this fact, because particular types of student from particular backgrounds are drawn to them (law and accountancy are probably good examples).[1]

 

  1. Gender has a significant impact on graduate employment and the graduate premium. Employment outcomes and salary rates will be skewed at institutions depending on the number of women graduating from a particular course. There could well be perverse incentives generated by a focus on employment outcomes to recruit less women in some subjects in order to skew employment outcomes towards higher earning male graduates.

 

  1. Higher education cannot protect people from changes in the labour market due to global economic forces or political intervention and it is wrong to link excellence to economic and political issues beyond the control of higher education institutions.
    1. In the case of economic change, we can cite many examples of how changes in supply and demand for skilled labour in certain areas has changed both employment rates and incomes of graduates with certain degrees. The most recent example would be economics, finance, and business graduates who have been affected by salary cuts and job losses in the City and wider finance and banking sector.
    2. In the case of political intervention, we can imagine the impact on graduate economic outcomes should a government be elected that changes the pay of teachers or doctors. Austerity is having an impact on public sector wages and further cuts will seriously impact on the income of many graduates.
       

Preventing perverse market incentives

  1. No matter what the TEF looks like, the pressures of market competition will intensify the need for institutions to focus on short-term decisions that often involve finding ways of cutting corners and gaming the system.
     
  2. Institutions may well find it easier and more desirable to find illegitimate ways to improve the figures in metrics rather than actually focusing on improving the quality that the metric is supposed to measure. We see from use of the National Student Survey in some institutions that institutional-level market information used to drive market competition leads institutions to focus their attention on the data rather than the underlying drive to improve standards. The risk of gaming is far higher in a TEF that is based on quantitative data and outcome metrics, as statistics are often easy to manipulate.
     
  3. There is also a key problem with a TEF creating incentives for institutions to limit their focus to the delivery of courses which are the most likely to drive up TEF results. Institutions have been known to close down courses that are of excellent quality purely because they do not attract the right students to maintain their reputation as a selective and prestigious institution. Excellent courses may also be dropped because they do not produce the same employment outcomes as other courses, or require far more effort on the institutions part to improve the social capital of the students on the course in order to lead to better outcomes. These risks are especially high if the TEF is focused on output measures around employment outcomes and salary rates.

 

Applying comparative metrics at subject level

  1. TEF will not work at an institutional level because there are likely to be important variations in teaching quality by subject and department. Without generating “smart” metrics to account for subject-specific effects and challenges, an institution cannot identify specific areas for improvement.

 

  1. Any “soft” version of TEF at institutional level, using aggregate statistics and broad metrics would undermine institutions by averaging out quality across the whole institution and encouraging a “broad brush” approach to improving quality, which is less effective.

 

  1. It also would not take into account the fact that improvements would undoubtedly have to happen at a departmental or even a course level, and tailored recommendations and approaches to driving up quality would require far more detailed and subject-specific evidence.

 

The proposed connection between fee level and teaching quality

  1. NUS completely opposes any attempt to further raise tuition fees, and believes that students have a right to excellent education, no matter where they choose to study. But we can also highlight several reasons why a framework designed to measure teaching quality should not be used as a means of setting the price / inflationary cap on tuition fees.
     

Price does not represent quality in higher education, not even in uncapped markets

  1. It is clear that the tuition fee levels set at institutions do not in any way represent the quality of the teaching on a course. Cuts to direct teaching funding for institutions, the need to raise revenues to cross-subsidise, and the lack of market incentives to compete on price have meant that most courses at most institutions will cost around £9,000. Whether you study economics at London Metropolitan University or the London School of Economics, a home student will pay £9,000 a year for their BSc.
     
  2. There is some variation in other uncapped fee markets, such as in international fees and postgraduate taught fees, but the differentiation is mainly down to the boldness of a few institutions to utilise their reputation, brand power and position in the market to ensure that they can attract enough students at the premium price. There are also no real access regulations on the uncapped fee markets, so if the students are all wealthy overseas men, it isn’t so much of an issue. This is why LSE can charge up to £30,000 for an MSc in Economics where others in the Russell Group might only charge £4,500. Thus, while we may infer that the course at LSE is of high quality, and this is a necessary condition for it to maintain its reputation and subsequently its high fee price, it does not explain the huge difference in price from another highly selective institution with a good reputation, as this institution will probably also provide a very high quality course.
     
  3. Considering these facts, there is no reason to expect that inflationary variation in fees could create a market where the price is differentiated by underlying teaching quality in any meaningful way.

 

Linking fees to the TEF could adversely affect market behaviour by providing misleading information to prospective students

  1. Building on the issue above, it seems likely that what differentiation in fees will do is act as a proxy for teaching quality, but not truly represent it.
     
  2. We are already seeing in the postgraduate and international markets that institutions are raising fee levels in line with more prestigious institutions in order to make prospective students believe that the high price is a sign of good reputation and, subsequently, good quality.
     
  3. The problem we have in the unregulated HE markets is that fee levels have been allowed to drive reputation, rather than fees following reputation. International students in particular, who rely on imperfect and often biased information from agencies, are being led to believe that price equals quality, when this relationship is clearly more complex.

 

  1. With the logic of higher education as a “Veblen Good” already fairly commonplace in the UK, it seems reasonable to assume that linking fee increases to a TEF stamp of approval will have a similar effect. The central problem here is that every university will start off from roughly the same point, and as we already know that the fee level has hitherto had nothing to do with teaching quality, future differentiation in fees will be down to a) when an institution “passes” the TEF, and b) the inflation rate.

 

  1. Inflation is variable, which means fee increases are variable. This means that the variation in fees is mainly down to the state of the economy, not the quality of teaching.

 

  1. For example, imagine that University X did will in the TEF in Year 1 and was allowed to raise their fees. They do so with inflation for the next five years; the inflation rate is steady over this period at 3%. Their fee, which was £9,000 in Y0, would change to £10,433, in Y5. University Y was a new provider in Y0 and was still developing, so it did not break though in the TEF straight away, but over the course of five years had improved dramatically and was now considered to have some of the best teaching of any alternative provider. It is allowed to raise the cap on fees in Year 6, but in Y6 the inflation rate drops to just 0.5%. University Y’s fees thus stood at £9,045 in Y6, while University X’s fees were at £10,485. The difference in fees for Universities X and Y are not really representative of the difference in quality. The information it is providing to students is misleading and may ultimately lead to unfair advantages for certain institutions.

 

Cost of provision varies by discipline and institution

  1. We know that the tuition fee does not reflect the quality of a degree, but it also doesn’t reflect the actual cost of provision either. Fees are used in low cost subjects to generate surpluses to cross-subsidise other areas or to generate capital for investment in infrastructure.
     
  2. It is also safe to assume that “quality” is more costly to produce in some subjects than it is in others. For instance, the cost of procuring the best academics in a particular field may vary according to the supply of academics in the labour market. If there are only a handful of top professors in economics, the market rate for their labour would be high.
     
  3. Cost of provision also varies by institution. It is certainly feasible that one institution will be able to find more financially efficient means for providing a high quality course whilst another may have certain barriers to lowering cost that are difficult to remove, such as the size or specialist nature or the institution.
     
  4. All of this makes the link between quality and fee level all the more absurd. It also doesn’t help to improve the issue of cost transparency for students.

 

Allowing fees to rise across a whole institution does not reflect any difference in quality between disciplines and departments at that institution

  1. If institutions are allowed to raise all tuition fees as a result of some aggregate measure of teaching quality in the TEF, the incentives on institutions to ensure that quality is high across the board is reduced. This may allow poor performing departments to coast along without improvement.
     
  2. This will also create a potentially unfair situation for students, as they may pay be paying a higher fee for a course that is of lower quality just because the institution as a whole was deemed good in the TEF. TEF really only works if it is implemented at the department or subject level. Its efficacy is significantly decreased if institutions are allowed to aggregate their results.

 

No guarantee for the student that uncapping tuition fees means that the quality of those courses will increase

  1. The previous two issues lead us to question whether the increase in tuition fees would actually lead to improvements in quality for the students who pay higher fees. Increases in fees would hopefully increase the revenue for institutions to spend on courses, but there is no guarantee that this money will be spent directly on the course a student is on, and no guarantee that by spending the money the quality of the course would improve. 

 

 

By penalising poor performance in the TEF using financial measures, you make it harder for institutions to improve

  1. This approach risks creating even greater inequality within the higher education system. There could be many reasons why an institution might not yet meet the requirements for “excellence”. It may be a fairly new provider that is still developing its offer, developing its brand, and finding its position in the market. It may be an institution that suffered disproportionately as a result of changes in student demand, or because of cuts to public or private funding in a particular area. If such institutions fail to meet the criteria first time round, not only will their reputation be severely tarnished, but their financial position will also be affected. This will make it harder for them to catch up with and compete with other institutions.
     
  2. Institutions may well require additional funding to improve. They are much less likely to get the support that they need as a result of linking fees to the TEF. We could well end up unfairly stigmatising some institutions and leaving them in a financially insecure position. The end result could be a two tier system of institutions, or the creation of an underclass of “sink” institutions that find it almost impossible to find the funds to shake off a bad reputation, leading to their collapse.

 

Reputational risk remains one of the key drivers of institutional behaviour in higher education

  1. Considering the relatively small increase in tuition fees being offered to the sector, it is unlikely to provide any clear differentiation in fee levels for a considerable number of years. At the same time, the increase in revenue from an inflationary rise in tuition fees would be of secondary importance to the influence of the TEF as an influence on student demand. If an institution loses even just a handful of students to another institution as a result of its result in the TEF, the revenue loss could be considerably more than what an inflationary fee increase would generate.
     
  2. There is considerable evidence of institutions responding to reputational risk and changing practice as a result. We know that institutions respond to the publication of complaints statistics at individual institutions and often change practice to avoid bad publicity. We also know that the publication QAA Higher Education Reviews has often led to changes at an institution in order to avoid bad reviews, or to use a good review as a marketing tool to improve their reputation.

 

October 2015

7

 

[1] See: de Vries, R. (2014) Earning by Degrees: Differences in the career outcomes of UK graduates