Business, Innovation and Skills Committee
Oral evidence: Assessing quality in Higher Education HC 572-i
Tuesday 17 November 2015
Ordered by the House of Commons to be published on 17 November 2015.
Witnesses including written evidence where submitted:
At 9.15 am:
At 10.00 am:
Members present: Mr Iain Wright (Chair), Paul Blomfield, Richard Fuller, Amanda Milling, Amanda Solloway, Jo Stevens, Michelle Thomson, Craig Tracey, Chris White
Questions 1-63
Witnesses: Professor Simon Gaskell, Treasurer, Universities UK, and Alastair Sim, Director, Universities Scotland, gave evidence.
Q1 Chair: Good morning, gentlemen. Welcome to the Business, Innovation and Skills Select Committee. We are very grateful that you are coming to give evidence to us regarding quality assessment in higher education. For the purposes of the record, gentlemen, do you mind telling us who you are and which organisation you represent?
Professor Gaskell: I am Simon Gaskell. I am Principal and President of Queen Mary University of London. I am also the Treasurer of Universities UK and I led the task and finish group that produced the report on university regulation just recently. I am currently leading the follow‑on task and finish group looking at quality assurance. I am also chair of the board of the Higher Education Statistics Agency.
Alastair Sim: I am Alastair Sim. I am the Director of Universities Scotland, the representative organisation for Scottish university leaders.
Q2 Chair: Thank you very much. May I begin by asking: is the current system of quality assessment broken? Does it need fixing?
Professor Gaskell: I do not think it is obviously broken, no. One has to look at the current system from two perspectives, which are equally important: what it means to students who study within the UK, and what it means to students who study for British university degrees but outside the UK. There is a difference between the two. In the latter case, there is huge attention paid to the accreditation given by the quality assurance agencies. For example, in my own institution where we have 3,500 students studying for our degree in China, they are enormously reassured by the kitemark of quality that is provided by the Quality Assurance Agency.
The perspective is slightly different for UK‑based students. We do not advertise to our UK students that we are accredited by the Quality Assurance Agency; they take that for granted. In the UK context, it is very much a baseline of quality that provides an assurance. There are other mechanisms that enable students to decide which universities they wish to go to, based on the perceived quality of the offering they are given, but that is not provided by the Quality Assurance Agency. What the Quality Assurance Agency does, very helpfully, is provide a baseline to assure students, employers, Government and the general public that there is a baseline of quality achieved and other measures are taken to permit the judgment between different universities.
Q3 Chair: In terms of the regime for quality assurance and in respect of what the Minister wants to do with the teaching excellence framework, has he got the approach correct?
Professor Gaskell: There are many details of the approach that we are not clear about yet, and that has to preface any remarks that we make. The intention of the Minister, reflected in the Green Paper, is an intention that we would all agree with: the centrality of the student interest, the importance of widening participation, particularly for an institution like my own. They are absolutely critical.
Q4 Chair: Is he right to say there is patchy provision in teaching in higher education?
Professor Gaskell: It is an interesting word, “patchy”. Would it be sensible to say that every lecture given in every British university is of a uniformly high standard? No. It would not be realistic to say that, of course not. I am sure we can all think back to our own student experiences and remember particular lecturers who were inspiring, others who were less so and probably some who were really not very good at all. The term “patchy” implies a level of problem that does not exist, and so the level of teaching is, I believe, generally high.
That is not to argue for complacency. Indeed, if any institution in the UK were inclined to be complacent, then that would be a recipe for decline, because students these days, and appropriately so, are very demanding. They have very high expectations of all aspects of their teaching experience, not simply the quality of the lectures they get, but the feedback they get on their work, the promptness of that feedback, the extent of personalisation of their education. There are many mechanisms by which students make appropriate demands.
When I say that the system is not broken, that is not to say that it is perfect. It is not to say the quality assurance system is perfect. It has become, in some degrees, over‑burdensome for established providers. There is a lot there that we need to make sure we retain, because otherwise we risk the quality lessening.
Q5 Chair: That is very helpful. Could you set out to the Committee what you think we should retain in any new regime?
Professor Gaskell: If one assumes that it is appropriate to have a marketplace in higher education—we can argue about that, and I am sure around the table there are differing views—and one defines the market loosely in terms of an environment in which there is significant competition between the players, then we have to have a system supported by regulation and legislation that ensures that that market cannot fail. The principal way in which that market can fail is if the minimum standard is not achieved.
That is perhaps most problematic for some of the new providers and it is entirely appropriate that any regime should involve close scrutiny of new providers. Those providers with a long history of quality—and remember we are talking about some institutions that are hundreds of years old—should not reasonably be expected to undertake the same level of frequency of their scrutiny, not because the standards should be different, but because the requirement to assure those standards can be less burdensome in those cases. When we are talking about a minimum that is required, we are, I believe, talking about a minimum that ensures there is a baseline of quality. That is the first point.
The second point is that there are mechanisms in place, which can largely be self‑governing, with oversight, to ensure that standards are scrutinised repeatedly, internally, by the institutions concerned. It is the baseline of quality, according to national standards, and the existence of mechanisms to ensure that those standards are repeatedly scrutinised in turn.
Q6 Chair: Alastair, may I turn to you? Simon was very helpful in saying that there were two different audiences—domestic students and international students—and that the reputation of the UK higher education sector is well renowned throughout the world. Is there a risk that, in terms of different quality assurance regimes according to the different nations of the United Kingdom, you are going to have more and more confusion in the system? What is your sense about that?
Alastair Sim: There is a bit of a risk there. From our point of view, it is important to maintain a core of common UK standards, so that a potential student looking from anywhere around the world knows that there is an integrity to a UK degree. At the moment, there are UK‑wide structures for assuring that quality: a code, benchmarks, standards for subjects, a qualification framework—a whole mechanism that is managed at UK level by the Quality Assurance Agency.
At the moment, we feel, in Scotland, that we have the best of both worlds. We have those UK standards that give confidence to the international student that a UK degree is something of real deep integrity, and we also have the potential in Scotland to manage our way of looking at whether an institution is delivering high‑quality education in a slightly different way from England. We have a process called enhancement‑led institutional review, which is very student‑focused and is not simply about checking your quality but checking whether you are on an enhancement journey of continually improving your offer to the students and the quality of your education.
It is up to Ministers in England to decide what to do for England, but we want to ensure, as the process of thinking about the Green Paper goes forward, that, where there are aspects of UK infrastructure that are of benefit across the whole UK, those are maintained while maintaining our devolved capacity to operationalise those in different ways.
Q7 Chair: Could the system in Scotland be applied equally in England? Is there anything to stop that from happening?
Alastair Sim: I will let Simon comment on that. In England, in a sense, there is a difference. The Green Paper is talking about a link between the teaching excellence framework and fees. That introduces a different dynamic that simply is not present for undergraduate education in Scotland, so you can see why you would think about things differently. In essence, if anyone were designing the next evolution of a quality framework, they would want to think about how to put students at the centre of that. I am sure that will be part of the debate in England.
Q8 Chair: Do you think there should be a link between the teaching excellence framework and the ability to increase tuition fees?
Alastair Sim: I will not express a view on that for Scotland, because it is simply not an issue for us.
Q9 Chair: Simon, I think I know your views, but what is your view about that link?
Professor Gaskell: There is a certain logic to it. A starting point is to recognise that, considering tuition fees in isolation, they were introduced to replace a reduction in direct Government funding. It was not new money to universities. The value of that £9,000 fee has been eroded, so, by 2017—there are different estimates—it is worth between £8,000 and £8,200. There has been that decline anyway, so there has to be, at some stage, a mechanism for catching up, or this will simply be a continuing decline in the real value of those fees, which will, of course, reduce the ability of universities to invest in teaching facilities and in the teaching of students. There has to be a mechanism to increase the fees according to inflation.
I do not fundamentally object to the notion that there should be some sanction over poor teaching, which could be a financial sanction applied in that way. In other words, if there were evidence that minimal standards of teaching were not being increased, then I would have no problem in that being linked to a denial of the opportunity to increase fees.
Q10 Chair: Do you think the Government should go further? The Minister proposes that any increase in fees should be limited to inflation. Do you think that, if an institution gets a level 4 in the TEF, they are allowed to charge what they want in terms of tuition fees?
Professor Gaskell: That is part of a broader issue as to whether there should be a gradation. In other words, if it is a binary decision, which it looks as though it will be in the initial implementation, so that a minimal standard will be considered to be a threshold at which allowance will be given to raise fees with inflation, that is one thing. If you then get to the point of having a gradation, which may be a continuous gradation as in the REF—research excellence framework—or in line with some suggestions about having a division into different categories, that becomes problematic, not so much on principle but because of the extreme difficulty in assigning institutions to those different categories. It is that aspect of the mechanism that I am more concerned about than the principle linking the increase in fees to teaching quality.
Q11 Paul Blomfield: Simon, you said in your opening comments that you supported the Government’s focus on teaching quality, and I am sure that is true for all of us around this table. I guess what we are interested in exploring as a Committee is how we get this right. One of the strengths, I guess you might agree, of our university sector is its diversity. How easy do you think it will be to have a common set of metrics that enable quality to be measured across that range of institutions?
Professor Gaskell: That is exactly the right focus and, by saying we all agree on the underlying motivation, that is really the easy bit, isn’t it? These are not contentious points for the vast majority of people involved in higher education. The real question is behind your question: whether particular proposals would work in favour of those objectives or might perversely work against them.
To give an illustration from my own institution, if I may, Queen Mary University of London is a Russell Group university with a very atypical student profile for a Russell Group university. Two thirds of our students are from ethnic minorities. About 86% come from state schools and a high proportion from less advantaged social groups. One of the problems that they have, which we are working very hard to counter but it is a problem, is many of these students enter with very low levels of social capital. They are very bright. To get into our law school, you have to have the same A‑level requirements as to go to Cambridge. This is not a question of ability. But do those students who graduate from our law school move into the top City law firms? No, they do not. That is not because of ability. It is not because of the quality of the education they get from us. It is because of their relative deficit in social capital. Context is everything here.
One of the lessons we are learning from that, incidentally, is that we need to do more and are doing more to build up their social capital. This is not an issue that all Russell Group universities have. In the context of the TEF, that raises the very significant question of how you really judge the quality of the education on the basis of outcomes that are in part certainly associated with the quality of that education, but in part also associated with the background of the students, and their levels of social and financial privilege. We could get to a situation where we are penalising universities not because they provide a poor‑quality education, but because their mission, such as ours, is dedicated to providing that high‑quality education to relatively disadvantaged students.
This example is an illustration only. It is an illustration of how the apparently logical metrics might have an effect that would, for example, be counter to one of the stated aims that we all agree with, which is widening participation.
Q12 Paul Blomfield: It is interesting you raised that, because I was going to come back to you specifically around the suggested metrics and around the one on employment statistics. You have probably already answered my question, which is how far you think that metric would run counter to the Government’s admirable objective of doubling the participation rates of lower income groups.
Professor Gaskell: It all goes down to the sophistication of the metrics and whether, indeed, the metrics could be that sophisticated. For example, one could look at the employability of graduates. I have a problem with that term. Of course our graduates are employable. They are very smart and they are well educated. That is not the real question. The question is whether they are getting the jobs that their education and their talents deserve or, equivalently, are they making the societal contribution that they should be able to make on the basis of talent and education? That is very difficult to measure, and it certainly cannot be measured, for example, by looking at salaries on graduation or salaries two years after graduation. We will be crucified by a metric such as that, despite the fact that we are enormously proud of what we do for students who come from disadvantaged backgrounds.
Again, it is an illustration only but an illustration of the need, if they are indeed achievable, for very sophisticated metrics. The corollary of that is that using imperfect, naïve and imprecise metrics might have exactly the opposite effect to the one we are all aiming for.
Q13 Paul Blomfield: But, as I understand what the Green Paper is saying at the moment, there is no proposal to develop new metrics. It is using existing data. Are you worried by that?
Professor Gaskell: Yes. Existing metrics are useful in the context in which they are used at present, which is not to produce a ranking or a judgment on teaching quality. For example, we worry a lot and work very hard to increase or maximise our retention rate—in other words, to minimise the number of students who start their degrees with us and do not graduate. If that gets above a certain figure, we are very worried. Of course, if we were so inclined and if there were a funding system that gave us this incentive, the temptation, which I am sure we would resist, would be to say, “There is a very easy way of achieving 100% retention. You minimise your educational standards to the point where no one fails—job done.”
People have talked about an equally naïve statistic of the relative proportions of students who graduate with first class degrees. Again, in the current context, we look at that to make sure we are maintaining standards and, equally, to make sure we are being fair to our students in comparison with other universities. But, again, if that became a metric to judge quality, it would be very easy. Give everyone a first class degree—job done.
Q14 Paul Blomfield: Perhaps I can extend this out to both of you, Simon and Alastair. What sort of metrics would you be looking for as a proper measure of teaching quality? For example, would it be something around value added and how would you do that?
Alastair Sim: I honestly think you have to take an approach that is both metrics‑based and qualitative. You cannot have a conversation simply based on the metrics about whether teaching quality is really doing everything it should be. There are other elements you need to build in there. You need to build in the student perspective and you need to build in an element of peer review from other institutions to come and give an external view as to whether the teaching that is being provided is truly excellent. However sophisticated you are about metrics, they will give you questions to ask that will require further qualitative drill‑down, rather than giving you, in my view, a genuinely nuanced picture of whether the teaching at an institution is excellent and whether they are on a journey of continued enhancement of what they are able to offer students.
Professor Gaskell: If I could add to that, absolutely what we want to measure is value added, or learning gain as it is sometimes described. There is not any question that that is what we would like to measure. There is a real question over the sophistication of currently available metrics to do that. It is easy to talk about learning gain as being the parameter we should measure. It is easy to talk about context‑dependent metrics and adjusting metrics for context. To some extent, that is already done. For example, if you look at the Office for Fair Access, they set targets for participation of ethnic minority students in particular universities in the context of those universities. There is an element of a context‑dependent metric setting that is already achieved.
Again, this becomes much more critical if there are very significant financial consequences, either directly through an allocation, following the TEF, or indirectly. Supposing, in the context of my own institution, for reasons that were related to imperfection of metrics, King’s College London, with whom we are friendly rivals, were awarded a 4 and we were awarded a 3, or vice versa, that would have a huge effect. Would that be a real reflection of quality? Probably not. It may reflect quality in some discipline areas but not in others. It is the increased sophistication required in the new context that is critical.
To Alastair’s point about whether one can do everything with metrics, we are anxious, and appropriately so, to emphasise the differences between the research excellence framework and the teaching excellence framework. There are many differences. But one lesson we should learn from the REF is that, over the last 30 years, we have discovered that, although we can make an increasing use of increasingly sophisticated metrics, there is no evidence that that produces a satisfactory result on its own. Recent reports following the last REF, REF 2014, have indicated that some peer review moderation of measurements by metrics is still required. Although there are many differences between the TEF and the REF, that lesson will be transferrable from the REF to the TEF, related to the fact that, over 30 years, we still have a REF system that is adapting and gradually improving.
Q15 Paul Blomfield: Indeed, the Government’s Green Paper acknowledges the importance of narrative reporting, which is perhaps a development of thinking from the initial TEF. But I wonder if I could ask one further question. Given the complexity, there is some concern about the potential timescale. When the Secretary of State was before us as a Committee, we explored this issue. While the Government are keen to implement by 2017, he was also clear that that was dependent on working in partnership with the sector and said that, if there was sufficient anxiety within the sector that 2017 was a premature implementation date, then the Government would be minded to listen to that. What do you think about the prospects of achieving successful implementation by 2017?
Professor Gaskell: The first point one has to remember about any innovation, at least on the educational side and in higher education, is that you have to have a long lead time, because you have to tell prospective applicants, who begin at a particular time in a particular academic year, at the time they are applying. They have to know everything they need to know about a particular university at the time they are applying. That means that, for any implementation of the TEF in whatever form in 2017, those decisions have to be made very early in 2016, because it is in the middle of 2016 that we essentially start the recruitment round for admission in 2017. There is a long lead time; that is the first point.
The feasibility of introducing a TEF does depend on the level of sophistication or complexity. The suggestion that TEF 1, as I guess we should call it, would be based on individual institutions having a record of recent and therefore current success in a Quality Assurance Agency assessment is a crude measure, but one that certainly could be implemented essentially immediately. The irony of that is that it addresses precisely the point I was advocating at the beginning of the value of a quality assurance system to achieve and assure minimum standards. That is a necessary but not particularly sophisticated assessment of quality. If the first iteration of TEF were based on that, however, then implementation in 2017 is entirely feasible.
Something more sophisticated than that, employing more complex metrics, would certainly require additional work, and I do not see the feasibility of introducing anything more complex in 2017. 2018 at the earliest, I would say.
Q16 Chair: Can I just follow on from what Paul was asking you there? Would it be better in terms of having the system embedded if we had some sort of shadow TEF before proper implementation came in place?
Professor Gaskell: It is an interesting point, which I had not thought about. One would have to think that through, because of both the direct and the indirect effects of TEF. A shadow TEF would not be without consequence for universities, because of the effect on reputation and the influence on prospective students, particularly those coming from outside the UK. It is an interesting suggestion, but we should not think of a shadow TEF as a purely, if you like, academic exercise, because it would have an influence. Even if there were no direct financial consequences, there would implications for universities of that exercise. We should not fall into the trap of saying, “Let us have a shadow TEF. It does not matter if it is particularly good or not or whether it is sophisticated enough. Let us do it as an experiment”, because the experiment itself would have consequences.
Chair: Alastair, have you got a view on that?
Alastair Sim: Not really. That really is an issue for England.
Q17 Chair: Okay, thank you. I do not think anyone would disagree with the notion that we do not want to see students short‑changed and we want to see teaching excellence, but is the Government’s priority slightly out of focus? Rather than having some sort of teaching excellence framework, should we not be focusing more on learning? Is that not what happens in terms of the model of modern higher education?
Professor Gaskell: That is exactly right. That is essentially an illustration, again, of the need for sophistication in the metrics and measurements that we use. You are absolutely right. There is no merit to a university that provides wonderful teaching where the students do not learn. One would hope there is a certain correlation there, but it requires more.
Perhaps to be less frivolous, this directs attention to mechanisms of teaching and learning. It may have been appropriate when I was at university to have the primary means of learning sitting in lectures. That no longer is at least the sole method. At my own university, for example, essentially all lectures are recorded and they are available online to students afterwards, and that is an essential part of their learning. Assessment is often done online. Course materials are provided online. They do not have to go to the library physically to do that. They get access electronically. The mechanisms of teaching, and therefore the effectiveness of learning, should be correlated, but we are fundamentally looking at two different things, which I think is the point you are making.
Q18 Chair: Universities UK has said, with regards to the current system, that it has become “overly complex, process driven and disproportionately burdensome on many providers”. How do we stop the new quality assurance and the new TEF from doing the same? How do we avoid it being big, bureaucratic and bossy?
Professor Gaskell: It is a critical point. This is an issue that indeed was raised by Universities UK and others before any suggestion of the TEF or any review by HEFCE of the Quality Assurance Agency. The issue is not relating to the objectives of the exercise; it is relating to the burden that is placed on institutions to meet requirements. It goes back to the point that I was making earlier: that, if you have a new provider, and possibly a new provider whose motivation for being in the sector is quite different from the motivation of established institutions, then it is entirely appropriate that the scrutiny of that provider and the teaching and learning opportunities provided for their students should be very close indeed, because students will have a reasonable expectation that, when they go to a university in the UK, they can assume a minimum level of teaching and learning experience.
If you have an institution that has been through many cycles of quality assurance checking, has been providing a fine learning experience for its students for decades, if not hundreds of years, then it is perverse to expect those institutions to devote the same amount of time to periodic review as those that are newly in the market.
UUK has taken the position—and there is a broader issue here in relation to regulations more generally—that entry to the market should require the meeting of very significant hurdles and the establishment of a very clear commitment to the achievement of high standards in teaching and learning. That requires close scrutiny. It requires a frequency of scrutiny that is reasonably high. However, once there is a record within institutions—and we should not forget the amount of self‑scrutiny that takes place within institutions—and those processes are established, and demonstrably so, then it is reasonable for external scrutiny to be not less rigorous, but less frequent, because a greater frequency equates to an unnecessary burden. The more we spend on exercises such as that, the less we have to spend on teaching and learning facilities and services.
Q19 Chair: But is it possible to have a light touch risk‑based approach, given the huge diversity in the sector? If we are going to go down to individual course level, so I am deciding whether to do a particular subject at a particular institution and I want to assess the TEF in terms of helping me decide, and look at the wider learning and student experience, how on earth do you encapsulate that in an easy, light touch way? It is not possible, is it?
Professor Gaskell: No, I think it is possible. You have to remember that we have a long history, in the UK, of co‑regulation. For example, within any institution, there is internal scrutiny of academic standards and of the quality achieved, at course and indeed at module level—so individual modules of 10 lectures, or equivalent seminars, laboratory classes and so on—by the academic senate or by departments’ academic boards. Then the governing body has the obligation, re‑emphasised in the Committee of University Chairs’ recent code of practice, to assure itself that its internal mechanisms are in place to provide confidence in academic quality. They are answerable to HEFCE, at present, for that.
We should not imagine that there is a freedom of activity for rogue lecturers to present very low‑quality courses because they do not have another QAA assessment for five years or whatever. Nothing could be further from the truth. Indeed, part of the external scrutiny must be to provide assurance, seek assurance and gain assurance that those internal mechanisms are in place, robust and closely monitored. Co‑regulation is absolutely key here.
Chair: Alastair, do you want to comment upon a light touch risk‑based approach?
Alastair Sim: I want to use that to address a potential concern in Scotland, which is, while we are observers rather than participants now in the development of the TEF, because we are essentially content with the way we are pursuing quality assurance and enhancement in Scotland, there may be an unintended impact down the line, particularly if you are looking at international perceptions of higher education. If there is a TEF in England that has a gradation of four different bands of whether people are perceived to be delivering quality teaching and learning, and Scottish institutions are not part of that, there is going to come a time when Scottish institutions start to think, “Well, even though we are content with what we have developed so far—our enhancement‑led institutional review process—is there an international marketing reason why we potentially need to be looking at joining in a TEF that was really designed for a different purpose, partly as a mechanism for being able to adjust fees in England?”
That would involve costs. I am very conscious that HEFCE reckoned that the REF costs about a quarter of a billion pounds a year to run. I have a bit of a concern that, down the line, we may find ourselves in Scotland, for reasons not really linked to quality assurance, having to think about whether we join in a TEF that might have significant compliance costs.
Q20 Paul Blomfield: I just wanted to, if I could, Iain, follow up on the point you were making about diversity within institutions, because we were talking about across the sector earlier. I represent both Sheffield’s universities. They are both fine institutions. They are both committed to work on teaching quality, but they would both acknowledge that there is a range of quality across subjects within their institutions. Do you think it is reasonable to institutions or fair to students to simply give a kitemark or a grade to a university, given that huge diversity within the institution?
Professor Gaskell: I would make a couple of points there. In principle, of course, you can. The reason I can say that is because it is precisely what we do with our internal assessment, which is done on an annual basis of the quality of the teaching in different courses. We assess how good the teaching and learning experience is in different courses.
Q21 Paul Blomfield: The TEF will be branding you, as a university, uniformly.
Professor Gaskell: That is exactly right. That is a problem. What is done in the REF—and parallels are drawn here—is that the University of X will say, “We are ninth overall, but in medicine we are fifth and in politics we are 15th”, and so on. In principle, you could do that. Whether that is the right approach is another question.
The key difference between the REF and the potential TEF is the pace of potential change. I can illustrate that with reference, in fact, to my previous institution, the University of Manchester. This is 10 years ago. The school of dentistry, in the National Student Survey, ranked last in the country. They were very upset about this, as they should have been, and they decided they really needed to work on that. They put a great deal of work into that and, one year later, they were first in the country. If Manchester had been branded as bottom in the TEF for dentistry between exercises, they would carry that label for five years.
In research, things do not change so quickly, so the quality of research that is measured is a pretty good indication. Of course, things change over time, but not with that same frequency. The risk of labelling at the subject level and not acknowledging the changes that can be made rather quickly is a significant danger and an unintended consequence of a TEF.
Q22 Jo Stevens: We have heard about the proposed link between TEF and fee charging, being able to charge higher fees. Do you think that a consequence of this may be a shrinking of the number of universities available? How do we feel about that in relation to widening participation?
Professor Gaskell: There are all sorts of pressures on universities for financial reasons. We are working in a situation at the moment, in the established sector, where we have some very perverse financial incentives, and the risk is that the TEF would add to those. Again, if I may take my own university as an example, if we were driven purely by financial considerations, we would close our medical school; we would close our dental school; we would certainly cut back on engineering and science. We would probably close our drama school. We would not educate any PhD students, because you lose money on every one. We would have a very financially successful university that did economics, law and business and management.
That would be the market speaking, I suppose. It clearly would not be in the national interest and it certainly would not be in the interests of those hundreds or thousands of students from East London who find an opportunity to move into medicine, dentistry and so on through Queen Mary. We are living already in a situation with very perverse financial incentives. You are right to draw attention to the fact that the TEF could add to those, but it is an addition, not a qualitatively new perverse incentive.
Jo Stevens: Alastair, do you have a view on that?
Alastair Sim: It is not the TEF that is going to drive change in Scotland. We, as in England, are facing up to a very tough spending review, which will lead institutions to think very hard about what they can best provide, but that is not something driven by the quality regime in Scotland. That is really driven by trying to do your strategic best for students and for research in a difficult financial environment.
Q23 Chair: Can I push you, Simon, in particular? I was very interested in your dentistry in Manchester case study. Do you think a poor TEF score would mean you are going to see more courses close? With the perverse financial incentives that you were mentioning, will there be a greater specialisation, so that you will close down your medical school and concentrate on economics and law? Is that what the modern HE sector will do and will TEF push you towards that?
Professor Gaskell: I think it would; at least, there is a risk of that. Let us be clear, again, that we are talking about a TEF as a general principle. As we have been recognising, the devil really will be in the detail, and the key is to avoid these sorts of perverse incentives. It is certainly the case that, if there were, for example, a TEF ranking for my own institution that was lower than we wished and lower than expected, that would have an immediate consequence, I believe, particularly on recruitment of overseas students. That would have, therefore, a very serious financial consequence.
It would also have the consequence of reducing educational opportunities for UK students, because one of the points of the discussion—and I know we do not want to stray into overseas students too much, but it is a relevant point—is that, if, for example, we recruited fewer Chinese students to our engineering courses, particularly at postgraduate level, they would close. Therefore, we would deny UK students the opportunity to study those programmes, because they were uneconomic to run. So, yes, as you increase incrementally these financial pressures, the notion that that simply is an incentive to drive up quality is only part of the story.
Q24 Chair: But I am not so much interested in the incremental changes. I am interested in: “I run dentistry in Manchester and I have come bottom of the TEF with regards to that.” Surely the university will be thinking, “That is a stain for us for at least five years. The reputation has gone for half a decade. There is no incentive for us to invest, to improve. Therefore, let us close it down now.” Is it that we are not going to drive up standards as a result of this; actually, we are just going to see courses cut quickly?
Professor Gaskell: We could reduce diversity in the sector. My use of the word “incremental” was perhaps ambiguous. I meant that this is incremental in the context of a number of other financial pressures, so it is just one more, as it were. It is an extreme scenario, which I raised the possibility of, I guess, by introducing the example of dentistry in Manchester, but it is not an implausible one.
Q25 Paul Blomfield: Just on a related point, one thing that is positive about the diversity in your sector is that there are a whole range of different responses to improving teaching quality: the different initiatives, the different conclusions that different institutions reach. Do you think there is a risk that the TEF will supress some of that innovation by introducing a more uniform approach?
Professor Gaskell: I would say yes. In your own city, there is a very nice illustration of that. Sheffield and Sheffield Hallam are both very fine universities, absolutely at the top of their respective games. They do not recruit the same students, nor do they seek to, nor do they seek to emulate each other’s achievements because they recognise they are of complementary value. That diversity within the sector is part of the richness and part of the success of UK higher education, so any pressure towards uniformity would be detrimental, ultimately, to the students.
Students will decide, “Absolutely, Sheffield Hallam is the right university for me” or “Sheffield is the right university for me”, and that in part derives from differences in teaching styles and learning experiences, which are not quality, fundamentally, but which you would need to measure in different ways, because you are catering to different students with different backgrounds, different aspirations.
Q26 Michelle Thomson: This is a question for you, Alastair. Picking up on your point earlier on Scottish universities vis-à-vis potentially having to consider the use of the TEF, I wonder whether there is a residual issue in terms of Scottish universities marketing themselves and their brand through the existing channels. Might they have to do that, to make their voice heard on the international stage?
Alastair Sim: There is a genuine issue there that, as the TEF evolves, institutions will have to think about. Genuinely, I think the TEF has been conceived as what you might call an English devolved issue to deal with the English fees regime, but, as with so much in this Green Paper, a lot needs to be thought through about how it impacts on devolved jurisdictions.
From conversations with my members so far, I think our preference would be to continue to evolve what we have in Scotland, which is the maintenance of a UK‑wide standards framework, but an enhancement‑led approach to looking at how institutions are developing their teaching and learning. The instinctive position of members is not to be attracted towards a TEF that has been designed for purposes that do not particularly apply in Scotland. But we do need to keep a close eye on its development, because there will come a point where Scottish institutions will have to think, “How is this being perceived in India? How is this being perceived in China? If we do not have a top‑level kitemark from the TEF but we have a judgment from QAA in the Scottish system that we are effective, how is that playing in international markets?” It will be an issue.
Q27 Michelle Thomson: I suppose that is my point. Is the rationale for the issue the fact that Scottish universities, as part of a UK umbrella, may struggle to make their voice heard if it has to be squeezed into something that is ostensibly led in English‑based education? You say it does not quite fit. Is there an issue there for how Scottish universities market their brand, which is distinctive to that of English or sometimes UK universities? That is the question I am asking.
Alastair Sim: We benefit, as institutions, from being able to have two sets of strong branding. The UK higher education brand is really well recognised internationally. One of the other things from the Green Paper we are interested in is how liberalisation of who gets university title and degree awarding powers will interact with that international perception of the UK brand. That needs to be thought through quite carefully. Within that, we have a very strong Scottish brand sitting within the UK brand, and very strong institutional brands.
It may be that institutions reach a judgment that there is enough faith in the overall integrity of the UK and Scottish systems that we do not need to join in the TEF and get that top‑level kitemark to be credible in India, China, Nigeria or wherever; or it may be that, as experience unfolds, having that top‑level kitemark in the TEF becomes something international markets are really paying close attention to, and so we have to start thinking about how to retrofit TEF into a Scottish regime. I do not know the answer to that yet.
Q28 Michelle Thomson: Or potentially a stronger brand for Scottish universities per se might work.
Alastair Sim: I genuinely believe we are in a sweet spot at the moment, where the credibility of UK higher education is extremely strong and having a distinctive identity within that has really helped us.
Q29 Richard Fuller: Can I just press you on that, because there is probably a different point of view here? It sounds to me, from what you are saying, that any pressure to push a distinctively Scottish approach to higher education is doomed to fail internationally because the value that potential students oversees look to is in UK higher education. They do not look distinctly to the fact that a university is in Wales, England or Scotland; it is the UK.
Alastair Sim: It is a bit of both. Just to give some international experience, yes, the perception of the integrity of the UK brand is as important for Scottish universities as it is for universities anywhere in the UK. When we market abroad, we increasingly also use a degree of Scottish branding. For instance, if you are going to an international recruitment fair, there may be Scottish institutions in one corner of the room with Scottish common brand messages. They are all competing, but they are benefiting from footfall that comes and thinks, “Oh, Scotland—there is something a bit distinctive there. We know a little bit about that. We know it has a reputation for academic integrity. We know it has a reputation for being a country that is welcoming to international students.”
I genuinely believe both brands have a real value for us as we are competing. I would have to say that the recognition of the UK brand is higher. The Scottish brand fits within that, and the integrity of the UK brand is certainly important to our marketing.
Chair: Gentlemen, thank you very much for your time. We really appreciate your insight and experience and you conveying that to the Committee. Thank you again.
Examination of Witnesses
Witnesses: Katie Akerman, Director of Quality and Standards, University of Chichester, Professor Neil Ward, Pro Vice Chancellor (Academic), University of East Anglia, Professor Colin Riordan, Vice Chancellor, Cardiff University, Professor Graham Virgo, Pro Vice Chancellor (Education), Cambridge University, and Professor Belinda Tynan, Pro Vice Chancellor (Learning Innovation), Open University, gave evidence.
Q30 Chair: Colleagues, thank you very much. Thank you for attending the Select Committee. For the purposes of the record, could I ask you to introduce yourselves and tell us the institution that you are from?
Katie Akerman: I am Katie Akerman. I am the Director of Quality and Standards at the University of Chichester.
Professor Riordan: I am Colin Riordan, Vice Chancellor of Cardiff University.
Professor Tynan: I am Belinda Tynan, Pro Vice Chancellor of Learning Innovation at the Open University.
Professor Virgo: I am Graham Virgo, Pro Vice Chancellor for Education at the University of Cambridge.
Professor Ward: I am Neil Ward. I am Pro Vice Chancellor for Academic Affairs at the University of East Anglia.
Q31 Chair: Thank you for attending. I am going to put words in the Minister’s mouth, but I am not exaggerating too much. The reason there is a need for a new TEF is because the HE sector and universities have not prioritised teaching as something that the students need to appreciate. Is that a fair criticism? Do you not prioritise teaching as much as you should? Is research given higher priority?
Katie Akerman: I do not know that I would necessarily agree with that. Generalising across the sector, we tend to refer to research‑led institutions and teaching‑led institutions, which is perhaps something of a misnomer: we are all learning‑led institutions. I was struck by the point you were making earlier about a focus on learning.
Chichester is known as a teaching‑led institution, and we work very hard to ensure that the quality of teaching is appropriate, fit for purpose, benefits our students, supports our students, enables them to grow. Chichester has a particular strength, in that we punch above our weight in terms of taking students from lower socioeconomic backgrounds, students with disabilities, students from state schools, and we put a lot of mechanisms in place to support those students, because I am not sure that teaching quality is just that which happens in the classroom. It is about all of the things that happen outside of the classroom. It is about developing the curriculum so that it is appropriate. It is about the mechanisms we put in place to support our students—disability support, for example. So I am not sure that is necessarily an applicable comment. We work very hard to ensure that we have high quality teaching.
Professor Riordan: I do not agree with the premise either. From my experience, we expend a huge amount of effort focusing on teaching, learning, the quality of the student experience, making sure that, when students answer the National Student Survey, what we get back we respond to. We have a great relationship with our students’ union. I would say, certainly from my point of view as Vice Chancellor, I expend at least as much of my effort focusing on teaching and student experience as on research. If you look across the university, every academic and every member of the professional services will have different focuses, but I would say that, on the whole, there is at least a balance there.
Having said that, there is a caveat. It is quite clear that there is a culture in research‑intensive universities, and perhaps in all universities, that does value research more highly than teaching, but it is a cultural matter. Many people come into the profession mainly because of the research, so you are slightly swimming against the tide in terms of the cultural value or the validation that is applied to research, as compared with teaching and learning. But teaching and learning has a huge number of champions, and increasingly influential champions, among the academic staff. I would not say that that culture is to be found among the professional services and support staff in the same way. There, the culture is much more that they are both important. There is a reality that research is valued more highly among academics in research‑intensive universities, but I would not say that the institution therefore follows that route, certainly in our case, and I think I can speak for a number of universities on that.
Professor Tynan: I would echo what my colleagues are saying here and probably add that the diversity of missions will reflect the kind of emphasis an organisation or university would hold in terms of the learner experience and the research frameworks that are within those organisations. At a place like the Open University, which is well known for its part‑time cohort and very diverse demographic, the learning experience for us has been reflected very kindly, in fact, in the NSS. For the last 10 years, we have been above 90%. Our emphasis on that learning experience and the teaching and learning process is incredibly important for the demographic we are bringing through the system.
I will not give a direct answer that there is more emphasis or less emphasis, because I believe it relates to the kind of diversity that we want within the sector about what universities are providing, for not only the economy but the social fabric of the country.
Professor Virgo: I do not agree with the premise either, although it is fair to say, coming from a research‑intensive university, the profile of teaching is probably not as high as the profile of research. That may be in part because of the focus through REF and the desire of academics to make their mark in research, but the vast majority of them are fully engaged in teaching as well. There is a very high profile for teaching within the institution, but, in terms of the external awareness of what is going on, perhaps that is not as obvious as research.
Professor Ward: There is no doubt that there is increasing emphasis on managing and improving the quality of teaching in universities, as a result of £9,000 fees and liberalised recruitment, where we are competing for students with no caps. However, with successive research assessment exercises, there is no doubt that there is a much more sophisticated repertoire around measuring and monitoring research quality. At the level of the individual academic, but also sometimes academic departments, it is easier to see the emphasis on research or research quality. On the teaching side, the metrics are much newer and it is a much more recent development. It is research assessment exercises for over 30 years, which are of increasing sophistication, that has led to quite a lot of effort on managing and supporting research quality.
Q32 Chair: Given that I am hearing there is not really an issue, is TEF necessary?
Professor Riordan: I would not say it is necessary. I would like to see more evidence of the problem to which TEF is going to be the solution. However, just because it is not absolutely necessary, it does not mean to say we should not do it, if you see what I mean. I can see the point of it; I can see the idea behind it. We have a research excellence framework, and perhaps that is what gives research the profile that it has. If we had a teaching excellence framework, perhaps, as my colleague Graham said, externally there would be more recognition of what we would regard as the undoubted profile and quality of teaching and learning internally.
If it can have that effect, that would be great. The question is, practically, whether it can be done, because we have been around this buoy before and, so far, it has been pretty difficult to come up with ways of assessing teaching. There is a very simple reason for that. In terms of research, the whole scientific method of peer review is just how it works. You automatically have a way of assessing quality and excellence, because, basically, somebody puts something out there; everybody else has a look and says, “Is this reproducible? Do we think this is good quality or not?” You come to a collective view on it and, in certain cases, you can almost prove something to be the case. It is just a very different process with teaching. How you, through the expert panels that we hear about in the Green Paper, adapt that process to teaching, which I think is the idea, is as yet not clear.
Professor Virgo: I agree with that. I would just add, whether or not TEF is necessary, we do not know yet what TEF is. We are lacking an awful lot of detail. If it could be an acceptable measure of teaching excellence, we can argue about whether it is necessary but it would undoubtedly be very useful in raising the profile of teaching and helping us to say, “This is proof of our teaching excellence”, rather than us just simply saying, “We know our teaching is excellent.” The question will be in the detail as to whether TEF can actually achieve that, which is as yet unclear.
Q33 Chair: I want to push you on that, with regards to Colin’s point about practicality. You were deliberately chosen, if I may, to come and give us evidence because you are incredibly diverse. You reflect that huge diversity in the higher education sector. If I am considering doing a particular subject, English, at Cambridge, Open University, Cardiff, or wherever, how on earth will I be able to compare and contrast? Is it possible that, given that huge diversity in the HE sector and in your institutions, I will be able to have single rating to say I should go to this particular university? How can you capture that diversity without having the risk of just becoming a homogenous lump?
Katie Akerman: The only way you could manage that at the moment would be through the league tables, because the league tables will be subject‑focused, but league tables are far from perfect. One of the things the Quality Assurance Agency has always sought to do is make sure that its judgments on the reviews it undertakes are not something that can easily be translated into a league table, because it does not want to contribute to the development of those.
Our concern with the teaching excellence framework is that it is potentially reductionist, in that it is reducing the student experience of university to their experience of teaching only. If you look at something like the KIS data for traditional subjects, like history for example, a student may spend only 20% or 25% of their time being taught in the classroom. The rest of their time is spent researching, in the library, hanging out with their friends and everything that a university experience is about. It would be a sad loss to the sector and to students were their experience to be reduced to that of their teaching.
Professor Tynan: There are potentially some other challenges I can see that sit around learning gain, which is also quite disputed, even in our field. What we are looking for is what distance a learner travels in their journey: whatever university they are going to, what do they come in with, where are they going out to and what with, and what have they gained in that period of time? That is also another measure.
For me, it is probably difficult to say, “Let us compare directly English here and English there.” What you are really looking at is the performance or the learning outcomes that the students have achieved. That is a real challenge for us. I know that there are some projects going on at the moment to look at that, to see how we can measure learning gain. If we could crack that one, we could have something quite interesting that would frame or support a TEF.
Professor Ward: On learning gain, there is not a strong consensus, but it is something that is very common in education, before you get to higher education, looking at the value added and looking at how students progress through different stages of their education. It is not particularly well developed in a higher education context, but I would anticipate, in two or three years’ time, with the work that is going on, led by HEFCE, we might well have something a bit more robust and consensual than we have now.
Professor Riordan: Just on how you measure this and whether you can measure it, one of the ways is metrics. We know that that really does not work for researchers. That is why, so far, we have always stuck with peer review for REF, or RAE as was. Research metrics are just not very reliable, whether it is citations, field ratings, citation indices, that sort of thing. I think we will have similar problems with teaching and learning. The question then is: how do you reproduce something like peer review? That is the question we just do not know the answer to. I do not know yet how you do peer review, or something equivalent to peer review, for teaching.
Professor Virgo: Specifically on your question, it is really important for us to emphasise that we are a very diverse sector, and so, when students are making their choices about where they wish to study, there are all sorts of factors that will be taken into account. If it is just boiled down to one particular issue through TEF, that will lose the subtleties of the diversity of the sector.
Q34 Paul Blomfield: Following on from that point of diversity, at the moment it has been boiled down to three crude issues, has it not? Employment, retention and NSS are the metrics that are in the Green Paper. How adequate do you think those are to measure teaching excellence across such a wide‑ranging sector?
Professor Tynan: I might speak first on this one, if you do not mind. I might speak to the issue of progression and retaining students. One of the complexities in this particular area is that not all organisations are graduating students on degrees. For the Open University, for example, more than 38% of our students are graduating with sub‑degrees, and so their aspirations and also employers’ aspirations sit with short courses, diplomas, certificates, less than that degree. It goes back to that point about learning gain. If we are thinking about what we are measuring here, the types of students that are coming through these diverse missions are exiting our universities with different kinds of qualifications that meet their aspirations and the employer’s aspirations too. A single measure that is just using current data around progression and retention may not serve the missions of universities that are serving the most disadvantaged students.
Professor Virgo: Of those three metrics, from the University of Cambridge perspective, we would be content that retention we do very well in; we have statistics that we have good employability with good salaries; and we do well in NSS, but that is not a good reason to adopt those metrics across the sectors, because what will work in one particular institution and may be some evidence of teaching excellence will be inappropriate and unfair in other institutions. With some of them, particularly employability, there is no real sign that that is a metric of teaching excellence.
Professor Riordan: I do not think that a good NSS result necessarily equates to teaching excellence. It equates to students who are very happy with what they are being taught. It does not tell you anything about the standards, in other words how good their achievement is in terms of academic progress. Similarly, retention, obviously, is great for Cambridge and it is fine for Cardiff. It might be more difficult for universities that take a far more diverse range of students for whom it is more difficult to finish their studies. It might still not tell you whether the teaching is excellent or not. There are lots of reasons why people have to give up their studies and do not progress. It may not relate to the teaching.
You could make a similar argument on employment, that, just because you have high employability, it might be the mix of subjects, that you have a lot of vocationally orientated subjects. It will favour those that have law, medicine, dentistry, I would imagine, over those that do not.
Professor Ward: I am particularly concerned about the Green Paper mentioning the ambition to move to earnings. That could have all sorts of difficult unanticipated consequences around the rise and fall of different academic disciplines, on the basis of how much people earn when they graduate in that discipline.
Katie Akerman: The point about diversity in the sector is an interesting one. We have a number of collaborative partners we work with, including a number of further education colleges, and the number of students they take is too small for them to be reported in the National Student Survey. Whatever data sets we might use if we go ahead with the idea of a TEF, it would have to be something that everybody can take advantage of rather than a select number of institutions simply because they have the correct number of students. There are also issues, for example, around nesting of data, so, where a university might be running a combined programme like English and history, it is then which data set that particular programme is absorbed into. The metrics that are being proposed are far from perfect.
Q35 Paul Blomfield: Can I come back particularly on the employment metric? Graham, you made a very strong statement on a metric that your university would do very well out of. The Prime Minister has outlined an admirable objective of ambitious targets for improving widening participation in the sector. How do you think that metric would assist that objective in terms of widening participation? One of the things we are interested in exploring is perverse and unintended consequences of measurement, so how do you think universities might react, to up their employability scores?
Professor Virgo: If the issue is encouraging people to come to university and get the benefits of study at university to improve their life chances, in terms of better chance of getting graduate jobs, it will be possible to use employability to assist in widening participation discussions. But it is not a test of teaching excellence. It is a very different metric.
Q36 Paul Blomfield: You are all agreeing in that; I see your heads nodding. Can I come to the issue, then, of learning gain, which a number of you have mentioned and which I guess, to us from the outside, would seem an obvious way of measuring teaching excellence? Do you think any of the metrics that are there at the moment, those three we talked about, effectively measure learning gain? If not, what new metrics do you think we might look towards?
Professor Tynan: From my perspective, I do not think they are quite there yet. It is a very difficult area to understand and to measure, and there is nothing easy that you can pick off the shelf. What you are doing is assessing, basically, what somebody comes into the organisation with and what they end up leaving with. If someone comes in with three A‑levels and leaves with a first class honours degree, what kind of learning gain is that compared to someone who comes in with one A‑level and leaves with a first class degree? What are we measuring? What do we see as being important here?
The work HEFCE is funding currently across the sector has a range of projects that are looking at both statistical and qualitative modelling, which will help us ascertain what kind of measure we could pick up and look at. There are probably lessons to be learned from what is happening in the school sector in this space and how that is measured there. That could also hold some clues for us.
Professor Riordan: We used to call this added value, so you compare the UCAS tariff or similar when they arrived and then what they left with. The danger with that and the league tables is grade inflation, which is mentioned in the TEF. We have to do something about that. The problem is that, if you do start measuring that sort of thing, then people start gaming it. We have not got the learning gain programme in Wales. It is a HEFCE thing, so I am not personally involved with it. The immediate issue I think we will encounter is, if there is some sort of test at the start and a test at the end that are not directly to do with their studies, to avoid the grade inflation problem, universities would potentially start teaching to those tests or at least teaching to the second test. That is the problem. It is how you come up with a metric you cannot then game, in a sense.
Professor Ward: We have one of the HEFCE projects. I think there are 12 being funded in different universities or groups of universities. We are experimenting with three different techniques. At the most basic level, it is the relationship between the qualifications and the level of skills that a student has coming in to the degree programme and then what they have at the end. Some of that can be done by looking at the grade point averages, or the scores as they pass through the different stages of study, and the educational qualifications on entry. That is the most basic level. In some disciplines, there are quite well developed tests of students’ grasps of concepts, and so they can do a test at the start and a test at the end. It is going to take some time for that work to be pushed through and for the sector to come to a settled view, because it is very new stuff.
Professor Virgo: If learning gain is simply an issue of distance travelled, then, from a university where we have a very high entry level, it is going to be more difficult to prove distance travelled in comparison with other institutions. Therefore, we would be happier with retention and employability. That would fit what we do best in.
Professor Tynan: It speaks to the diversity of the sector in what the different missions of organisations are and what they are trying to achieve with their learners.
Q37 Paul Blomfield: You are painting a picture of complexity, and there is the work going in HEFCE and a need for thought to get this right. When we had the Secretary of State in front of the Committee, he was very clear that he wanted to develop this in partnership with the sector. There was a desire to have TEF implemented for 2017, but he acknowledged that, if the sector were uncomfortable that it could be achieved effectively within that timeframe, then there was some flex there. Do you think that we can effectively introduce TEF as you would want it to be, as an effective tool in measuring teaching excellence, by 2017?
Professor Riordan: I am slightly out of it because it is Wales, although we do want to be involved, because we cannot have something happening in England that we have nothing to do with. That is very, very difficult. If you think back to when REF was started—was that 1984 or 1982?—it took years before REF became reasonably defensible. It was RAE. The early iterations of RAE would be difficult to defend now, I think. I would imagine that you could have something in place but certainly not, as you put it, as we would like to see it. You could make a start, but I think you would be looking at iterations over maybe a decade or something—it could be as long as that—before you really worked out how to do this and whether it could be done in a sensible way, particularly as we do not see a clear path forward at the moment, because of some of the issues that we have already outlined.
Professor Virgo: We would certainly want to see significant pilots for all of those reasons before TEF proper could be rolled out because, as I said earlier on, we just do not have the details yet to really assess what is on the table.
Q38 Chair: Can I just push you on that, because I think that is very important? Do you think it is impractical to have a target implementation date of April 2017? Is that not achievable?
Professor Virgo: TEF with a rigorous framework I do not think is achievable. The Green Paper identifies the most basic framework, which is QAA‑related, and that is clearly achievable, because that is a very easy threshold to cross.
Professor Tynan: In pushing forward too quickly, I think there will be too many unintended consequences that could occur. We have not even touched on here any issues around brand for the UK and its nations, and the unintended consequences to that, the unintended consequences potentially for widening participation of students, diversity of missions. There is a lot to think through to get that right.
Professor Ward: It is interesting to compare with the introduction of impact and impact case studies in the REF. That was just a change to one part of the REF, and there was a full pilot across different disciplines and across a range of institutions, a very thorough pilot, to test that out. Here, we are looking at something that is going to gauge teaching excellence, which is quite nebulous, very, very complex. It needs a level of piloting at least comparable to the impact agenda within the REF, if not something far more extensive.
Professor Tynan: We have seen that the OECD has just said AHELO, which was measuring employability‑type measures, is not going to be pursued. There has been huge investment in that. It is deemed to be unsuitable, so the OECD itself is now taking a step back from that and trying to develop something extra too. There are probably lessons for us to learn from that.
Professor Riordan: It is an excellent example from Neil there, and that took about five years, I think, to go from “let’s pilot”—or maybe even longer; we can easily check—to it actually happening in 2014. That was a long process for one bit, 20%.
Q39 Paul Blomfield: What other lessons do you think we might learn from the REF? I know it has taken time to get it right. I think people feel that it is now broadly successful. One you have touched on, which is more emphasis on narrative reporting. Would you agree with that? What other lessons might there be from the REF?
Professor Riordan: That it takes a long time; that you really have to think it through; that you have to build these things up slowly; that you do need to have pilots and you have to monitor and evaluate. Thinking of it in those terms, if somebody said, “Let us invent REF now and it will be in place for April 2017”, can that be done? No, very, very straightforwardly. If there are any parallels, that is one lesson we can learn: that it does take time and it does need thinking through.
Professor Virgo: I do not think we should regard REF as a perfect model. The Green Paper identifies the significant amount of time, the bureaucracy and the expense involved in REF. One of my concerns is, if we visited that upon TEF and had that level of bureaucracy and expense, it would undermine an awful lot of what it is being sought to achieve.
Professor Tynan: In contrast, I would say, by doing nothing, we risk not highlighting very excellent teaching practices here in the UK in particular. As you were saying when you opened up, the focus on learning and teaching has not been as much in some institutions as in others, and why would we not want to be looking at benchmarking, seeing where that sits on a world stage and understanding how we can improve that?
Q40 Amanda Solloway: Belinda, it was a couple of your comments that made me think about this. Looking at teaching and then looking at learning, you clearly indicated that learning was a great success within the Open University. I wonder if that is dictated by attitude of people who apply, as an example. I wonder if there is some difference.
Professor Tynan: In the distance learning field, we quite often say that the students are very motivated to come and study with us, to take on seven years’ worth of study part time. If you are working and you have children, to undertake that, you have to come highly motivated. It is our job then to keep them, to keep them motivated so that they can complete. That is part of the demographic that we attract, but I think people are always in a bit of shock. They first come to us and they think, “I am motivated; I am ready to go”, but they do not really understand fully the commitment that is required.
Q41 Chair: What would your retention rate be? The model of distance learning could be problematic. You are not going to see your tutor face to face, are you?
Professor Tynan: It depends what the students are enrolled in and where their exit points are, because it is a staircase. Students come to us in entry qualifications and they might go into foundation qualifications, and so it is difficult for me to say, “In this part it is this, and in this part it is that”. 38% of our students, as I said before, are in that sub‑degree level and are doing extremely well. Completion rates on a modular basis rather than just on a degree basis are far easier for us to give you accountability for. Does that make sense?
Professor Riordan: One issue with retention rates is that, as learning becomes more flexible these days and into the 21st century, you are going to get people who will want to do 30 credits or 60 credits, then park that and go and work for a couple of years, and then build a degree up. It could give you a really wrong impression. If they go to Cambridge, they are pretty well likely to do a three‑year degree but, if they go to other universities, it could look very different. Retention might not give you an idea of the quality of what is happening there in terms of learning and teaching.
Professor Tynan: It really rests on the aspiration of the student and quite often the employer.
Q42 Amanda Solloway: That is quite a key point. I wondered again, and probably to everybody, whether there is something more that we should be addressing prior to either further or higher education in terms of equipping students to learn more. Maybe, Katie, you could think about that first.
Katie Akerman: That is a really interesting question and one that is quite difficult to answer. It comes back to the point about the diversity of the sector and the diversity of the students we take in. For example, changes to disabled students allowance are going to quite badly impact upon institutions that have a high proportion of disabled students and how we are able to support those.
Similarly, there is a question in my mind about the linking of fees to institutions that are high on the teaching excellence framework. The Green Paper is talking about social mobility, and obviously there are targets around increasing student recruitment from disadvantaged backgrounds and so on. A student coming from a disadvantaged background might be put off by an institution that is charging £9,000, plus whatever inflation is, versus another provider, a private provider or an FE college, that is able to charge considerably lower fees.
What we might end up with is that the very institutions that need help with their teaching excellence are not the ones that are able to access the additional funding. The students that we are trying to encourage might end up going to the potentially weaker institutions, in terms of a TEF, than we might otherwise want or want to promote.
Professor Ward: This issue of how students learn how to learn is a really interesting one. I see it from looking at my children, who are 15 and 12. Learning how to learn is a very prominent part of their education. Historically, universities have perhaps not engaged with the way that learning and teaching is changing in primary schools and secondary schools. I would say that an element of teaching excellence for an institution is the extent to which it does engage upstream. There is no mention of that sort of thing in the Green Paper.
Professor Virgo: I agree. Of course it is incredibly important to be aware of what is going on in schools, but universities do have a responsibility to really engage with the learning experience. At my institution, we spend a lot of time investigating the transition from school to university, to make sure that, when the students arrive, they have the learning skills to be able to cope with the independent study and the demands of a university education. If we feel they are lacking, then we spend a significant amount of time assisting them at the start to make sure they know what is expected of them and how to learn.
Q43 Amanda Solloway: Just going back to Katie’s point, do you notice that there are certain different entry levels in terms of learning from different types of social backgrounds?
Professor Virgo: To some extent, but, in my experience at my institution, almost regardless of school background, there is a need to assist with learning at the university level.
Q44 Chris White: To start with, the Chairman might have led the witness a little bit in his previous question about whether we could achieve this by 2017. Graham, you said no, this would not be possible, but I think we would be disadvantaged perhaps by missing yet another cohort out, because that would be another three years.
I suppose my question really is about going back to the past and thinking that there would have been a Select Committee talking about Ofsted. Bearing in mind some of your reservations about TEF, Katie, and how you manage to encompass all the different schools, the diversity and all the different things an HE organisation could provide, we have found that Ofsted, however irritating it can be, improves standards. It recognises quality of teaching.
I suppose, whatever we do here, whatever legislation is passed, you will still see it in The Times every year. I am sure you all grab the copy of the paper to see where you feature on that particular list, which does include some of the issues you talk about. My question is: do you think that, maybe in 2020, we will be sitting here again with the same panel saying, “Yes, it was a difficult thing to put in place, but it has recognised the quality of teaching. It has improved standards where standards needed to be improved”? Do you not also think that it will be improving transparency and accountability for young people who are looking for a different, alternative provision for their higher education? Sorry, that is a bit of a ramble, but does anybody want to respond to that?
Katie Akerman: This might be where I get to mention Zen and the Art of Motorcycle Maintenance. Pirsig is talking about the metaphysics of quality, and, to boil it down to a summary, what he is saying is that quality is in the eye of the beholder. That means, for universities, it is around transparency; it is around managing expectations. If I promise you Waitrose and deliver Lidl, you are not going to be particularly impressed. On the other hand, if I say what we are giving you is Tesco and then you arrive and it is more like Waitrose or M&S, you are going to be perfectly happy with your experience.
Chair: Other supermarkets are available.
Q45 Chris White: Can I come straight back to you on that point? If you are judging a supermarket by its service, its cleanliness, the quality of its staff, how quickly you get through or the cost of its food, it does not matter if you go to Lidl or wherever. There are measures by which you could directly compare Aldi and wherever, without depressing either one, or in fact making a customer think it was worth more or less. That is another example, perhaps.
Katie Akerman: It is very much around managing expectations and transparency. The Competition and Markets Authority guidance is encouraging the sector further. I think we are doing a good job, but that is moving us forwards. There is also the Private Member’s Bill going through and we will see where that gets to. Again, that will be something that moves us forward, if it does go through completely.
Because it is about managing expectation, while universities and other institutions can do as much as is humanly possible in terms of the information we put out on websites, what we say at open days and so forth, what we cannot manage is media expectations, what career advisers tell prospective students, what parents think about universities. If people have got what might be classed as a traditional idea of a university, perhaps somewhere like Cambridge for example, they will be disappointed if they come to somewhere like Chichester. This why things like open days are absolutely crucial, because students know what it is that they are signing up for.
Professor Virgo: Can I pick you up on the 2017 and whether we were led or not, and also, in 2020, where we will be looking back at it? That starts to raise the really big issues as to exactly what the point of TEF is, and I think there are two. One is reputational; the other is fee level. Certainly, TEF could be introduced in 2017 with most institutions being level 1 on the basis of already published quality assurance decisions. In reputational terms, we are already there; we can say to our students we have been quality assured. Therefore, the only benefit is going to be one of raising fees, and, if that is the assessment of TEF, then that raises all sorts of other issues as to whether that is really what is behind it.
Your question was premised on providing information for students. In 2017, there will not be any additional information provided to students about teaching excellence or the quality of the learning experience. As to whether that will be available in 2020, again, we go back to where we started. We do not really know what the metrics are going to be and whether they are going to be sufficient to be able to really say, “Yes, we accept that is a sign of teaching excellence.”
Professor Riordan: You mentioned Ofsted. Schools are in a very, very different position. At least in England, there is a national curriculum, which schools follow. You can make comparisons. There is a defined and relatively small number of subjects that are being taught. Earlier on, we talked about comparability and whether English at Cardiff can compare with English at Chichester, if it is taught, or English at Cambridge. It might be very different but you might nevertheless be able to compare. What about a subject that is multidisciplinary or interdisciplinary: digital journalism or biomechanics? There are all sorts of things you cannot really compare because we do not just fall down into a nice defined set of history, geography, English, maths, physics, chemistry. It just does not work like that anymore.
That is without even taking into account what we have already discussed, the very different missions of different universities and different expectations. You come to a university like mine for a particular reason, which might not at all be the same as the university down the road. You might prefer that university because it is doing something quite different from what Cardiff is doing. That is what makes it extremely difficult to say, “Let us have an inspection body that is going to go and inspect and check that standards, i.e. the academic achievement, the quality, the management of the learning and teaching, are comparable from one university or even one discipline area to another.” It is extremely difficult at university level.
Q46 Richard Fuller: Let me tell you some of the impressions I have got from what you have said so far: “Getting it right is going to take a long time”; “Evaluation is so nebulous”; “They tried these statistics on employability but they did not work”; and “You have to remember”—this is what we always hear—“the diversity of the sector”. I am going to ask Professor Tynan, because she is the only one who is a bit more gung‑ho about it, why is everyone else so grumpy?
Professor Tynan: That will be my colonial past. We have mentioned some things here today around diversity of mission and what universities are trying to do, and no university is trying to give a bad learning experience. Everyone is trying really hard to give students the best possible learning experience, the best possible outcomes, so they can get jobs. There is no one actively not doing that. I think how we organise ourselves will limit where we put our emphasis and what we see as being important in that journey. That is where protecting that diversity of mission is incredibly important. You get some universities that are really into employability, and their whole curriculum is focused on that and it is driven through the curriculum. In other organisations, the focus could be completely different. It could be on learning how to learn, producing critical thinkers or problem‑solvers.
When you ask us as a sector to say, “We all want to be measured on the same measure”, we want to throw it back to you and say, “We have really relished our diversity.” You have encouraged us to be diverse and to have this market where we can all play differently, and now you want to measure us on the same thing. If you measure us all on the same thing, what could that create for us? It will drive behaviours for sure, because we will all then try to get these measures the best we possibly can, so we do not miss out on being seen as excellent. It could also force a mediocracy in the sector, where you have something that is actually very vibrant.
Q47 Richard Fuller: I really do not find that that compelling, if I may say so. If what was being proposed was the measure, it was a straitjacketed measure and that was the way in which people made their assessment, then I would absolutely agree with you, but this is a measure. It is something about teaching. There are many other things that people will look at: do they like the campus, is it near to home, are there the sports facilities that they want? That decision about diversity remains with the individual. Your responsibility is to provide that piece of information. You are among the brightest minds; why have you not done this already?
Professor Tynan: Do you not think we are already doing that? If I was now to take a more defensive response to that, I would say that I think we are doing that. When you look at the national fellows, you see the excellence in teaching promoted by our excellent teachers. When you look in the Times Higher Ed every week, you see excellent examples of what our teachers are doing. You look at the international brand and you see that international students want to come to this country and to study here. We are already displaying areas where we are excellent, and we have great examples of that. On a daily basis, you could go to any organisation and say, “Tell me what you are doing that is innovative here. What has improved the students’ learning experience here? What are you doing this year?” The QAA has also helped us do that, so there is a whole range of mechanisms that already exist within our organisations.
Q48 Richard Fuller: But a whole range of mechanisms does not mean, “I have got five minutes; let me just check something. Where is the authority?” As Mr White was saying, for schools—his point about the comparison with Ofsted is very useful here—parents can now quickly go and see this. What you are suggesting back, Professor Tynan, is that the information is there. Yes, it is, but it is disaggregated information. It is not on an easy, comparable basis. Would it be fair to say that set of information in that format does not currently exist?
Professor Tynan: I do not think that exists here in the UK in the way that is being described to us and as is being articulated in the Green Paper, which seems to be desirable. I do not think anyone here is saying we would not want to open our doors or be transparent. I should be careful not to speak on behalf of my colleagues, but certainly at the Open University we have a very open door. We are an open university. We are very transparent. Anyone can come in and look and see what we are doing. Trying to get that into a format that others can readily understand is, I think, part of the challenge of TEF. I feel quite positive about the TEF and what it could achieve for us, but taking the time to understand how we are going to display that in a way that others can understand it readily is part of the challenge that we have ahead of us.
Q49 Richard Fuller: Can I just one other broader question, for clarification? What other international examples can we draw on to help guide us? I have been going through the document trying to find some. I was just wondering if you might have some better insights. It is back to you, Professor Tynan.
Professor Tynan: I have been out of Australia now for nearly three years, so the higher education climate there is quite different, but I have seen various iterations of things that are not dissimilar here. Quite often, our countries compare and adopt and then de‑adopt, etc, so there is a lot that we can learn from each other. The way Australia has approached this is through a quality agency approach, so the Tertiary Education Quality and Standards Agency. It has moved away from a process‑oriented quality assessment. It is more around outcomes as well. Some of the published things you are talking about here form a framework which then is agreed with institutions, in terms of a compact or an agreement about what an organisation wants to be measured on.
You could look to Canada. Canada does something very similar where they agree with their organisations, “What are you prepared to make public and be measured on?” alongside a set of standard pieces of data, like the HESA data we collect here, for example. Sweden has very good examples of this as well. You could go to almost any country in the world, do some kind of benchmarking and say “How are they approaching this and what mechanisms go with it?”
What is missing here from what we are hearing in this paper for me has been around the incentivisation. Currently, the incentivisation is sitting very much around raising fees, but what if your organisation did not want to or did not see financial gain as being the reward for making quality learning and teaching experiences transparent and open? There is something about getting down to the academic and teaching staff themselves, saying, “How do we incentivise those people to be even better than they currently are?” There are many examples you could draw upon. It is a matter of sitting down and saying, “Let us look to the countries whose education systems are performing in the ways we would like to perform. How are they measuring teaching excellence?”
Professor Virgo: Could I just come back and revisit your first point, which is that many of us were rather down on TEF? That is certainly not my position as a matter of principle. If TEF works as a kitemark of excellence across the whole sector, that will be very beneficial, because then we can say we have satisfied the kitemark. Maybe we can say that to a large extent already with quality assurance, though it is not specifically teaching excellence, but there are elements of that within quality assurance. The reason I am concerned about TEF is when we start getting into the details and having the confidence, and making sure the sector and all the stakeholders, particularly the students, have the confidence, that the measures being used, the metrics being identified, are the appropriate ones. That is where the discussion has to be.
Q50 Richard Fuller: Just so I can be clear, you are very proactively involved, in your submission, in trying to solve that problem, and other universities, saying, “We have to get this right. We want to have buy‑in, so let us make sure we are using this consultation to collectively say, ‘We can do it, but here is the way that we need to do it.’”
Professor Virgo: Absolutely right, and so we are talking among ourselves, within institutions, between institutions, in responding to the Green Paper. There is a lot of discussion going on.
Q51 Richard Fuller: Just to be clear again, does that mean that there will be a collective response? You said you have to get buy‑in or at least a very clear similarity in the number of responses about the things that can be done and need to be done. Do you think it will be, “We think this” and “We think that”, and actually therefore they cannot do it?
Professor Riordan: This is a very complex set of issues and there is not going to be any simple, straightforward solution that has no problems with it. To come to your question, there will be collective responses. There will be more than one. There will be one from Universities UK, which will try to embrace the diversity of views through the whole sector. Then there will be one from the mission groups, the Russell Group, million+ and so on and so forth, and certainly from individual universities as well. The fact is that, whatever system we end up with, it is going to be very, very difficult to have something where some bits of the sector do not feel, “That is not very good for us” or “This element of it is not very good for us.” That is going to be inevitable, because the diversity is real. Like it or not, it is just there and it does create a set of issues that we then have to cope with.
We are being asked, “What do you think about TEF? Are there problems with it?”, and so we are answering, “Yes, there are some, and we need to know more detail.” There will definitely be, as Graham has said, a collective will to engage with it. We in Wales could just simply say, “TEF is nothing to do with us.” I really do not think that is going to happen. We are going to want to be part of it, although there will not be a link to fees, because it is very important that we have a UK‑wide system that is a kitemark, that does illustrate the quality and excellence of our teaching. We would love to make that happen. It is just it is not easy. It is very, very difficult, and we must not try to pretend that there is some simple solution.
Professor Ward: I am looking through the written submissions to this Committee, and I read through most of them over the weekend. There is talk about diversity of the sector, but I was quite struck by the commonality across different mission groups. You have about 30 individual institutions all feeding in. There is a set of issues that are very, very common across the submissions: concerns about the timescale, about the unsophisticated use of metrics and the risk of some of the unanticipated consequences.
Katie Akerman: If I am being grumpy, it is not about the TEF.
Richard Fuller: I wish I had not used that word now.
Katie Akerman: Ultimately, the TEF is perhaps something that would benefit an institution like Chichester. My concern, given my particular position in the university as a director of quality and standards, is about the HEFCE proposals for the future of quality assessment. The idea of trying to manage two potentially complex, burdensome processes for what is a very small university and a very small quality assurance team could be quite problematic, so my concern is about the two things taken together, rather than the TEF itself.
Q52 Chair: For my own clarification, Professor Tynan, you mentioned Sweden, Australia, Canada and others.
Professor Tynan: There are probably many more.
Q53 Chair: My question is this: are we behind the pack when it comes to having in place a good, sophisticated teaching excellence framework?
Professor Tynan: No, we are not. In fact, I think it is an issue that is grappled with within every country in terms of how you measure this, and then how you reward it, how you incentivise it and how you improve it. If you look at any higher education policy in any of those jurisdictions, you will see a litany of things that people have tried. It is similar if you look here and see what has been tried over the last 10 or 15 years.
Q54 Chair: So we should not beat ourselves up as a country with regards to this. Is that fair?
Professor Tynan: No. I have said in other public forums, as a relatively new entrant to the UK, that UK higher education is widely respected around the world. That is proven by the kind of international student pickups that you have coming here. This country should certainly not be beating itself up unnecessarily about trying to measure itself.
Professor Ward: I am not sure it is widely understood how robust and rigorous the quality assurance framework currently is. We have just been QAA’d at the University of East Anglia. A team of very experienced reviewers come in for a week. They speak to a whole range of different groups, including student groups, and really mine very deeply how the teaching is delivered and how quality is managed and assured in the institution. It is a very, very thorough process.
Q55 Jo Stevens: I will ask a question that Colin may have partially answered already. Thinking about the future, promoting the sector, strengthening the sector, is the potential to have different quality assurance mechanisms in Wales, England, Scotland and Northern Ireland problematic for marketing universities?
Professor Riordan: Certainly from our point of view, and I think from a UK point of view. In terms of the QAA quality assurance system, I think we are leading the pack. We have very often had people come to us and look at what we do. We play a leading role in Europe as well, in the European quality assurance system, and always have. One of the great strengths has been that it is consistent across the piece.
From a Wales point of view--I am sure it would be similar for Northern Ireland; I cannot speak for Scotland because they very much speak for themselves—there is a general sense that a UK‑wide consistent system is going to be much more use to students. In Cardiff, 65% of our home undergraduates are English anyway. If they are thinking “Bristol or Cardiff?”, they will want to be able to compare those two, so I hope it is going to be possible to do that. I do not think it matters whether the fees and funding system is connected to it or not. That, in a sense, should not really matter. What matters is that quality assurance and whatever measures we can have for excellence in teaching are consistent.
Q56 Jo Stevens: That is quite interesting, because we heard in the earlier session about the link between fees and the measuring. I am struggling to see how that is going to be disaggregated under what is in the current proposals from the Government.
Professor Riordan: With REF, the assessment is done and that is just done on a free and fair basis. It does not matter what university you are at. HEFCE then decides how it is going to distribute the QR on the basis of that. HEFCW, the Welsh funding council, decides how it is going to distribute funding, and it does not have to be the same. It could be completely different. It is quite similar, because the principles are widely accepted, but it could be quite different. The assessment is one thing and then how you fund on the basis of that is a separate thing. That has always been a major principle of REF, and it should be one of TEF as well.
Q57 Amanda Milling: This might be a bit of a woolly question. You talked a lot about diversity and also in terms of diversity of outcomes as universities. I cannot help thinking there must be a common mission somewhere. I would like you to articulate what you believe the common mission is for the sector.
Professor Riordan: Do you mean in terms of the education of students?
Q58 Amanda Milling: Yes. You talked a lot about the diversity, so I am trying to pin down what the similarities are. What are we trying to achieve?
Professor Riordan: There are two aspects to this. A university degree is giving people the ability to think for themselves and take off from a higher intellectual platform. It is not just about learning outcomes. It is not just about learning things. It is learning how to learn and to really be able to develop not just yourself personally but your career, the organisations you work in and the business that you create. It might sound a bit nebulous, but it is really about giving people the capacity to continue to learn throughout their lives and to understand that that is a way of proceeding in life.
The other question you have to look at, behind your question in a sense, is: are degrees from Chichester and Cambridge pretty much the same in terms of outcome? There might be different missions there. It might be that the trajectory from one university is different to the trajectory from another, but what we would always want is a threshold, a standard that everybody reaches. That is what the quality framework tries to achieve in terms of standards: that, in each case, the standards reach a threshold level so that you can be as sure as you can ever be that a degree is worth a degree.
Professor Virgo: The answer to your question, and it is a woolly answer, is an educational experience, which clearly has elements of teaching and learning as well that together form the educational experience. Immediately you start to go beyond that, we then start to see the diversity of the sector. I could say, in Cambridge, it is a residential educational experience, which is strongly affected by the research, so it is research‑led teaching, etc. The Open University, for example, is going to be different. It is set up in a very different way, but we still have the common objective of education.
Professor Ward: British higher education takes students to the frontiers of knowledge really, even at the undergraduate level, particularly in research‑intensives, where you are being taught by people who are actively doing research and pushing forward the boundaries of knowledge, whether it be in science and technology or in arts and humanities subjects. There is that sense in higher education of giving students a taste of knowledge production and what it is to be producing new knowledge, new ideas and new technologies.
Professor Tynan: You have asked a really hard question, because it goes to the heart of the philosophical basis of what higher education is for and what education is for. When I think about that in relation to the Open University, I am thinking about the kinds of graduates we are trying to produce, graduates who can operate in a really complex world, who can be adaptive, who can manage three different careers in their lifetime. You are trying to develop people who can contribute to solving the world’s greatest challenges that face us, poverty and producing food and the digital economy, all that sort of stuff.
We need a man who recently passed away, who was very famous actually, called David Watson. He was at Oxford. It would be worth reading his book. It is called The Question of Conscience: Higher education and personal responsibility. It is a fantastic journey of higher education in this country. It is quite a thin little book, but it gives a really interesting overview of how the missions of universities have changed, largely to do with policy and how Governments have shaped what higher education aims to achieve. But it is a really hard question.
Q59 Chris White: I am going to follow straight on the back of Amanda’s question. The three of you have not mentioned, as part of your shared mission vision, employability. I was just going to see whether you would like to add that or what your view on your role in employability would be.
Katie Akerman: That is a really interesting question, because my immediate response to your colleague’s initial question was Plato and The Republic, and his ideas around philosopher kings. In some senses, I very much personally treasure the idea of education for the sake of education. That is why I did a history degree, because I loved history, and I still love history. As Belinda was saying, universities have shifted incrementally over decades and decades and decades, mainly through the impact of Government policy. Although, in a sense, universities are resisting becoming what they see as training organisations, there is obviously a definite sense from Government policy that what we are trying to do should be about employability. Universities are responding very much to that.
For example, one of the things we ask our external examiners to comment on in their reports is the extent to which employability is embedded in the curriculum. Again, for example, we use external advisers from industry in terms of programme approval. One of the slight concerns we have around the HEFCE proposals for the external examining arrangements is that, if there is a central register, we may lose some of those people from the professions, from the industry, whom we currently use, because they may not be willing to sign up to a centralised register.
Professor Riordan: Employability is extremely important for all universities. The fact that we did not mention it then is because we were thinking about it in a slightly different way: what is the purpose of a degree? The purpose is not just to get somebody a job. You can get a job without a degree. It is a question of what kind of a job and what the needs of the economy are. I do not think there is a university in this country that is not extremely committed to employability, that does not put an awful lot of effort into it and that does not have people employed specifically to help make that happen.
Academics in all departments, including philosophy, English, if I go round the place, will assure me of their commitment to that because they care about their students getting a job. They realise they have to earn a living, they have gone through a very, very lean time over the past five or six years and now things are picking up. You should not be in any doubt at all about the real commitment to employability, a word that did not even exist 10 years ago, by the way, in universities in this country.
Q60 Chair: I just want to touch upon what I think is a very important part of the TEF, which is the link between a TEF rating and the ability to increase tuition fees. Do you welcome that link? Would you like to see more freedoms and flexibilities other than allowing it to rise by the rate of inflation? Graham, I am going to come to you first. Would you want to see a large amount of freedom, so that, if Cambridge as a whole gets a level 4, you can raise your tuition fees to £20,000? There would be a market for that. Is that something that you would agree with?
Professor Virgo: We would not want to do that. We are very concerned about the effect of raising tuition fees on widening participation. We spend an awful lot of time engaging with students, seeking to widen participation, and we keep getting the message of concern about fees and the level of fees, despite the fact that we offer very generous bursaries to support those students. Any link between TEF and raising the fees is a cause for concern.
Q61 Chair: Should it be taken out?
Professor Virgo: I would not go that far, because it is quite clear that, across the sector, there is concern that fees are not keeping up with inflation. Cambridge University’s position is that we have no desire to increase fees dramatically. If we were allowed to increase them up to the level of inflation, we would be content with that, but we would not want to go beyond.
Q62 Chair: Is that a consensus across relevant institutions?
Katie Akerman: I would certainly agree for Chichester that we would have some concern about raising fees, for the reasons already outlined. Right at the very beginning, we were talking about perhaps potential inequalities between perceptions around research and around teaching. One thing that I would imagine strikes a number of people, including myself, is that, in effect, the sector is paying for the REF but, with the TEF, we are expecting students to pay for teaching excellence. That strikes me as a real inequality in the proposals.
Professor Riordan: We in Wales want to take part in TEF, but I am sure that we are relieved that there will not be a connection to the raising of fees and that we are going to be looking for different ways to finance and fund the higher education system.
One of the most obvious reasons is the potential for perverse incentives. If employability is going to be a key metric, are you going to be inclined to put on programmes that do not necessarily have high employment rates, but might be a very, very good thing to be doing and ought to be something that, in a properly cultured country, you should be doing? On retention, are you going to be inclined to take more widening‑participation students who might be less inclined to continue the whole programme? On NSS, if students believe or know that you can put up fees if they are highly satisfied, are they going to be highly satisfied when they fill in the survey? I put it to you that perhaps not.
Professor Tynan: Similar to Katie, from our perspective, there is something ironic about this. In comparison to REF, with TEF, our widening participation schemes are going to be the ones footing the bill for it. That is an unfortunate outcome.
Professor Ward: I thought this was an issue where there was a strong commonality in the submissions that you have received. There is a lot of concern about the link with fees. We are in a cost/price squeeze, though, in that staff costs are going up and the general level of inflation in the sector is higher than in the rest of the economy, so the fees are capped and yet our costs are going up. I think we would very much like to be able to raise fees. The unfortunate thing is that it is linked to the teaching excellence framework, which is a problematic link, really.
Chair: The final question is from Paul.
Q63 Paul Blomfield: I will also just strike a final note. I think we are all conscious, as Jo Johnson says in the introduction to the Green Paper, that our universities are one of the great success stories of this country and we have some of the finest universities in the world. The points you have made about the work that is already going on on teaching quality, in relation to HEFCE’s research, and the strength of the QAA are issues that we will explore with both HEFCE and the QAA subsequently.
I wanted to ask two questions, if I could. One question is whether measuring teaching excellence by institution, as Colin touched on much earlier in the discussion, is the best way of recognising the huge variety. I represent two universities in Sheffield. Both are excellent but both are stronger in some departments than others. The question is whether institutional measures are right or whether we should be focusing on departments more? What impact will that have on the fees issue, in that some departments that may not be strong will get higher fees, presumably, under a one‑size‑fits‑all approach?
The second question is just to leave us with thoughts as we meet with subsequent witnesses. Everybody here is committed to improving teaching quality. What are the unintended consequences you worry about that we might need to explore with subsequent witnesses?
Professor Riordan: I will start on the institution or department. It is problematic. The bit that causes the problem is the peer review part, so the independent panels of experts. There are issues with that. I do not know who these experts are out there who will know how to assess teaching, on what basis or how you define excellence in teaching. There are an awful lot of unanswered questions about that. That is very difficult to do for a whole institution. If you are going to have a peer review element to it, with independent panels of experts, that almost has to be on a departmental, disciplinary level. That is unless somebody can come up with another way of doing it, but I cannot think of it.
Then the question is how you compare between universities, because we are not always organised in that way anymore. The programmes are so different. It is back to the diversity issues. There are a lot of issues with that, which is why we have a quality assurance system that looks at the whole institution and does not attempt to make comparisons any more—we used to do this—between departments and does not really profess to assess teaching excellence. It is more the whole circumstance and what you have in place to ensure that teaching is going to be excellent. Then the external examiners are the ones who look at the outcomes and come to a view as to whether the outcomes show high standards being achieved. You can derive from that probably that teaching is good or excellent if the outcomes are and the whole environment is good. That is the way we do it at the moment. Moving to more of a peer review process is going to be difficult.
The second one is unintended consequences. I think there will be far fewer of them if the money is not linked to teaching. I sat in front of a committee like this, six years ago or so, chaired by the now Lord Willis of Knaresborough. I chaired an inquiry for HEFCE into exactly this issue, and we did not have the answers then. We were asked that question: why not have financial incentives for good teaching? One obvious one is that, where the teaching is rated as excellent, there will be more money available, so you can make it better and better. If you have not got the money and it is not rated as excellent, there might be a downward spiral, so there is a danger there.
Also, there are the unintended consequences of perverse incentives that I just mentioned a moment ago, so I will not go through them again, but there are plenty of examples where you might find people playing games, such as, with the learning gain part, teaching to a test at the end rather than focusing on getting a good degree in terms of their education.
Professor Virgo: The answer to the first question about focusing on disciplines rather than institutions in terms of the teaching excellence, from a student perspective, and particularly giving information to the student, makes an awful lot of sense. Students are more concerned with the quality of the teaching they will get in their particular discipline, perhaps, than the nature of the whole institution. In terms of linking that then to fees, having differential fees based on discipline, involving allocation of resources within universities, that will create a great deal of bureaucratic complexity.
On the second point about issues to be aware of as to potential downsides, frankly, I think it is simply damaging the brand. As you have just said, higher education institutions in this country are very well respected nationally and internationally. If a TEF is created that has the effect of identifying a significant number of those institutions as not satisfying whatever metrics are chosen and being used to determine excellence, I am concerned about the impact on the brand.
Professor Ward: I am very concerned about the potential introduction of the graduate earnings indicator in the basket of metrics. That opens up a whole load of risks around incentivising universities to change their shape. We know that women earn less than men in the graduate employment market. You could see universities potentially changing the profile of their offer so that they are just recruiting students to particular disciplines that earn more. We know that there are differences according to ethnic category as well. These are problems in society, but to start to reward universities on the basis of how much their graduates earn three, four or five years down the line is a recipe for disaster.
Professor Tynan: I would just add to what my colleagues have said here that it is important we remember part‑time and flexible study and that the aspirations of learners and employers can be quite different. If a degree is the only form of measure, there are other qualifications there. I still have another question, which is unanswered, around the kind of red tape that will occur and how it is going to fit carefully with the QA processes that exist. As my colleague was saying here, the costs attached to rolling something like this out do need to be taken into account.
Katie Akerman: As we outlined in our submission, again, we would be happy with a departmental or discipline‑level approach. To add, in terms of unintended consequences, it is very much for us around fragmentation and the brand, UK HE plc. We need to take care that, with the TEF and with the HEFCE QA proposals, those are managed.
Chair: We have kept you longer than anticipated and I apologise for that, but that is a reflection of the excellence of your evidence. I know I speak for all the Committee in thanking you. We have got an awful lot out of that. There is an awful lot to think about. Thank you again for your time. We really appreciate it.
Oral evidence: Assessing quality in Higher Education, HC 572-i 15