Education Committee
Oral evidence: Exam results 2022, HC 718
Wednesday 12 October 2022
Ordered by the House of Commons to be published on 12 October 2022.
Members present: Robert Halfon (Chair); Apsana Begum; Miriam Cates; Kim Johnson; Ian Mearns and Angela Richardson.
Questions 1 - 98
Witnesses
I: Dr Jo Saxton, Chief Regulator, Ofqual; Jill Duffy, Chief Executive, OCR; Mike Howells, President, Workforce Skills, Pearson UK and David Hughes, Chief Executive, Association of Colleges.
Written evidence from witnesses:
– [Add names of witnesses and hyperlink to submissions]
Witnesses: Dr Saxton, Jill Duffy, Mike Howells and David Hughes.
Q1 Chair: Good morning, everybody. Thank you very much for coming to the Committee today. For the benefit of the tape and those watching on Parliament TV, can you kindly introduce yourselves? If I start with you, Jill, please.
Jill Duffy: Good morning, I am Jill Duffy and I am the chief executive of OCR.
David Hughes: David Hughes from the Association of Colleges.
Dr Saxton: Jo Saxton, chief regulator of Ofqual.
Michael Howells: Michael Howells, president of workforce skills at Pearson.
Q2 Chair: Thank you. I just want to start off, if I may, with the delays to BTEC and CTEC results. We know that thousands of students did not receive their vocational results on results day this year: 7,000 did not receive their Pearson Level 2 BTEC grades on results day and 3,300 Level 3 BTEC results were not received. A further 3,200 students were affected by the delayed Level 3 results from Cambridge Technicals recorded by the OCR.
David, in a letter published by you, you asked exam boards to be open and clear about how many students had delayed results and the exam boards have been accused of not being transparent and sharing details about the delays. We know there has been quite a disproportionate impact on schools and colleges and, in one college, at least a third of their results were outstanding.
I should have declared at the beginning that I am proud to have a huge Pearson HQ in my constituency which does the publishing arm. I should declare that.
Having said that, Jill and Mike, do you agree that the process was shambolic and left pupils who undertook vocational qualifications, yet again, feeling like second class citizens?
Jill Duffy: Thank you for inviting me here today. First, I just want to apologise profusely to any student that did not receive a Cambridge Technical result on results day. We did deliver the vast majority of Cambridge Technical results, 39,000, on results day, but I know from the calls that I had personally with teachers and students who were waiting for a result, that this was adding to their stress and their anxiety and I deeply regret that. Within a week we had delivered 1,500 results to all those eligible students who were waiting for those results for a university place and at the moment, as I speak, we are running an internal review to see what happened, what went wrong this year, and to make sure it does not happen again.
Q3 Chair: How many young people were unable to take their next steps, whether to FE, apprenticeship or university, because of the exam boards vocational results delays?
Jill Duffy: I am not aware of anyone that did not get their university place because of a delay to the results. As I say, within a week, we had released all results to students with urgent university places, so I am not aware of anyone.
Q4 Chair: David, do you agree with that or not?
David Hughes: It is very hard to tell because we do not know how many students and which students, so it is quite difficult to be certain. I think the anxiety was the real problem.
Q5 Chair: Mike?
Mike Howells: Thank you, Chair, and thanks very much for giving me the opportunity to be here today. Very much like Jill, the first thing I want to do, on behalf of Pearson, is to apologise to everybody affected. I personally am very sorry for the stress that this caused for many students and their families. Any delay in a result is unacceptable, as far as I am concerned, and, like OCR, we take full accountability for our role in that. We are also in the midst of an investigation into exactly what happened, and I can share some of those findings and the data about that with you today. There are certainly some improvements that we need to make to our processes, and I am happy to explain those to you.
To your point about students affected, to my knowledge, there was not anybody who did not receive a university place if they were progressing through UCAS. Everybody who met the standards for their university offer through BTECs received a place. There were a couple of cases where students had concerns about receiving a place through clearing that Pearson assisted them with directly but, to my knowledge, there was not anybody who did not receive a place who was entitled.
Q6 Chair: When all this was going on, a spokesperson from Pearson said that a tiny percentage of grades were ineligible, and this was the case every year. They went on to say that the proportion of missing grades was, and I quote, "Typical for this stage in the awarding process compared to pre-pandemic years". I just find that statement incredibly depressing and surely you would agree that calling thousands of pupils affected by this, "A tiny percentage", is dismissive and fails to acknowledge the stress and misery and upset caused by the delays on results day. I remember, I did not enjoy exams at all when I was growing up and it was enormous stress waiting for those results. Let me ask why you think that thousands of students on vocational courses missing their grade is a typical state of affairs?
Mike Howells: Well, first of all, I do not think anybody missing a grade—
Chair: That is what your company said.
Mike Howells: That speaks to some of the questions we should get into around what the data tells us. I think what the Pearson spokesperson was explaining at that point is that if you track historical trends in data and, in particular, when we receive requests for grades from schools and colleges and when we receive information to show that students have completed their courses, their course work, their examinations, and so on, that data was showing typical trends in terms of—
Q7 Chair: Do you not think that is a pretty insensitive thing to say on a day when thousands of students are not getting their vocational results? It is just like, “Let them eat cake” from Marie-Antoinette.
Mike Howells: I believe very strongly any missed result is unacceptable.
Q8 Chair: The reason why I am annoyed about this is it is so typical of the way we treat vocational qualifications in our country. That they are seen as something less and thousands of students not getting the results, that is typical or it happens every year. The impression that you gave is, “Who cares?”
I want to understand from Jo, why, if this happens every year as has been described, the exam boards have not done more to address this? I know why. It is because you are all graduates and so vocational qualifications do not mean the same. They are forgotten every year. Even two years ago when we had the exams debacle, they changed the grading for exams but not vocational qualifications. They had forgotten about vocational qualifications if you remember, whatever year it was, 2020, I think it was. Do you want to respond to that?
Dr Saxton: Yes, thank you, Chair. The Committee will remember, I hope, that I came to this post determined to do whatever I can to support parity of esteem and I myself have two vocational qualifications in my own history. While I am conscious that there are complex issues at play here, and there are important flexibilities in vocational technical qualifications which are absolutely in the interest of students, providers and employers, I can tell the Committee that this has absolutely shocked me to the core in my first year as chief regulator.
Q9 Chair: Even though it goes on every year?
Dr Saxton: This has been the thing that has shocked me the most.
Q10 Chair: It has gone on in previous years so why is it such a shock?
Dr Saxton: This is my first season as chief regulator so it is not something that I had experienced before. I am absolutely determined that, on my watch, I will do everything in my power to make sure that students are protected from similar stress. Again, that is why I have commissioned the widest scale review of its kind—in fact, I think the largest review that the regulator has ever called—because I want to know exactly how students have been affected. We have done an unprecedented call to both the sector and the students to ask for a wider evidence picture. We want to understand their experiences so that we can make recommendations to make changes for next summer’s awarding.
Chair: Previous reviews and previous people have promised that it will not happen again, and it just happens all the time.
Dr Saxton: This chief regulator has not made that commitment to you.
Chair: Ofqual has and the exam boards.
Dr Saxton: I cannot speak about the commitments made by previous chief regulators.
Q11 Chair: David, is the issue of delayed vocational results a systemic issue, and what, in a nutshell, should be done to change it?
David Hughes: I would say they are two issues. I think there are some systems issues which we absolutely need to talk about and we need to get right for next year.
If we just go back to results day, I do not understand, when the awarding organisations here found out the problem existed, they certainly did not tell us about it. There were thousands of students on results day expecting a result after two really difficult years of interrupted learning, who were suddenly told that there was no result for them. Imagine the stress they were under and the anxiety that caused. I think it was just shocking. Even worse, and this is where I think, yes, it was shambolic, but more than anything, it was disrespectful to those students to not come out immediately and apologise and give the numbers immediately and tell people what they were going to do. I went on the media on the Friday and Saturday and Sunday because these two organisations decided not to and did not put any spokespeople up. I went on to try and give some reassurance to students that actually somebody really cared about them.
I would really like Pearson, and OCR to just tell us a bit more about the decision-making, about why they did not communicate an apology immediately, why they did not put out communications to reassure, and then I think we need to get onto some of the systems issues.
Chair: You have read my mind with the next question, I will come to that in a bit.
Q12 Ian Mearns: Just on that point; David, are you not surprised that the organisations did not contact colleges prior to the day to let them know that there was a looming problem?
David Hughes: As I understand it, they were making contact with the exams officers. The exams offices are fairly junior people in colleges, dealing with 30 or 40 exams boards, completely overwhelmed at that time of the year. So yes, we stepped in to try and help the college leaders understand what was happening because they were not being told. So, there was no plan in place for—
Q13 Chair: As I understand it, the exam boards knew about these problems as early as July because they resulted from delayed school results, yet action was only taken around results day. I am asking Jill, Mike and Jo, when did the exam boards first realise there was a problem, the first results would be delayed, and when did you inform the Secretary of State and Ofqual?
Jill Duffy: We first knew there was a problem on what is called restricted results day, which is the day before results day. What we were noticing was we were having calls into our customer support centre that were taking longer to resolve and this continued on results day. So actually, we did put out a public statement on results day. We did apologise on results day, and we had a meeting with the stakeholder forum on results day, and there was a member of AoC at that meeting.
Q14 Chair: When did you let the schools know?
Dr Saxton: We were talking to the schools all the way from restricted results day onwards. A couple of things have not come out here. Chair, you said this happens every year. We were looking at a scale of late results that was probably four or five times what we would see in a normal year. To explain the reason for this, Cambridge Technicals, they are unit-based qualifications so we rely on a school and college claiming for a unit. They are a mixture of examined qualifications and coursework that are assessed in schools and then we moderate.
We had a range of pandemic adaptations this year; there were several of these. A school or college could apply for what we call a reduction in assessment. That means they did not do the assessment for one unit. They would do the teaching for it. They would also have unit tags from the previous year, and they would have assessed grades if they had not been able to sit their exams in January. What all this meant was that schools and colleges needed to tell us about all of this, to tell us about these units, and then the qualification grade is automatically produced around results day.
So the issue that we had here, vocational qualifications like these are more complex than A-levels, but we had pandemic adaptations that we had brought in for the very best of intentions and those intentions were to make sure that vocational learners could progress, could get their qualifications and could move on to their next step. In putting in those adaptations, top of mind here was that we did not want vocational learners disadvantaged against general qualifications where there had been a series of adaptations put in place.
So yes, it was more complex this year. We knew we had a problem the day before results day. We put out a public apology on results day, and we were in constant communication with schools and colleges during this time.
Mike Howells: I do not want to repeat everything Jill said, but it is essentially the same process for Pearson, and we took the same steps like OCR. This began to come clear to us on the day before results day, on the 17th, when we were issuing those results to schools and colleges, and they were informing us that they were having some gaps there.
To your point, Chair, we are in deep partnership with all the schools and colleges. We work very closely with them throughout the year. We work with them before deadlines for submission of information on 5 July and obviously in that period, 5 July to 18 August, or 25 August for Level 2, and that is the period in which we do a lot of the reconciliation and remediation of challenges. So, as David mentioned, there is a huge amount of support there to help colleges and schools to work through any outstanding results, any mistakes they might have in their data, any gaps they might have.
The point though, if I may just go back, about normal data patterns concerns the fact that there are always a number of students who centres, schools and colleges let us know earlier in the academic year are likely to be expecting a grade, but then ultimately do not complete that year for a range of different reasons. It may be because they have chosen to move into work, it may be because they have chosen to extend or to change the nature of the qualification that they are going for. That flexibility is a very valued part of the qualification but what that does create is a number of cases in systems where there is an indication that a student might be expecting a grade but where in fact they are not. Some of them do proactively withdraw their application to UCAS, for example, but many of them do not. When the Pearson spokesperson was talking about typical pictures in data around unresolved cases, that is what they were referring to.
Q15 Chair: I will bring in my colleagues in a minute. We know in 2020 for some students the BTEC results were delayed for more than a fortnight or so, and you made quite clear your unhappiness at that time. You are now saying you are having a review even though Ofqual, given what has gone on in the past, would have known that these kinds of problems happened and it should have been, in my view, on top of it rather than just waiting for something to go wrong to say, "There’ll be another review". What I am trying to understand is how will you ensure that another review will make sure that it does not happen again next year? We would just be having groundhog day in Committee next year.
Dr Saxton: 2020 was a totally different set of circumstances, Chair.
Chair: Yes, but you still get—
Dr Saxton: I can only make commitments as the chief regulator now that I am going to do everything in my power to protect students from this happening again. One of the ways that the regulator acted, for example, is we urged the boards. This is after all their data, and they are responsible for the delivery of results, but I urged both Ms Duffy and senior Pearson presidents to provide the data, put it into the public domain, because I believe that would enable the sector to best support students to increase that transparency. Mr Hughes and his colleagues were very supportive in helping the call to the sector to meet the needs of students.
What I have tried to do is put myself in the shoes of students. I feel their frustration. I cannot pre-judge the outcomes of our review yet, but we need to establish the facts and separate out which things are systemic, which things related to Covid mitigations, and we will make wide-ranging recommendations and implement those which are in our power, as I say, to protect students from this happening again.
Q16 Chair: Do you think, in essence, that the mistakes and errors from Pearson necessitate a review into its status as an awarding body? We know that, sadly, Pearson is facing the largest fines in the history of Ofqual. You have published two regulatory notices against Pearson based on problems in awarding GCSE and A-levels between 2016 and 2019.
If I just make a wider point that I am supportive of BTECs and I lobbied the Government on BTECs and I think all this, all it does is undermine the case for BTECs. All you have done is shoot yourselves in the foot with this. I just wondered if Jo could respond and then Mike.
Dr Saxton: I think Pearson is an interesting case of where, if an exam board does not come up to scratch, the regulator takes strong action. You referred to the enforcement action which we have given our notice of intention to undertake, but I must put in context that Pearson and OCR, between them this season alone, delivered something like 2 million qualifications to students, the vast majority without incident and without error. Any mistake and any late result is, of course, absolutely unacceptable, but we must, in the interest of public confidence, put it in that context.
Q17 Chair: But on the academic qualifications, the exam boards issue thousands of qualifications without error, but you do not have thousands of people in academic qualifications missing their results.
Dr Saxton: And I think there is, because of the scale—
Chair: You are implying, "Oh, it is okay because they got most of it right".
Dr Saxton: I am absolutely not saying it is okay.
Chair: That is what you just said.
Dr Saxton: Any late delay is unacceptable.
Chair: You just said that.
Dr Saxton: I said any late delay is unacceptable, but the vast majority were delivered in time.
Chair: You just said, “Oh, but you’ve got to remember that they did a pretty good job with most of them”. If this was some of the academic exam boards who had thousands of students not getting their results, you would have gone mad. And so, by the way, would the academic sector, the educational establishment. But because it is vocational qualifications, you say, “Actually, they got most of it right so that is okay”.
Dr Saxton: I am saying the majority of the vocational qualifications were also delivered on time and without incident. Any late result, whatever the qualification, is absolutely unacceptable and the exam boards have felt the full force of my frustration about that. We will publish our review.
Q18 Chair: You are going to publish a review, and then what will happen next year that is going to be different?
Dr Saxton: Our review is wide-ranging and is being undertaken as we speak. It will be published during 2022 and, as I have already said, we will implement any of the recommendations which are within our power to do so. I have already talked to the Secretary of State about my thoughts on some of the changes for protecting students in 2023.
Q19 Chair: David, what do you think of everything you have heard so far? Be honest.
David Hughes: I think you are right. They treated students who are doing their exams with disdain in not communicating clearly. I think the Ofqual review is good, but I think there are some simple things that need to happen quickly.
I would really urge you, Jo, to make the changes now and not wait for the outcome of the review because I think there are some obvious things that could be done quickly that we do not need a review to find out. And, having had discussions with OCR and Pearson, I think they would be willing to do it. Let us move really quickly because things take time in systems change. If we wait for the review to report and then implement, I think we might miss next year and we might be in the same place we are now.
Chair: In my experience, reviews are often excuses for doing nothing and just trying to shove the issue under the carpet.
Kim, please, you had a question, and then I will bring in Ian.
Q20 Kim Johnson: Thank you, Chair, and good morning, panel. I just wanted to pick up on a couple of points. Firstly, how soon was the Department for Education informed about the debacle with results day and what are both OCR and Pearson doing to ensure that does not happen again? When are we likely to see the review?
Jill Duffy: Thanks, Kim. We told Ofqual and Ofqual would have told the FE on results day. So we were in communication with Ofqual on results day about the late results. We had a meeting with the Secretary of State, I think it was on about 22 August, about the late results and what we were doing to ensure we got the results out as quickly as possible.
In terms of what we think we can do to make sure this does not happen again, David has mentioned we have been talking to the AoC, we have been talking to lots of other stakeholders as well. I think there are three key things that we can do. We can make sure that we improve the data sharing, and there are two points to this. Firstly, the data sharing with schools and colleges so that we know, and schools and colleges know, where students are in terms of the progress of their qualifications and where they are in terms of the units they have got so that we are aware, from schools and colleges, which students are expecting to get a grade on results day. And so that schools and colleges can track the progress, if you like, of their students so that they are also confident that students will get a result on results day. That is one thing we can do.
With UCAS as well, on results day, 18 August, I requested from UCAS a report on all the students who had a university place that had a Cambridge Technicals qualification as part of that. I have been talking to UCAS since then to make sure that we have better data sharing with them to have that information, not on results day, but through much earlier as well, so that we have a better indication of which students are going on to university. That is one thing.
I have also talked to David about this as well. We think that we should have an earlier results day for Cambridge Technicals to give schools and colleges more time as a safety net, if you like, just to check that they are getting the results. We did have an earlier results day before the pandemic, and then it was aligned with the GQ results day during the pandemic so it is time for us to have a look at that across the system and say, “Should we be giving these vocational results out under embargo a week earlier, just as that safety net so that schools and colleges can check?”
And the third thing that I think we need to do, we did an awful lot of communications into schools and colleges—
Chair: We have to get on.
Jill Duffy: Yes, okay.
Q21 Chair: Sorry, Mike.
Mike Howells: I will be brief, but thank you for the question. Ofqual runs a series of forums throughout the year to make sure that we are sharing and communicating progress in awarding, which of course we participate in fully, but there is a lot of daily and sometimes hourly communication over particular challenges and cases. There was a lot of regular, very high quality, dialogue between Pearson and with Ofqual. We met with DfE officials on 21 August, and then with the Secretary of State on the 22nd.
To your question about improvements, we are in the midst of an in-depth review at the moment as well, but there are three findings that have already emerged. If I can maybe run through those very quickly? The first is around communications and our support to schools and colleges, making sure that the information and the assistance that we can and do provide to them is as effective as it can be in helping them manage some of the challenges that they are going through. Secondly, the process and deadlines, and I think we have shown probably too much flexibility actually, particularly during the pandemic, in supporting late submission of data and that is something we need to improve and work on. As David and Jill mentioned, look at different kinds of deadlines.
And then lastly, clarity around the data itself. One of the important things to remember about vocational qualifications is that the idea of a results day is actually a relatively recent concept. People can roll on and roll off on these qualifications at any point in the year. Many students complete at a point of their choosing because it suits their particular needs and circumstances. One of the great successes of vocational qualifications is how valuable they have become at producing a new route into higher education. One in four students who goes to university does so with a BTEC, so we have worked very hard to make sure that results are available for them to do that on a results day, but it is a more mixed picture than that. There are a lot of students who complete at other points throughout the year. So, reconciling those two different approaches in the data, working in partnership with David's colleagues, is something we need to do.
Q22 Ian Mearns: Prior to results coming out, it emerged that some exam boards had provided incorrect information regarding the content of the exams. There are some good examples of that. In July this Committee wrote to the Secretary of State and the chair of Ofqual, expressing concerns about errors and Ofqual responded on 31 August saying, “In the small number of cases where errors did occur…we expected exam boards to communicate swiftly and clearly with students and their teachers”.
So, Jo, this summer, with the errors made by some exam boards regarding their exam papers and the advance information provided, what steps did you take to ensure those exam boards were held accountable? And when you say there were a “small number of cases where errors did occur”, how many is a small number?
Dr Saxton: First of all, any error in assessment materials is unacceptable and Ofqual continues to bear down on them. While there are errors which have been unacceptable this season, there were fewer than the last time exams were sat in 2019. There were also fewer leaks.
Q23 Ian Mearns: Fewer meaning what against what?
Dr Saxton: In the context of advance information specifically, there was poor communication in some cases, but only two actual substantive errors. I have had have assurances from the exam boards that those were dealt with either through students getting the full marks or at the point of awarding so that they were not directly disadvantaged in marking or awarding. Although I do appreciate they will have suffered distress.
The number of actual errors in assessment materials is official statistics which I can publish later in the autumn, but broadly it is about 20% fewer than the last time exams were sat. In terms of the advance information, as I say, there were two substantive errors and, if I can just put that in context, again with the caveat that any error is unacceptable, we must remember that this was an effort to do something unprecedented. It was novel. Advance information had never been provided before. It was done at pace to support students with their revision and taking exams when we had had no exams for three of years, and it covered more than 300 qualifications and many more exam papers than that.
Q24 Ian Mearns: Even though it is an official statistic, 20% fewer this time than previously, what does that mean in actual numbers?
Dr Saxton: I can write to the Committee when it is appropriate to do so with the actual specification.
Ian Mearns: If you know that it is 20% fewer, you must know what the numbers are already, surely?
Dr Saxton: Any mistake in assessment materials is unacceptable and Ofqual continues to bear down on them and hold the boards to account, as you have seen with our recent Pearson case where they breach our conditions.
Q25 Ian Mearns: I am just surprised at your answer, Jo, because if you know that it is 20% fewer problems, right, you must know what the numbers are because otherwise you would not know it is 20% difference. So why can you not just tell us the numbers?
Dr Saxton: I think I am advised I am not to disclose national statistics at this point. I will write to the Committee with more specific information.
Q26 Chair: Why can you not disclose them? These things are not secret. I do not understand.
Dr Saxton: Perhaps my colleague—
Chair: I am happy for you to consult, but I am not quite clear what the problem is.
Ian Mearns: We are not talking about top secret information here, we are talking about statistics which really you have said a percentage fewer, so you must know the numbers. If you know the numbers, surely you can tell us, for goodness’ sake?
Dr Saxton: I would be happy to write to the Committee.
Q27 Ian Mearns: Okay. Well, could we have that prior to it being published, please?
Chair: Can you just explain why you cannot tell us now?
Dr Saxton: I genuinely cannot remember the statistic.
Chair: That is fine.
Dr Saxton: I know it was 20% fewer, but it is not a statistic I committed to memory because I was advised it was a national statistic and sensitive.
Q28 Ian Mearns: Okay. So last year it was 10 and this year it was eight or, you know: I am sorry.
Mike, what steps did you take to ensure that students' grades were not affected by the errors and what steps are you taking to ensure that errors like this do not happen again?
Mike Howells: Throughout the summer, before results day, as I mentioned, we are in deep dialogue with all of our schools and colleges who are facing challenges and may have gaps or incomplete information or mistakes in their data. So that level of care is something we provide to all schools and colleges. Obviously when it became clear on the 17th and then with specific cases on the 18th, specific students did not receive a grade that they had expected, we worked extremely intensively with schools and colleges to obtain any missing information to allow us to award a grade. As soon as we received that information, we could process them in real time. So, we were processing hundreds and, in some cases, thousands of new results in the course of a day when we received that information that day. We worked extremely hard with schools and colleges through that period.
As I mentioned, we have identified three process improvement areas that we want to work on in partnership with the rest of the sector. This is a complicated system with multiple different dependencies between stakeholders, so I think it is really important that, as a group, we work to create a system here that has the flexibility that students desire and that schools and colleges find very valuable but brings the level of clarity and transparency that the system obviously needs in future.
Q29 Ian Mearns: The errors which have been brought your attention, were those errors identified by an internal process or was it people flagging up errors that they had spotted in information that was sent to them?
Mike Howells: Whenever we have a request from a college or school for a grade for a student but we do not have the information to award that grade, we identify that issue and we take that issue up with the school and college. In some cases it is the fact that there might have been a mistake or some information is missing and we will work with the school and college to obtain that. When we have all of the information we, as an awarding body, need to award a qualification, we do so immediately. In some cases though, we do not have that information because the student has decided not to complete that year or to change the course or they may have left college.
One of the areas we need to improve on is in making sure that we truly know who those people are because, at the moment, the knowledge about that resides in the school and college naturally, they are the ones who know their students best, but it does create—
Q30 Ian Mearns: I am sorry, but in terms of errors which are included in advance information from yourselves, how were they addressed? Were they brought to your attention by outside people who received that information, or was that an internal process?
Mike Howells: There were no errors in Pearson advance information. That concerned other awarding organisations.
Q31 Ian Mearns: Okay. David, earlier on you mentioned that, because of the pandemic, students who were getting awarded grades this year had two years of interrupted learning, but we know that the levels of interruption to learning have been different in different parts of the country. There has been much greater loss of learning in some parts, particularly, for instance, the north-east of England. What is going to be done because next year's students will have had a year of interrupted learning? Are there any lessons that any of the exam boards or Ofqual have learnt from this year which are going to mitigate the disparity which exists around the country because of regional variations in the amount of lost learning? It is as much as over 15% of lost learning on average in the north-east, by comparison to 11% in London and the south-east.
Chair: Just to add to that, Ian, if you do not mind?
Ian Mearns: Not a problem.
Q32 Chair: Just looking at the figures, a 9.3% point gap between the north-east and London in terms of top grades in 2019, and that is now 10.2%. So in your answers, particularly Jo and David, it would be good to understand what your findings say about the GCSE attainment gap as well.
Ian Mearns: From my perspective of representing a constituency in the north-east, youngsters up there have got enough to put up with without having additional problems pitted against them by exam boards and grading structures.
Dr Saxton: It is an unfortunate fact that there are differences in outcomes in qualifications in different parts of the country and even within different centre types. Those differences existed prior to the pandemic. I know, certainly when I was a school leader, every other school leader and people across the sector are determined to see those gaps closed. That said, it is not the role of qualifications to mask inequalities, it is the role of qualifications to set out appropriate content for students to understand and to support teachers in teaching it. I have a statutory obligation to make sure that qualifications that are awarded are the most accurate measures of what students know, understand, and can do.
Of course, we put together the most generous package of support that has ever been seen to support students sitting those exams for the first time in 2022. One of the things I went out my way to do as chief regulator was to hear directly from students about what they wanted, including talking to students in the north-east. It was very interesting that, having received AI, what the majority of students told me was they did not really want advance information again, what they really wanted was to take their exams but they wanted to be in their classrooms with their teachers being taught specifications as their predecessors had been. We took that very much into account in our decisions in respect of 2023.
Q33 Ian Mearns: But surely, prior to this whole process, you will have known that there were differentials in the amount of lost learning on average in different parts of the country. Did you do anything at all to mitigate that?
Dr Saxton: We put in the most generous package of support, for example, allowing advance information—
Ian Mearns: But that was across the whole of the country, did you do any regional variation in that based on what you knew in terms of statistics on lost learning?
Dr Saxton: I can assure the Committee that the Ofqual academics and researchers spent many days, hours, and weeks looking at whether it was possible to do something bespoke, but our research found that even within the same centre, under the same roof in the same school, the impact of the pandemic was so varied that it would not be possible to do something that would be meaningful or fair. The reason that exams are fair is because the same rules apply to everyone. Advance information was the fairest way we could come to of giving everybody a revision aid. We required the boards to be significantly generous in their grading, given the context, but it is not the role of qualifications, unfortunate as it is, to mask the inequalities that exist in our society.
Q34 Ian Mearns: Well yes, those inequalities existed before the pandemic, but the pandemic has exacerbated the outcome. Given the fact that you had some fairly robust statistical information about on average lost learning, there was nothing at all that you could do to mitigate that in a national grading structure? Nothing?
Dr Saxton: The most important thing was that the qualifications are the fairest measure possible of what students know, understand, and can do. They have to have meaning and a national measure. That is why we put in place the package that included advance information, permitting the boards to publish that—this is something that previously they would have encountered significant fines for—and requiring them to be generous across the board with their grading approach. That was consistent with what students wanted.
Q35 Ian Mearns: Maybe I am being a bit sort of regionalistic in my attitude here, but that sounds to me like, “Well, I’m a national awarding body or a national organisation which oversees the awarding bodies, and it’s only them up there, they’re nearly in Scotland, they do not matter really”.
Dr Saxton: I can assure you, Ofqual colleagues spent days, hours, and weekends looking at whether it was possible to do something regionally, something by town, but it just would not have been fair and it just would not have been viable because within the same school some young people had successive isolations and some young people suffered disruption to teaching but were not out of school. There was just no way that it was nationally implementable. What students told me loud and clear was that they wanted their exams. They did not want another year of teacher assessment. And that is what we delivered for them.
In terms of the north-east, I think you are right that we saw an increase in the disadvantage in the A-level outcomes, but at GCSE, it is a huge testament to the teachers and leaders and support staff in the north-east, because in the GCSE results in 2022 we have seen the biggest increase, improvement, outside of London, in the schools and colleges of the north-east. As I say, that is a huge testament to the efforts of support staff, teaching staff, and leaders in that region.
Q36 Ian Mearns: David, in your perspective, apart from the pandemic, what do you think the key reasons are for the widening of the attainment gap between the north of England and London and the south-east?
David Hughes: The inequalities are massive, are they not? They are a national disgrace. The fact they got wider is really quite worrying. We know that there are lots of students who had massive disruption, a really big increase in mental health issues as well and anxiety. So you have got loads of students coming through the system who are not getting the support they need and Sir Kevan Collins put together a £15 billion proposal which was funded at a level of about 10%. I think it is going to carry on and what I fear, Ian, is that next year you will be asking the same question, “Why is this inequality gap so big? Why has it widened? Because you have got young people who did not have laptops, who did not have support at home, did not have somewhere quiet and warm to learn in, who have lost a big chunk of their learning, lost their confidence, they are more anxious, they are stressed, they have got bigger mental health problems, and it is not being addressed.
Q37 Ian Mearns: But historically there have been approaches which have been done on a regional or subregional basis, like London Challenge for instance, which actually have had a lasting effect. Do you think we should have a sort of London Challenge style thing for the north-east of England?
David Hughes: I think there is loads to learn from the London Challenge because it was led really strongly, was it not? It was a community effort. Schools coming together to get better.
Ian Mearns: It was only Tim Brighouse, what does he know?
David Hughes: But why do not we have a Tim Brighouse for the north-east?
Chair: That was what the opportunity areas were supposed to be about.
David Hughes: Yeah, but what they have not done is they have not put the degree of resource in to support the students adequately and the mental health issues, I think, is a really big problem. We have got school and college teachers and lecturers acting like social workers now because the mental health problems are so big and the NHS is so stretched. We have got a big problem that I do not think the Department for Education are recognising, and they are certainly not putting in the resource that is needed to rescue that group of learners who have really suffered.
Ian Mearns: And sadly the pandemic has not gone away. My own borough now, Gateshead, is at the very top of the league for the number of cases per thousand. It is pretty prevalent and still going on there.
Lost learning is still occurring in places like the north-east of England at a much greater measure than it is in other parts of the country and I really do think that Ofqual and the boards need to have another think about what is going to happen next year. The students next year will have had a year of lost learning out of the two on their course, and they are probably still being affected to some measure now in a disproportionate way.
Q38 Kim Johnson: My question is directed at Jo. Picking up on the education attainment gap, last year Ofqual undertook an analysis and I just wanted to know whether Ofqual will be undertaking the same level of analysis this year on the attainment gap and when will it be published?
Dr Saxton: Yes, we absolutely will. It was not something that Ofqual had done prior to the pandemic, but I think it is incredibly important and we will be doing that and we will be publishing it alongside the Department for Education’s statistics. I believe in November.
Q39 Kim Johnson: David just mentioned a number of factors that contribute to the widening of the education attainment gap. In your opinion, what are the key drivers? Do you think that poverty is a key driver in terms of the widening of the education attainment gap, given that children cannot learn on empty stomachs? What do you think the Government needs to do to reduce that?
Dr Saxton: As a former school leader, I completely agree with you on the importance of feeding students. In the schools that I ran we gave all students free breakfast to get them off to a good start. What the evidence shows is the importance of teaching. Again, when I have talked to students preparing for qualifications up and down the country this past year, they wanted to be in classrooms with their teachers. Quality teaching is the thing that makes the biggest difference and being in school and providing a good breakfast is definitely a brilliant way to encourage students to attend. Again, the evidence shows us that students who attend at the national average achieve passes and above national average in their qualifications.
Chair: From the breakfast?
Dr Saxton: Well, in my experience as a leader, breakfasts were a key driver in helping students attend. And when students attend at and above the national average, they tend to achieve their qualifications.
Q40 Chair: I work with Magic Breakfast club to try and get more breakfast clubs around schools and I have seen it in my own constituency where, exactly what you are saying, when they provide breakfast, especially for disadvantaged children, more children go in and they work harder in the day.
Dr Saxton: It makes such a difference.
Q41 Kim Johnson: Jo, just one final question. In terms of the analysis that you will be undertaking, is that broken down in terms of ethnic differences as well?
Dr Saxton: Yes. We will look at all inequalities in our analysis.
Q42 Kim Johnson: When did you say that was likely to be ready?
Dr Saxton: I understand it would be November. My colleague can confirm. Yes, November.
Kim Johnson: Thank you.
Q43 Chair: Very quickly on the private school issue. We know that the proportion of top grades fell this year at private schools compared with state schools, dropping from 70% in 2021 to 58% this year. And last year, there seemed to be a huge amount of grade inflation, particularly with private schools. Some people have suggested that this is evidence of private schools gaming the teacher-assessed grading system. What is your view about that?
Dr Saxton: As tempting as it is to make comparisons between 2021 results and 2022, it was a totally different form of assessment. That said, Ofqual takes all allegations of malpractice and cheating extremely seriously and we require the boards to investigate any credible evidence of malpractice and cheating. I know that there are also ongoing investigations.
Q44 Chair: Just as a general observation—and I understand it is different—given that the top grades in private schools fell significantly more compared to state schools, do you think there is a problem or not?
Dr Saxton: It is one of the reasons I am incredibly glad that we were able to reinstate exams. I think it proves that exams are the fairest form of assessment.
Chair: That is an answer generally, but I am talking specifically in terms of the private schools.
Dr Saxton: I suppose it is similar to the unfortunate issue that we see with regions. There were differences in results in independent schools and other school types that existed prior to the pandemic. Those results were exacerbated without exams and we have seen the results from 2022 are closer to those that existed prior to the pandemic.
Q45 Chair: Are you looking into this issue that people have raised about whether or not there was unfair grade inflation in private schools?
Dr Saxton: As I say, we take all allegations of malpractice extremely seriously and I know that there are live investigations which I cannot talk further about. We require the boards to undertake the investigations. Ofqual does not directly do them, but we monitor the investigations that the boards are doing.
Q46 Chair: Right. The boards are currently investigating some of the issues raised about private school grade inflation is what you are saying?
Dr Saxton: They are investigating individual cases of malpractice.
Chair: In private schools?
Dr Saxton: Yes.
Chair: Okay, thank you.
Q47 Angela Richardson: Thank you, Chair. Jo, thank you very much for your letter to the Chairman of our Committee on 29 September where you outlined arrangements for qualifications for next year. In that letter, you confirmed that they would return to pre-pandemic grading as the next step for getting back to normal and that this means that overall grades in 2023 will be lower than in 2022 to bring them in line with pre-pandemic grades. What assessment have you made of the impact on students, particularly disadvantaged students, of this change in approach?
Dr Saxton: Thank you. First of all, I set out a year ago that we would re-establish the tried, trusted and tested pre-pandemic grading arrangements but that, in the interest of students, we would do so in two steps. We delivered the first of those steps this summer. Because of the continued impact of the pandemic, which absolutely continues to cast a long shadow over schools, we followed through in our commitment about the second step for 2023 but we have put in place an additional protection, a protection that was deployed at the point that qualifications were reformed from 2017 onwards, to protect students from results being any lower than they were in 2019. In real terms, what that means is, should the quality of a students’ work be slightly lower than it had been due to the pandemic, they will not be penalised for that in grading terms. So that was the main mitigation that we have put in place.
We also urged the Secretary of State and the Department for Education to allow the continuation of exam aids, so revision equation and formula sheets, that they would be provided again in the exams because it benefits the majority of students across GCSEs when they are trying to grapple with so many subjects. I am very pleased that the Secretary of State supported that aid for students. And again, I heard from students about how that took a weight off their shoulders, so I am really pleased the Secretary of State agreed to allow that again.
Q48 Angela Richardson: You have talked a little bit about that protection that they have and a little bit about how it will work in terms of aids, et cetera, but what impact do you think that will have?
Dr Saxton: For all decisions Ofqual makes we undertake an equalities analysis. We worked really hard in this case, in respect of grading, to think about what was in the greatest interest of students. Throughout the year, we heard from teachers, but we also heard from UCAS and university admitting authorities, one of the things that was so difficult about requiring the boards to peg grading at an artificially higher point was that teachers did not actually know what that meant. It was not concrete. They were not able to refer back to previous papers, the resources that they had used over the years. In classroom terms, that means it was really difficult to say to a student, “This is the thing you need to do next to get some more points. I know that this is what you need to do”.
Equally, UCAS and universities have talked about finding it more difficult to be certain in their offer making and that would not be in the interest of students if we saw cautious offer-making for HE places again, as we saw in 2022. So, although it can feel counterintuitive, I genuinely believe that returning to the pre-pandemic arrangements for grading, with some protection for students, is the best way of empowering our nations’ teachers and leaders to support students and to advise them and hopefully to reward them with destination offers.
Q49 Angela Richardson: Do you see a point in the future where those protections will fall away, or do you anticipate that they will continue?
Dr Saxton: That is a very good question. I think it is too early to say. Again, in the same way that we made a refinement this year to the two-year decision we set out last September in recognition of the continued impact of the pandemic, we will, of course, look at whether anything else appropriate needs to be put in place. But, as I say, returning to the well-established pre-pandemic grading arrangements, I believe, is absolutely in the interest of students.
Q50 Miriam Cates: Jo, I just wanted to pick up on something you said about the need for grades to reflect a student's absolute ability or knowledge of a subject. It seems to me that in this approach we are in danger of losing that and looking instead for a kind of relative achievement, relative to the rest of the cohort.
Now I am not saying there is no merit in that, but actually looking at what happened in the pandemic, obviously we had this significant grade inflation, but that was not an across the board increase in grades—every student got half a grade higher than they otherwise would have. The fact was that some students who would normally have had a bad day on exam day and not met their teacher-assessed grade did not. Everybody who was given a teacher-assessed grade obviously got it. Would it not be better to write off the pandemic as an anomaly, awful as it was and learning the lessons as they were, but to move immediately back to the pre-pandemic situation and get realistic grades at the end of it?
Of course, that is difficult for students in terms of not getting as high grades as previous cohorts, but is it not a better reflection for universities, for employers, of absolute ability at the end of it? It sounds as though that is what students themselves are saying, that they would rather have a true measure.
Dr Saxton: I absolutely agree with you, and that is why we have made the decision that we have, but we are also putting in place a protection to students because if we did not, there is a strong possibility that results could be lower than they were in 2019. And, in the same way that in 2022 I know teachers and leaders in schools and colleges spent time thinking about how many more points this abstract grading arrangement might look like, I equally do not want people worrying about the opposite. Going back to what they knew occurred in 2019, knowing that there is what I have talked about as being a sort of soft landing for students is the fairest, most accurate way to move forward.
Q51 Miriam Cates: I see that from the point of view of the individual students, but the pandemic was catastrophic to students learning. Well, sorry, school closures were catastrophic to students learning. It did differently affect different pupils across the country. So, should grades not be lower than they were in 2019 because is that not an accurate reflection of how damaging the pandemic was?
Obviously from an individual student's point of view, there are problems with that. But given that each cohort is competing within its own cohort for university places, for apprenticeship places, for jobs, that is not actually a permanent problem once the student has got their university place, their first job, etc. So, from a national records point of view, would it not be better for the results to be a fair reflection of the damage that the pandemic has caused?
Dr Saxton: These results will be a fair reflection of what students know, understand, and can do in respect of any specification.
Q52 Miriam Cates: But you are saying they cannot be lower than 2019, whereas they should be lower than 2019?
Dr Saxton: We can put in place a protection to mitigate against that. But, of course, at this point no student has actually turned up and sat an exam.
Q53 Miriam Cates: So you do not know what is going to happen?
Dr Saxton: The important thing to remember as well is that the mark schemes, which control the number of points that students achieve and ultimately their grading, are published. Those are not being affected. So again, in the same way, students know that their work is what will return them the grade that they ultimately achieve, and admitting authorities and employers will also be able to trust that those are not being affected or changed in any way.
Q54 Miriam Cates: Would anybody else like to come in on any of those points?
Jill Duffy: Do you mind if I do just on your point? For going to university, they are almost exclusively, or not exclusively, but competing with their cohort. That is not the case when they go into jobs. So I think if we automatically deflated, if you like, that cohort, that could disadvantage them in later life. We need to be aware of that. And as other members of the Committee have said, there has been serious impact in terms of lost learning over the pandemic. It has been the most disruptive thing we have had in education and it is fair that we do take that into account in terms of arrangements for next year so that next year's cohort is not disadvantaged because they will still have suffered a lot of disruption from the pandemic.
Chair: Apsana. Welcome, by the way. Nice to see you.
Q55 Apsana Begum: Thank you, Chair. Good morning, panel. I just have a couple of questions around vocational technical qualifications.
Firstly, Mike, you said you will be undertaking a detailed review of this awarding series. What, in particular, will the review explore? I know you have mentioned three areas, but are you able to say more about when this review is able to be completed?
Similarly to Jill, as of 26 August I understand OCR said all results have been issued, but in the most recent update on 30 August we saw that there still appeared to be a small number of schools and colleges on complex cases where the necessary information to confirm student results is not currently available. Are you able to say anything more about what that small number is? And again, will you, like Pearson, be holding a review into the issue and, if so, when do you expect any review to complete?
Mike Howells: Our review will complete before the end of the year and I will share the findings of that with the Committee as well as with Ofqual. I mentioned the three themes that are emerging that we think improvements need to be made. We are looking at every aspect of the way the process works, but I think a really important point is in terms of putting in place reform and amendments to those processes. We want to do that in partnership with Ofqual, with schools and colleges and others, and fundamentally put the interests of students at the heart of this, because the way the system operates, it relies upon effective collaboration and coordination across multiple different stakeholders. We need to make sure this works ultimately for the most important people in this, who are the students.
Q56 Apsana Begum: Are you concerned about any of these stakeholders or the partnerships and making sure that you are able to implement the findings and the recommendations from that review? Are there any particular sort of stakeholders or partnerships that you think will actually be good to have better support or better working with them?
Mike Howells: I think, to be perfectly honest, the way that the partnerships work at the moment is very good, but I think we all recognise that there are some changes that need to be made and we are all committed to doing that. I can give you an undertaking that Pearson will absolutely put the resources behind it from our standpoint.
Q57 Apsana Begum: Thank you. And Jill?
Jill Duffy: We are conducting a review and aiming to complete that in November. We will be very happy to share the findings of that review with the Committee. It will be quite wide ranging. At the moment we are talking to a lot of schools and colleges and parents and students, I am doing some of those interviews myself, to get their perspective on what happened and what we could do differently in the future.
In terms of your question on the results, it has been a fluid situation. I asked my team yesterday, “Have we got any left?” We have one result left, but the school only contacted us on 29 September. So we have been dealing with schools contacting us, or us contacting schools, all the way through this. As I say, all the results for students with university places attached were released within a week for eligible students and we worked through the rest of them after that as well.
Q58 Apsana Begum: David, did you want to come in?
David Hughes: Yes. I am quite worried actually because I think it is great that OCR and Pearson are doing reviews, but they will probably come out with different findings. Most colleges deal with 30 to 40 awarding organisations. If they are all doing their own reviews, all doing different systems, all coming up with their different ideas, I think there is an absolutely priority task for Ofqual to work with the AOs to make sure we have a very united consistent approach to this that brings it all together. There is a massive efficiency to gain if the information required—when it is required; the format of the data; the census dates of when people have to say, “Yeah, these students are expecting results”—was all done in one way, and then the whole system would run much more smoothly.
There is a really important issue around all of the equalities data as well and it is very, very difficult to assess the equalities impact across all of the cohort of 16- to 19-year-olds, because there is not one data set for the whole cohort. There is really strong data on GCSEs and A-levels because there are only a few awarding bodies. There is really strong data on UCAS applicants because it is a finite group, but actually because there are so many awarding bodies, it is very difficult to link all of that with all of the vocational technical qualifications data. It is very difficult to link it to free school meals. It is very difficult to get all of the equalities data assessments. So I think there is a really big job for probably Ofqual and DfE to actually get hold of the whole cohort data and own it and understand what every student is doing. For some reason that does not exist. It is a kind of wild west.
When you say, “How many people are doing vocational technical qualifications each year?” We cannot tell you the answer because nobody knows. The equalities impact assessment is really difficult to do—pulling data from all sorts of different places rather than having one data set. So I think there is a really big systems change that is needed to get a hold on the whole data. In Scotland it is really easy. They can tell you because they know what every single student has done.
Q59 Apsana Begum: Thank you, David. That actually takes me on to my next question to Mike again. How can you ensure that vocational and technical qualifications are given the same importance as GCSE and A-levels?
Mike Howells: As far as Pearson is concerned, they are absolutely given the same importance. I have hundreds of staff who are very proud to work on this every day. As I mentioned, I think one of the great successes of BTECs and vocational qualifications more broadly has been the fact that they have allowed a new route into university. So one in four students in the UK—
Q60 Apsana Begum: They do not seem to have been treated very well though, in this awarding series.
Mike Howells: Well, I think they are. We are ultimately responsible for students and we treat them absolutely the same, making sure that they are successful is how we measure our success. Some of this has come out in the course of the discussion, there are some very significant inherent differences between the ways that vocational qualifications work though, and general qualifications like A-levels and GCSEs.
To the point about pandemic disruption, some of that flexibility has actually helped the vocational learners in some ways because I think the vocational qualifications system has been bit more resilient because coursework marking, for example, quality assured assessment by teachers, has been a factor of the way that vocational qualifications worked for a long time. And so switching to that model was slightly less disruptive in vocational qualifications than it was in GQ, even though of course the pandemic was disruptive across the board. But those differences are very valuable; that flexibility, the fact that the student can control their own destiny, make the right choices for them about what to study and when, is something that we need to maintain as we look at ways to improve the ways that the systems operate.
Q61 Apsana Begum: I have just got a more couple questions around the impact on students of the delays to the results, so is anyone able to say what work you have done to really understand the impact on the students? And particularly to David, you have highlighted concerns about the difficulties communicating with Pearson in your letter about the delays, saying that college staff were left to pick up the pieces without all the information to hand. What did you encounter, and have you seen any improvements since your letter in August?
It is really important for us in the Committee to be able to understand how many students have lost out on college and university places because of the delays as well.
David Hughes: I think the big lesson that everyone needs to take from this is that the communications need to be multi-layered. There was a breakdown of communication with college and school leaders about the scale of the problem. There was a confusion and an overwhelming amount of communication to exams officers, which I think was really difficult. And the timing of it was really tight—just the day before the results come out—which made it very difficult.
There was also a really important media and social media bit of communication that was missing around getting some communications out to students to make sure they understood before results day, because it was really clear there were problems before results day but there was no communication to the students who, as Mike says, are the ultimate customers of this system. So I think there are lots of lessons in that, and I think there needs to be some escalation points built into the system so that people understand what is happening much sooner.
Chair: In a nutshell, all of you, because we have got to get on.
Jill Duffy: Very quickly, I just want to say that we were absolutely communicating with students. I was personally dealing with calls and emails with lots of students and lots of parents. We were also putting out messages every day and we were putting those out on social media where we knew students were. So, we were absolutely communicating with students throughout this period.
Mike Howells: The same. Obviously we provided a lot of support directly to students. Generally speaking, it is a feature of the system though, that obviously schools and colleges have the direct relationship with students and the AOs support schools and colleges. So we stepped in and stood up a lot of additional support to make sure that students affected had those problems resolved.
To your question, I am not aware of anybody who met their grade requirements to get a university place who then did not go to their first choice university. There are a couple of examples where clearing was impacted in two cases where we assisted those individuals directly.
Apsana Begum: I think they started their university place six months later.
Mike Howells: That is true in one case, yes. And in the second case we engaged with the university directly on behalf of the student to explain the delay. They would not share with us what that ultimate outcome was, as they should not because it is a personal matter, but we made sure that the university absolutely understood that none of this was the student’s fault.
Q62 Chair: Can I just ask about something totally different now? It is predominantly to you, Jo, and to David, the issue of the health and science T-level, which is at a very low pass rate and regarded that it was inappropriate by many for what the students needed to learn. Who is responsible for that error? Jo, do you want to respond?
Dr Saxton: Yes. First of all, T-levels are a brilliant new qualification and I was thrilled to see almost 1,000 students graduate certificate with T-levels in other subjects this summer. In respect of the health and science T-level and the health route in particular, there were absolutely failings and those failings are the responsibility of the awarding organisation, NCFE, whose materials in two cases, two specific papers, did not meet the standard of the materials they had presented at the time of accreditation. They did not meet the institute standards and they did not meet the regulator’s requirements for compliance.
Q63 Chair: Is it right that the Department for Education signed off on the exam papers and that the exam questions were tested beforehand by dummy papers?
Dr Saxton: The Department for Education oversees the whole of the qualification and how it aggregates. One of the things that is very difficult about exam materials is that nobody outside of the award organisations ever see them before they are sat, the actual ones that students will receive and get their grades on, to avoid all of the issues with leaks and security that have, in some series, plagued our exams system.
Q64 Chair: DfE signed off on this exam paper, is that correct? And they were tested by dummy papers?
Dr Saxton: They signed off on model papers, on dummy papers, not these specific ones. So the quality of the actual papers that were put in front of students in this summer series did not match the quality of those in the dummy papers at the point of accreditation.
Q65 Chair: So where does the buck stop? Does it stop, in regulatory terms, with the DfE, the exam board, with you?
Dr Saxton: It is absolutely NCFE’s failing and they have put their hands up and said that they have also written letters apologising to affected students and their teachers, and we are holding them to account. We also have assurances from them. In fact, they have published a written undertaking just at the end of last week that they will not make the same errors in the autumn series.
Chair: As you can see by the session we have had this morning, I am not someone who is an apologist for exam boards, but I do not understand why, in this particular case, it is just the exam board if it is signed off by the Department for Education.
Dr Saxton: Not those specific papers though, unfortunately, Chair. They signed off on proxy papers.
Chair: You said there were dummy papers, which had been very similar.
Dr Saxton: They are similar but not the same. And unfortunately there were assessment materials—
Q66 Chair: How different were they? How different were the dummy papers?
Dr Saxton: Every single question has to be totally different otherwise there could be leaks which would be unfair.
Q67 Chair: I am huge believer in T-levels, as you could imagine, but are these qualifications not being regulated to the same standard as A-levels? Again, going back to our earlier discussion, would this have happened with A-levels? Well, actually there have been some mistakes by exam boards on questions but still, you do not have a failure pass rate and so on in the way that we have seen.
Dr Saxton: It is important to remember it is a new qualification. As I said, 1,000 students have certificated without incidents. There absolutely have been issues with the health and science and NCFE hold their hands up to the assessment material mistakes they have made.
In terms of your question, Chair, about are they being held to account to the same degree? I think that the boards would argue that they are feeling more regulation in respect of T-levels because they have to work with the institute, with the Department for Education and, ultimately, with us as their regulator. And we are already holding them robustly to account for their failures.
Q68 Chair: What has happened to the students who did these exams?
Dr Saxton: We acted in the interest of students to ensure that they had clarity and the ability to continue their studies. We determined that the results from the NCFE papers were void and we asked the board to award them their first year grade based entirely on their employer-set project, which our review found to be a task of integrity.
Q69 Chair: They did that exam for nothing? All the stress and everything?
Dr Saxton: Unfortunately, yes.
Q70 Chair: To get awarded an assessed grade, in essence. Is that what you are saying?
Dr Saxton: No, because it was also an examined element. The employer-set grade was externally set and externally marked. I spoke to some students who have been through exactly this experience just last week. Bless them, they said that they were still very glad to be doing the course, they are incredibly excited about what they are learning on it. Their lecturer was talking about how they are doing things as 16-year-olds that, in many other settings, are what happens in a first year of an undergraduate medical degree. So the students have absolutely suffered but, certainly those that I met last week, are undeterred.
Q71 Chair: You say everyone did everything perfectly except the exam board. So the dummy papers, everything, DfE are not responsible, Ofqual are not responsible, it is just the exam board?
Dr Saxton: We set very clear guidance that the exam board has to measure up to the institute. Ofqual and the Department all scrutinised proxy materials. Unfortunately, the real materials did not live up to the expected standards.
Chair: You have not said how different the real materials were from the dummy ones.
Dr Saxton: Again, we will be publishing our regulatory action in due course, but they were significantly different.
Q72 Chair: They were significantly different from what was presented to both IfATE, Ofqual and the DfE?
Dr Saxton: Yes, there were a number of questions that did not appropriately match the specification and if we had discounted those questions there were not enough valid questions left upon which to award a grade. So I took the unprecedented step of saying that those results had to be discounted. We were able to protect students because they had this other examined unit.
Q73 Chair: Just so I understand this, how is it that an exam board can present dummy examples, which in essence is a drafting, and then produce something significantly different? What is the point of presenting the dummy papers in the first place if the paper is going to be different?
Dr Saxton: The aim of what is called the accreditation process is in the example of a new qualification with new materials and, in this case, an awarding organisation who had not done assessment materials in this subject before had to demonstrate to us as regulator that they had sufficient assessment expertise and capacity to be able to deliver materials of the appropriate standard. They did so at the point of accreditation, but their actual materials failed to live up to that standard.
Q74 Chair: Do you think there were too many cooks looking at these qualifications? You have got IfATE, you, the DfE, the exam board; should it just be one of you doing this?
Dr Saxton: Each of us has a different role. I think it is so important that employers are involved and IfATE absolutely channels that and they have worked so hard to understand what employers want, and that is the thing that will give students opportunity. Ofqual, we are assessment experts. And the Department for Education, I think it is noble that ministers are so closely involved in wanting to see this new qualification delivered. We do a very different thing. Ofqual, we regulate the awarding organisation, we will robustly hold them to account, and the institute represents the employer voice. They also procure the qualification providers and they contract with them.
Q75 Chair: I get all that. I know what they do. I am not surprised that you have had a problem like this because you have got so many cooks all putting in their own ingredients when you could probably just have one regulator doing the job.
Dr Saxton: As far as I am concerned, there is only one regulator in respect of T-levels.
David Hughes: Yeah, there are too many cooks. It is very unclear who holds the ring on this. Again, we had to step in to say to Ofqual, DfE, IfATE and NCFE, there is a problem here because colleges were telling us what the results were and there were lots of students getting U, very large numbers getting U, who were good students who would not have got U on other courses. It took time for those four organisations to come together to resolve what was needed, and in that time there were students, in excess of 1,000 students, saying, “I’ve just wasted a year of my life”. So there is a really important lesson in how the governance of this works and who ultimately takes responsibility.
It shows the risks of new qualifications and it shows the risks of defunding existing qualifications that OCR and Pearson deliver too quickly because this is not the only time this is going to happen. I understand why Jo is saying this is an NCFE issue. It is clear NCFE made mistakes, but there is a massive issue around the content as well. IfATE are doing a content review because it is really clear to us that what is expected of a student after only one year to reach a Level 3 in the breadth of what they have been asked to cover is just unreasonable and we have got to get responsibility for that as well.
What we found is that lots of students are okay with the way forward, using their employer assessed grade, but actually there are lots who are not. Lots have moved on to other courses, on to BTECs and BTEC Nationals. The students who were going to start this September have not because, mainly through social media, they found out what the problems are. So the whole T-level reputation is at risk in this.
We have got to have another look at the whole governance around T-levels and who owns them, who is responsible, how you get the level right, how you get the content right, because actually we are in danger of setting up qualifications in some areas that lots of students will really struggle in. If they defund all the BTECs and BTEC Nationals as well, there will be nothing left. So there are big risks in all of this.
Q76 Chair: Do you agree with what Ofqual are saying, that this particular case is just the fault of the exam board?
David Hughes: No. It is really clear NCFE made mistakes and I think they have owned up to those mistakes and they have been very straight about it. But the fact that IfATE is doing a content review, they are looking at the content particularly of that paper, what they should find is that they are expecting enormous breadth of a 17-year-old after one year, which is just not realistic. It is not really an Ofqual problem because they look at the assessment of what is being assessed. It is a content issue. And DfE are sitting there, I am not quite sure what their role is in that. I am worried that this is not the only T-level where this will happen. That there is an expectation from employers that young people will be all-singing and all-dancing across a whole range of things, which I just do not think are reasonable.
Dr Saxton: Chair, could I say two things? First of all, I cannot speak for IfATE or the Department, but I know that my colleagues in both those organisations are absolutely committed to getting T-levels right and we must remember this is a new qualification. I think we should welcome the fact that they are undertaking a review. This is content which has come directly from employers to help young people have employment opportunities they have not otherwise had. None of us would criticise high expectations. Of course, we have got to get it right for students.
In terms of whether or not Ofqual were slow to act, David might not be aware, but as soon as we were aware on results day that there were some students who were unsatisfied, my team were talking to students, we were talking with leaders, and in terms of any sense of a delay of giving students clarity, I hold my hands up because I wanted to know absolutely whether these papers had integrity. I would not allow new grades to just be awarded by the exam board until we could be absolutely clear what the evidence was, because I could not have students get another grade and then we find out that there were other mistakes. As it turned out, unprecedentedly, both papers had to be discounted. That has never happened before. But I stand by taking those days to get that right, so that when we were able to tell students what was happening it was a consistent, clear message that was not going to change.
David Hughes: That is right, but my point was different. That it was not just the problems with the NCFE papers, it was the content as well. And trying to get a communication across the four organisations involved was very, very difficult. And in that meantime, there were students not knowing what was happening. We have just got to understand what it looks like from a student's perspective when there is no information coming to them, the college or school are in the firing line trying to understand what is happening, and we have just got to get better at that bit.
Chair: Okay, thank you. I am going to bring in my colleague, Ian, and then we will wrap up.
Q77 Ian Mearns: On the subject that has just been covered by Rob, how many students were affected by this situation with NCFE?
Dr Saxton: I think it is 300.
Q78 Ian Mearns: It was 300? Can we get some clarity on that, please?
The other question I would have is, it is all very well for students having a clarity and an understanding of what has gone wrong, what they really need though is that time back. What recompense are they going to get for a failure of the system which has let them down really rather badly?
Dr Saxton: Mr Mearns, I apologise, it is 1,000 students who were affected, not 300. We have worked really hard, again with the institute and the Department, talking to college leaders and also directly with students about what is the right thing to do next. That is why we have, in effect, put in place a kind of no-detriment approach. They could have their employer-set grade. If they are not happy with that, there will be another assessment opportunity in the autumn. Where it has been the student preference, again, with huge help from AoC, they have been supported in transferring into other qualifications. But again, the students that I spoke to recently are determined to continue with it because they can see the employment opportunities that it is going to provide them that they have not had from other qualifications.
Q79 Ian Mearns: Could we have a systematic detailed review of outcomes for students from this, please? I think that is required so that all of the bodies involved actually know that there is going to be some significant scrutiny of them in the way in which they handle this for those 1,000 students who have been let down by systematic failure.
Dr Saxton: I think that is already in hand.
Q80 Ian Mearns: Well, I hope you are right. And if it is not already in hand, can it be put in hand and delivered so that this Committee can be assured that those students are not being further let down after this systemic failure?
Chair: And you definitely think it is right that this kind of spaghetti junction of regulatory boards looking at T-levels, whether it is DfE, you, IfATE, is the right way forward and that it could not be streamlined?
Ian Mearns: Somebody needs to put their foot on the ball, Rob, do they not, to take control.
Chair: You say you have slightly different roles, but surely it would be better for just one of you, I do not mind which one it is, to be entirely responsible.
Dr Saxton: I look at it as the different roles and the different experiences that we all bring. I can hold NCFE to account. The institute can bring the expertise and the connection with employers, and they can work closely with the awarding organisations.
Q81 Chair: But that is about the qualification itself, not about the results and the exam papers.
Dr Saxton: This is a case where you can see the regulator took really swift action. We acted in the interest of students within a week.
Q82 Chair: I am not saying you did not. Clearly, from what you have said, you acted really well. I am not criticising Ofqual. What I am just saying is, I am asking your view, and you seem to be reluctant to give it, of whether or not there should be one body overseeing all this, rather than the hotchpotch that we have got at the minute.
Dr Saxton: In general, my view is that many minds are better than one and I think there is real expertise, interest and support in delivering this pioneering qualification. For 70 years, people in our country have tried to find a solution that would help young people have opportunities into a skilled employment opportunity, and we have seen through those first 1,000 students it is really beginning to be delivered. I welcome the great minds in both the institute and the Department working with Ofqual to help students have that opportunity.
David Hughes: I think Jo is in a difficult position because their role is very clear. This is a real fantastic case of people pointing the finger at others and saying that they are the problem and NCFE, sadly, are in the firing line. They definitely made mistakes and the Ofqual intervention was right. But I keep coming back to this; it is not just about the mistakes. It is much bigger than that and nobody is owning that. IfATE are doing a review and DfE, I am not sure are owning that. It seems to me DfE needs to own this. They are pushing forward on T-levels. We are big supporters of T-levels: I think they are fantastic opportunities for young people, but we have got to get the content right. We have to make them realistic for the learners who do them.
If you look at the average grade of GCSE scores of A-level students, it is over six. If you look at the average grade of T-level students, it is about 4.7. The standard of students coming in, in the sort of assessment that T-levels do, written assessments is lower than A-levels. So you have to understand that when you are setting the standards. You cannot make them unrealistic if they are going to be for the whole cohort.
Q83 Chair: Just very quickly, what you said about T-levels being a major fundamental reform, I agree, but actually I also think apprenticeships alongside that are biggest change for post-16 education.
Dr Saxton: I have talked to plenty T-level students who are going into an apprenticeship following it, so it is opening those doors too.
Q84 Ian Mearns: Jo, if you do not mind me saying, I absolutely concur with you about great minds coming together to come up with some great ideas, but it only works if somebody takes ultimate responsibility. And if somebody is going to take ultimate responsibility for that it is probably going to be the DfE. A Minister might appear in front of us to see what it is, from that perspective, but somebody has got to take ultimate responsibility, otherwise it is all this finger pointing that David referred to.
Mike, you have recently been served a record-breaking fine of £1.35 million for what Ofqual described as, “Serious breaches which have the potential to undermine public confidence”, which occurred between 2016 and 2019. So, why did these breaches occur and what steps have you taken to ensure that they are not repeated?
Mike Howells: This is an example of regulation working. These were two errors that Pearson identified and shared proactively with Ofqual. We have worked together to address the underlying causes and make sure that they have been remediated already. Some of the enforcement action was paused during the pandemic so that both organisations could focus on the support we needed to provide to students and has now been implemented this year, as you have mentioned. But these are mistakes that were addressed historically and proactively communicated by Pearson.
Q85 Ian Mearns: The problems relate to 2016, 2017, 2018, 2019, that is all pre-pandemic, is it not?
Mike Howells: That is correct.
Q86 Ian Mearns: Right. And, Jo, from your perspective, given the fact that these errors occurred between 2016 and 2019, it is now 2022, does the regulator need to act a little bit more swiftly in this? If this is about identifying problems and rectifying those problems so that they do not impact on future students, surely between the original errors taking place six years ago in 2016, is there not some cause now for getting this process quickened up somewhat?
Dr Saxton: I completely agree. One of the aggravating elements of this case is that Ofqual wrote to Pearson about a finding of a similar issue with another awarding body, I believe back in 2017, and at that time Pearson said that it was not something that was an issue for them. They did, as Mr Howell said, later come forward saying that actually it was, so that was an aggravating factor in this case.
As soon as Pearson came forward, the first thing Ofqual did, again, to ensure the protection of students and public confidence in the awarding body and the qualifications it awarded, sought Pearson to undertake certain actions, including a review to make sure that students were not affected. We are confident in their assurances that this is not something that continues and has not been occurring since 2019. However, it was decided that, because of the pandemic, the enforcement action would be paused because we were confident that students were protected. And while many things went on behind the scenes, that information has not been published until after the pandemic and in particular until this season's exam results were published to students.
Q87 Ian Mearns: How satisfied are you that Pearson continues to fulfil its obligations and should continue to be recognised as an awarding body, or are they currently irreplaceable?
Dr Saxton: I am satisfied that they are compliant with our conditions.
Q88 Ian Mearns: If necessary, could Pearson be replaced?
Dr Saxton: It would not be for Ofqual to replace them, but I am satisfied that they awarded almost 2 million qualifications this season and the vast majority of those without incident or error and compliant with our conditions.
Q89 Ian Mearns: What is the proportion of exams overall that Pearson sets?
Dr Saxton: They have a significant market share. I am afraid I cannot remember the exact proportion of it. They have the largest market share of vocational qualifications. AQA has the largest market share at GCSE.
Q90 Ian Mearns: So the fact that they have a dominant position within the market does not affect your judgment over whether they should be replaced or not?
Dr Saxton: No, I think what we see is the sector choosing, they choose between the exam boards, where they like specifications, the support they receive from exam boards. It is very much a market that we regulate and Pearson have an active part in that market.
Q91 Ian Mearns: And the fact that the regulator reaction that you have taken in terms of imposing a significant fine, has that also brought with it, from your perspective, a closer scrutiny of Pearson as an awarding body?
Dr Saxton: Pearson is one of a number of exam boards who we scrutinise incredibly closely to uphold our statutory objective in public confidence. We are not doing any additional things that we did not already do. Again, our conditions, one of the things we require is the boards to report through to us. We anticipate risks. As I say, in this case, we asked Pearson about it. When they realised that they were undertaking this mistake, they came forward to us, so I am confident that our scrutiny is proportionate and appropriate.
Q92 Ian Mearns: Was it better now than it was between 2016 and 2019 when these errors occurred?
Dr Saxton: I have assurances from Pearson that this does not occur anymore.
Ian Mearns: I am not talking about Pearson, I am talking about the scrutiny that you impose on Pearson. Is it better now than it was between 2016 and 2019?
Dr Saxton: Our scrutiny is proportionate and appropriate.
Ian Mearns: Okay. Thank you very much.
Q93 Chair: I think it would be helpful to the public, I know you have set it out in the letter but for those watching, in terms of your proposals for exams next year and also just adding in terms of how those students who continue to miss school or are missing school, the absent children, how they are going to be helped in this. It would be quite useful for the public just to hear it first-hand.
Dr Saxton: For us to continue to explain our decision, is that what you mean, Chair?
Chair: In terms of what you are going to do for exams next year and how it is going to make sure that disadvantaged students are not left further behind, in essence.
Dr Saxton: Yeah, absolutely. We heard from key stakeholders, school or college leader representatives, UCAS, universities, employers and, again, from the mental health experts that we have worked with, the unifying factor for all stakeholders was clarity and that is why we worked very close to results to provide clarity for 2023 arrangements as early as we could this academic year. And that is partly also why we set out a two-stage decision last September, conscious that young people would be starting, in general, two-year programmes of study. We wanted them to start those programmes of study with a likely sense of how they would be graded at the end. So, providing clarity is something that is really important to me, and you can see that in the actions we took in this decision. Having published this decision, we are just absolutely at the beginning of continuing to explain it and we will continue to do that throughout the year.
Q94 Chair: You are going back to 2019 level exam grades, in essence, across the board?
Dr Saxton: Yes.
Q95 Chair: Okay. You do not need to respond to this, but my first ever speech in the House of Commons was about apprenticeships and I care deeply about this subject. Everybody always says we want parity of esteem every time, and then what happens is there is one problem after another that happens that undermines that parity of esteem. I just hope very much that the exam results for vocational students will be second to none. All of them, not just a few thousand missing out. They should have a Rolls-Royce service. I say that to the exam boards and to Ofqual, and that we will not be here again saying there is going to be another review.
Dr Saxton: Any student who is legitimately expecting their result on a certain date must get it. It is unacceptable for them not to. But I am conscious that we must not throw any babies out with the bath water in that there are flexibilities with vocational technical qualifications, which in very many cases meet the interests of students who could draw them down when they need to.
I need to gather all of the evidence, work out what has gone wrong and be clear about which things we can do which will make a difference. I think of the exams office entire team that I met in Blackpool Sixth, who are dealing, exactly as David said, with 40 awarding organisations. Everything is on the table in our review. We can look at whether consistent guidance around awarding would help, but we have got to make sure that we do not do things which cause other problems or perverse incentives. And that is why I am not able to share findings just yet.
Q96 Chair: I do not know if any of you want to comment, hopefully you are going to provide a Rolls-Royce service next year for all the students. Is that the case?
Jill Duffy: Yes, absolutely. As I say, from some of the things that we have already identified, we will be looking to improve data sharing, improve our comms so that everyone in school or college gets them, and more than happy to work across boards and with Ofqual to get greater consistency to relieve that pressure that David has talked about.
David Hughes: It is great that you have called us here because I think it has helped put the pressure on to make sure that next year is better. The consistency is really important and I am quite certain that the three people on this panel with me want to make it work better next year. So, with a fair wind, it should work better next year.
Mike Howells: Likewise, I very much welcome the opportunity to have this conversation and hear the Committee's questions and your interest in this. Accountability is a vastly important part of how we operate and we need to make sure that fundamentally the interests of students are protected here. As a sector, we have some work to do to maintain the value and the differential approach that vocation qualifications bring that Jo mentions but, as David says, in a way that reduces some of the complexity, particularly for schools and colleges, so they can provide a better level of service going forward.
Chair: We have a final question from Ian.
Q97 Ian Mearns: Sorry, I just wanted to take the opportunity since Jo is with us. Jo, in September 2020, your predecessor, Dame Glenys Stacey, when referring to exam grades which had taken place before the lockdown, told this Committee that, “Exam grades are reliable to one grade either way”. So were this summer's grades also reliable to one grade either way and has anything changed?
Dr Saxton: I cannot speak for my esteemed predecessor. What I can assure the Committee is that students, universities and employers can be confident in the grades awarded by exam boards this summer. Any assertion that exam grades are wrong is not borne out by the evidence. It comes from a misrepresentation of Ofqual’s research. And again, if you look at the analysis of reviews of marking, very few grades change when marking is reviewed. This year, the boards saw fewer appeals and results changing, which is a key piece of evidence that exam grades are right and the public can have confidence in them.
Q98 Ian Mearns: You think that the grades are much more reliable now than they were when Dame Glenys was in front of us in 2020?
Dr Saxton: As I say, I cannot put words in my predecessor's mouth, but I can assure the Committee and the public that they can be confident in the exam grades that have been awarded in the summer of 2022.
Ian Mearns: Thank you very much.
Chair: Thank you very much. Ian highlighted that the purpose for us is trying to stop it happening again. That is why we have done this, but I do appreciate you giving evidence and coming to the Committee, and I thank you for all the work that your perspective organisations and your staffs do. Thank you.