16
Examination of Witnesses
Dr Jo Saxton and Sir Ian Bauckham.
Q128 The Chair: Good afternoon and welcome to this session on education for 11 to 16 year-olds. Thank you for coming before us. There will be a transcript and opportunities to correct it, should that be necessary. Can I ask you very briefly to introduce yourselves?
Dr Jo Saxton: I am chief regulator at Ofqual.
Sir Ian Bauckham: Good afternoon. I am the chair of Ofqual.
Q129 Baroness Evans of Bowes Park: Your corporate plan currently lists shaping the future of assessment and qualifications as one of your priorities, so we would be very interested to hear a bit more about what work you are undertaking in support of this priority, focusing in particular on 11-16, which is the area we are particularly interested in. Perhaps you could just unpack that priority for us a bit more.
Dr Jo Saxton: The future of assessment is a very big subject. As regulator, it is really important that we set the guardrails so that students can benefit from innovation where it is available. We also have to make sure that it can be done at scale, so that it is fair for all. There are questions about national infrastructure and various underpinning factors.
On the specific work that we are doing, we have an ongoing feasibility study to look at things like the national infrastructure and what would be needed for high-stakes qualifications such as GCSEs to be delivered on-screen. We are also looking at the potential for digital or modern technologies, for example, to provide additional quality assurance for matters like marking. That is probably the best overview I can give.
Sir Ian Bauckham: In schools, there is a lot of interest in technology and how it might be applied to assessment. As well as looking at the potential of the technology itself to assist us with assessment, we have some bigger infrastructure questions to think about. In other words, is what we could design in theory likely to be practically deliverable for schools and colleges that are offering exams and assessments for students, given the state of IT infrastructure that exists? It is just about deliverability as well as the key points that Jo outlined.
Q130 Baroness Evans of Bowes Park: We asked this last week of Amanda Spielman, who is running Ofsted. As the chief regulator, who are you personally accountable to for the work that you do?
Dr Jo Saxton: I am personally accountable to Parliament for the work that I do, not to Ministers, and I am very conscious of holding students as my compass. My focus since I have come to the post has been to make sure that we tilt Ofqual’s powers, where possible, towards the interests of students.
Baroness Evans of Bowes Park: Could you expand on that a little? Do you have any tangible things or initiatives you have started that you can point to?
Dr Jo Saxton: There is a spectrum of things that we do, involving a huge range of visits. I did over 100 visits last academic year. We listen very carefully to students, but where there are difficult choices, we will think about the one that is in the interests of students. I had to take the unprecedented step last summer of setting aside a series of qualification results in a new qualification, because they could not provide reliable and valid results for students.
Sir Ian Bauckham: Perhaps it is also worth mentioning that, in the journey back from the exceptional arrangements for the awarding of qualifications during the pandemic, the perceptions of how we are going to proceed and the impact on students have been a central consideration for us in determining how quickly to go and what point to reach.
Dr Jo Saxton: I wonder if I might add a bit more to that. Particularly in terms of our approach to grading, with the summer in mind, we talked to employers and universities, and universities in particular were very clear that they had found it difficult to be generous in their offer-making to students when there was an abstract approach to grading. If we were able to reinstate pre-pandemic grading arrangements, it would support them in being more confident in their offer-making to students. So although it might seem counterintuitive, we are very clear that it is directly in the interests of students.
Q131 Baroness Garden of Frognal: What are the strengths and weaknesses of non-exam assessments? I used to work for City & Guilds, and for vocational qualifications a lot of the assessment would be observation, project work or coursework, as well as oral presentations. What would be the implications of reintroducing those forms of assessment for GCSE subjects, which are now so predominantly assessed by written exams?
Dr Jo Saxton: That is a really good question, and you are right that there is a huge amount of non-examined and observational assessment in the vocational and technical qualification space. Of course, there are still elements of it in general qualifications.
As chief regulator, my job is very much to make sure that the Government of the day, whose decisions these are, understand the trade-offs and the choices that they would be making, but I am very mindful that, prior to the reforms of the current GCSEs, there was the very high burden on schools of the non-examined assessment. Students in particular could in some cases do up to 100 hours of non-examined assessment, which teachers in many cases found very difficult. As regulator, we saw lots of problems with highly scaffolded answers, particularly when the qualifications are high-stakes. In theory, all these forms of assessment are possible, but it comes down to the choices of the Government of the day and what the qualifications are being used for.
In the context of GCSEs and A-levels, where you want to have a good sense of what the student’s work is, history has shown us that high degrees of non-examined assessment end up being very scaffolded by staff and not necessarily the best reflection of what students understand and know and can do.
Sir Ian Bauckham: If I can just add to that, it was also because of the sheer scale of them. My colleague referenced 100 hours of non-examined assessments in the previous generation of GCSEs. They were generally known as controlled assessments that were done in class. Given that volume, they consumed very large amounts of time in school that might otherwise have been used for teaching. By removing that and tipping the balance more in the direction of exams, teaching time has been freed up.
Baroness Garden of Frognal: A lot of young people respond very much better to doing something practical and being watched doing that than writing endless exams, but I can see that takes a lot more time.
Q132 Lord Knight of Weymouth: I will start slightly off-piste and then go on-piste again. What positives and negatives did you learn from the experience of centre-based, teacher-based assessment during the pandemic? Clearly, you are accountable to Parliament as a non-ministerial government department. You have a remit, as I understand it, from the Secretary of State. That points towards grade inflation and a real political concern about that. Clearly, it was a consequence of what happened during the pandemic, which created some grade inflation which you have now had to claw back. What are your reflections on all of that, please?
Dr Jo Saxton: I was not in post as chief regulator at that point. However, I came to post immediately having to make the decision about the reinstatement of exams and what the grading approach should be. I set off to talk to as many students as I could, particularly in schools and colleges serving disadvantaged students. I was overwhelmed by the extent to which students wanted their exams reinstated. It was quite extraordinary. I can think of only one school or college where the students did not have that view.
If I paraphrase the view of students, it was that they respected and loved their teachers, but they did not want them grading them. They felt trust in the examined system, where there are expert markers. Our research shows that most examiners have over 10 years’ experience. Students wanted that independence and the reassurance of that expertise, so a very clear lesson that came out of the pandemic was that students wanted their exams reinstated, which was probably quite a surprising finding.
I have heard from very many teachers and school staff who had to administer alternative approaches to assessment during the pandemic. They have talked about how they found the workload almost unbearable, but, out the other side of it, so many of them said to me that it was fantastic continuing professional development and that they learned so much more about assessment and awarding through it.
Sir Ian Bauckham: I was in post during the second bit of the pandemic, when I became interim chair of Ofqual in January 2021. From the point when examinations were cancelled at the beginning of January that year, through to the beginning of the exams season—or the non-exam season, in this case—in May that year, we had time to try to design the most watertight approach to grading that could be devised, using work that was being produced under as controlled conditions as it was possible to design during a pandemic in schools.
Despite the very best efforts of the best minds and the most experienced people in this space, it was very hard indeed to curb further upward drift in grading. That is because, when a teacher is asked in effect to mark and grade the work of the student they are accountable for teaching, the pressures for upward drift are very hard to resist. It is not an easy thing to do at scale.
Lord Knight of Weymouth: Before I go on-piste, we had a very interesting evidence session with the three witnesses before you, and we discussed a little the extent to which there may be something that looks a bit like norm referencing in GCSEs at the moment, or the extent to which there are elements of criteria-based judgments in the existing GCSEs. As a result of that, I have a question for you. What is wrong with grade inflation if that is showing that things are getting better? Does it worry you if the Prime Minister says, “It’s terrible that a third of kids aren’t getting to a certain grade”, when you have to regulate on the basis that a bell curve is maintained and a certain number of people are always going to be at the wrong end of grade 4 and 5?
Dr Jo Saxton: I am so glad that you asked this, because it is such a fascinating subject and there are so many myths. It is one of the most frequent conversations I have on my visits. I will answer in more detail, but if you could just allow me to be clear, GCSEs and A-levels are not norm referenced and not criterion-referenced, and there is no quota of grades. What really matters is that the underlying performance reflects the grades. We have a fantastic tool called the national reference test, which is an independent longitudinal measure that helps us to determine whether the underlying performance standard of 16 year-olds in this country in English and maths is changing.
Lord Knight of Weymouth: Are those outcomes published?
Dr Jo Saxton: Yes, they are.
Sir Ian Bauckham: Yes, every autumn.
Dr Jo Saxton: Ofqual can then use them to direct exam boards to change their approach to grading if necessary. We have this independent measure of where underlying performance standards are. The problem with grade inflation is not about politics, for me. It is about students’ qualifications continuing to hold value through the perpetuity of their life. If performance improves and students do better, there should be more top grades, and in the system that I am chief regulator of there can be.
Lord Knight of Weymouth: So there would be fewer lower grades.
Dr Jo Saxton: Yes, exactly.
Sir Ian Bauckham: By definition, yes.
Dr Jo Saxton: There are no barriers to that. People assume that the bell curve is a quota, and because the bell curve looks similar in successive years, they assume that it has been retrofitted on to the results, but grades are a direct consequence of the marks that students achieve. One of the great things about GCSEs and A-levels is that they are compensatory. There are many ways in which you can gain marks through a script. If you overlay the summer bell curves with the autumn resit bell curves, you can see that there is not a quota, because it looks completely different. I would be delighted to write to the committee and share that.
Lord Knight of Weymouth: That is really helpful.
The Chair: I have to say that I am a bit confused, given the evidence we heard from our previous session, in particular from Rethinking Assessment, which said very explicitly that this was the system that generated a third of students getting grades 4 or below. You are contradicting that quite categorically.
Lord Baker of Dorking: They were right and you are wrong.
The Chair: Can I just press you on that point?
Dr Jo Saxton: There is absolutely no quota and there is no requirement that a certain proportion of students must fail. I have spent a decade teaching in and leading schools, including very successful independent schools and schools serving disadvantaged communities. I would encourage the committee to look at the exam scripts. Look at the papers of students who are achieving grades 1 and 2, which, by the way, is achieving a GCSE. It is really heart-breaking when you see what they are not able to do. In some cases, they are struggling even to write their own name. It is not the GCSE that is failing students. The GCSEs are simply a measure.
The Chair: You made the point that it is possible under your system for the proportion of students who get good grades to increase and, therefore, the proportion who get poorer grades to shrink over time. How is that compatible with the idea of comparable outcomes from one year to the next, which suggests a certain freezing in the boundaries and in the proportions who are getting particular grades?
Dr Jo Saxton: It is a myth that comparable outcomes are a barrier. I have read the academic literature by those who created the process. The name is unhelpful, but, at its heart, it is a mechanism to make sure that grades can be compared from one year to the next. There is nothing in the comparable outcomes approach that determines that anybody should fail or be capped.
Sir Ian Bauckham: It is about holding the relationship between the grade and the underlying performance standard year on year. You get debasement or inflation when that relationship is not held and a grade indicates progressively lower levels of performance. Provided that a particular grade—let us say a grade B at A-level—relates to the same standard of performance year on year, not the same number of students, you hold the value of grades.
I want to answer Lord Knight’s question directly about what is wrong with grade inflation if things are getting better. If things are really getting better, I would not call it grade inflation. It is just grade rising. If grade inflation is happening when things are not getting better, the value or the currency of grades is being debased.
Dr Jo Saxton: If I could draw the committee’s attention to the national reference test that I mentioned, it is really interesting, because it shows an underlying increase in performance in the mathematical ability of the nation’s 16 year-olds between 2017 and 2020. That was statistically significant. It was significant enough that Ofqual would have required exam boards to make sure that what they are awarding reflected this increase, but the pandemic happened and it was lost in the wash.
Q133 Lord Knight of Weymouth: That is all very helpful to us. Going back on-piste, if the stakes at assessment and the way we use GCSEs for accountability at 16 were lower than they are currently, would that make things easier? Would it change or affect the way you regulate the qualifications?
Dr Jo Saxton: At the risk of frustrating the committee, the accountability decisions do not rest with me as chief regulator.
Lord Knight of Weymouth: As an example, I spoke at length last week to the relatively recently retired chief inspector for Ireland. When he changed its inspection system, it was at the same time as the Government and the Parliament of Ireland made it illegal to publish the test scores at 16 in such a way that league tables could be formed as a result of those, in order to dial down some of the accountability pressures and the stresses related to the system.
Here, we have a national referencing test that you have just alluded to, which would allow us, as a nation, to be able to see the trend in performance of the system. You could go to that Irish solution and publish the results for individuals. That is what people need in order to see in relative terms how they have done compared to others who have taken the exam at the same time as they have.
Dr Jo Saxton: The difficulty with the national reference test is that it is talking about the standard nationally. It is not talking about individual schools and colleges.
Lord Knight of Weymouth: We could hold individual schools to account. What I am driving at is this: if we took away the pressure of the test scores being used for other things apart from just how well that individual candidate has done, would that make the debate that we have just had about whether we move to a purely criteria-based system easier, because then you are not wanting to aggregate out scores to do Progress 8, see how schools have done and all that other stuff?
Dr Jo Saxton: I will just mention criterion referencing, and then my esteemed chair might have something to say about accountability. There are many contexts in which criterion referencing works incredibly well, particularly in vocational and technical qualifications when you need to be sure that a plumber or somebody dealing with gas can absolutely do the things that need to be done in order to be deemed fit to work.
On general qualifications, the criteria, by their very nature, require one route through and a defined set of things. The feedback from teachers has been that it made teaching much less creative, because it can, at its worst, become a form of teaching by checklist.
Lord Knight of Weymouth: We heard from the Institute of Physics in a very early evidence session that there is so much detailed knowledge in the curriculum that they cannot teach the big ideas of physics. If we were to move away from the emphasis on knowledge rich to more of the bigger ideas of the various subjects and across subjects, would it be more like music grading or drama grading exams, which are much more about criteria-referenced technique and putting an emphasis on the judgment of the examiner, rather than ticking the boxes of the key words that you have used and the bits of knowledge that you have demonstrated that you have retained?
Sir Ian Bauckham: In the lead-up to the decision to award qualifications in the way we did in summer 2021 and in developing the approach we took, we explored whether it would be possible to define a set of criteria that would give teachers sufficient guidance to be able to select grades for students that met their performance.
It proved almost impossible to do, because in the case of GCSEs, or slightly differently for A-levels, we would be looking at nine different, gradated criteria descriptions to match varying levels of performance in a field of knowledge where it was very difficult to describe the difference between, say, a grade 4, a grade 5 and a grade 6 in a way that was objective enough for a teacher to be able to reliably assign work to that descriptor.
It is much harder to do in the kinds of general qualifications we are talking about, GCSE and A-level, than it might be for practical matters—can this person weld two things together, drive a car safely, operate a piece of machinery or administer a medical procedure, or can they not?—because there is a definitive line there between two things.
Lord Knight of Weymouth: If we turned the dial down on the reuse of test scores for lots of things other than just how well a candidate has done, would that make it easier to regulate?
Sir Ian Bauckham: I was going to say something about the NRT. Just to clarify, the NRT is not taken by everybody. It is a large, statistically significant sample of the population.
Lord Knight of Weymouth: It would give us a sense of how the country is doing, because that is how you use it.
Sir Ian Bauckham: It gives you an accurate dipstick into the performance level of the country, but it could not be used for awarding qualifications per se. Accountability, as my colleague has said, is not the responsibility directly of Ofqual. That is a government decision and is open to different Governments to change, as and when they see fit, but it is quite hard to imagine a world now, where qualifications are awarded and recorded, in which somebody did not want to compile data about the relative performance of schools on the basis of those qualifications.
Lord Knight of Weymouth: They would want to.
Sir Ian Bauckham: They would want to, and one might ask whether it was preferable for it to be done properly and accurately or improperly and less accurately by a third party.
Q134 Lord Watson of Invergowrie: I am still trying to come to terms to some extent with your comments on quotas, which are certainly not what the committee had hitherto understood, I think it is fair to say. I am not a skier, so I am not sure whether my questions are going to be on or off-piste, to reference my colleague Lord Knight.
I have two aspects of grading that I want to raise with you. The first is one that we have received submissions on from organisations representing the sciences. It is what they call grade severity. Let me start with computer science, where the substantial evidence is that students typically get a grade lower in that GCSE than they do in comparable other subjects, and the same is true for physics and chemistry.
Baroness Garden of Frognal: And for modern languages.
Lord Watson of Invergowrie: Yes, and modern languages to some extent. Certainly the perception is that computer science is more demanding and too difficult, and that young people are being turned away from it, which is not helpful, given that we need more people in those areas. Indeed, it is government policy to increase the number of students taking science subjects.
I know you have received an inquiry in the past from the Science Education Policy Alliance, which gave us a submission in the last two days to say that it raised issues with your predecessors in 2016. In 2019, Ofqual said, “We don’t think that there is enough evidence here to change the grading criteria”. Ofqual is now investigating the grade boundaries in computer science GCSEs. You cannot comment on that just now because that is under way, but is it not appropriate to begin another investigation into the other sciences? We need more young people, especially girls, to be studying them for the jobs of the future.
Dr Jo Saxton: I completely agree with you that sciences are incredibly important, as are modern languages. On what my technical colleagues would call inter-subject comparability, Ofqual takes it really seriously and does extensive research on this. It is a slightly contested concept because to what extent is it meaningful to compare a student’s performance in art with, say, their performance in history?
Leaving that aside, there are different views about how you might approach a meaningful comparison between subjects, and Ofqual has done studies on all those approaches. There is a statistical approach, and the representatives who have written to you are largely referring to the statistical evidence of so-called grading severity. What is interesting is that, if you line up all the GCSEs for example, with “generous” at one end and “severe” at the other—to use the phrasing that stakeholders use—you have subjects like art and food at the less severe end, and engineering at the theoretically most severe end. In GCSE, the sciences sit broadly in the middle, although computer science sits closer to engineering.
Ofqual is doing work on computer science, as you say, and takes action where evidence in the round, including the statistics, demonstrates that that is the right thing to do. Ofqual has taken action, for example, in respect of some languages GCSEs and has required the adjustment of the standards in French and German to better align with Spanish. Where the evidence in the round, including very rigorous assessment of the statistics, indicates that there is a case to act, we act.
Lord Watson of Invergowrie: I accept that point, but the organisations representing the sciences are clearly motivated to implore us to raise those subjects with you, because they think that their subjects are not being fairly graded and that that is detrimental to the number of young people taking up those subjects. If nothing changes, there is no reason to believe that the number of young people pursuing those subjects is going to change.
Dr Jo Saxton: It is really interesting, because entries continue to grow for computer science at GCSE, so it is not holding up students in choosing it.
Lord Watson of Invergowrie: The British Computer Society says that it is.
Dr Jo Saxton: If you talk to school leaders, it is so popular that they are struggling to find enough staff to teach it. I can assure the committee that, where there is evidence to act, we do, and we meet with anyone who writes to me and makes those sorts of representations. I recently had extensive meetings lasting almost two hours with representatives from modern languages, for example, talking through and weighing up the evidence.
Sir Ian Bauckham: Particularly on the computer science GCSE, there were some significant changes to the content, I think in 2016 or 2018. We had very little evidence of the new GCSEs before the pandemic struck and exams were suspended.
Lord Watson of Invergowrie: But that was a new GCSE.
Sir Ian Bauckham: It is a relatively new GCSE. Now that we have it up and running again, we are absolutely committed to looking very hard at the outcomes in computer science GCSE to make sure that they are appropriate.
Lord Watson of Invergowrie: I look forward to hearing the outcome of that.
Dr Jo Saxton: Computer science GCSE is really interesting, because it is one where there are already components taught online. That was the key change that happened, which Ian is referring to. As you would expect, with it being computer science, it is appropriate for the domain that some of it involves direct assessment on-screen.
Lord Watson of Invergowrie: I am not an expert in computer science, but I am saying what we have received in submission from the British Computer Society.
Dr Jo Saxton: I have written to it at least once and would be very happy to engage with it again.
Q135 Lord Watson of Invergowrie: The second point that I wanted to raise is to do with grade reliability. One of the witnesses in the first session today mentioned your predecessor as chief regulator, Dame Glenys Stacey, and her comment that grades are reliable to one grade either way. Obviously you are familiar with that. Will the grades to be announced in August this year be reliable to one grade either way? If that is not the case, how reliable will they be?
Dr Jo Saxton: This is another really fascinating subject. With the greatest respect to my esteemed predecessor, I would not use quite the same words that she used. I am conscious, from some of your previous witnesses—for example, Dr Meadows, who was formally of Ofqual—that the concept of assessment grading unreliability is something that just exists in the world. A bit like the perfect hand-drawn circle, there is no such thing as the absolutely perfect assessment. There is evidence of imperfection ranging from driving tests through to degrees.
On the qualifications we regulate, I can assure the committee and young people who will receive their grades this summer that they can be relied on, that they will be fair, and that the quality assurance around them is as good as it can be. Quality assurance of marking and grading in England is up there with the best standards around the world.
In recent years, there has been a significant change in how marking and grading is administered in exam boards, not least involving the use of technology. Exam scripts, for example, are now divided up into questions and marked on-screen. To be able to access the marking system, the examiners must pass a qualification pre-marked question every time they log in to mark, and there are seed questions planted throughout the marking to check that marking consistency continues to be accurate. It also protects the student from the particular approach of one marker versus another. There have been significant advances in these approaches in recent years.
Lord Watson of Invergowrie: You quite rightly make the distinction between marking and grading, and that leads me to the question of people who are not happy with grades and what they do about it. There are a very small number of appeals in any one year. In 2022, there were only about 1,000, I understand, but there were 230,000 challenges. Challenge, of course, is the first stage in the process.
I do not want to go into the detail of challenge then leading to a review of the marking, but my point is that there is a great gap between the number of challenges and the number of those that then go to appeal. Schools, which are often cash-strapped, have to bear the costs, so to what extent have you considered whether schools are not taking forward challenges into appeals because of cost?
Dr Jo Saxton: As a former school leader, I was very conscious of the costs involved in a formal appeal. That is why we have, through regulation, increased the transparency around this. It is common already for all students, as well as getting their grade, to be able to see their marks. The grade boundaries are published by the exam boards, so they have a sense of how close to a boundary they were.
On transparency, as of last summer, across all regulated GCSEs and A-levels, it is now possible for the school to request the script, so teachers can sit down with a student and say, “Do you know what? Something has gone wrong with that marking. We’re going to spend that money and request an appeal”, or, “Do you know what? You just fluffed it that day. The marking is fair”. That transparency is absolutely critical to fairness for the students.
Lord Watson of Invergowrie: I take that point, but my question was whether there is any evidence that schools are feeling restricted from advancing to appeals for financial reasons. That has not reached you.
Sir Ian Bauckham: I do not think there is any firm evidence on that. Perhaps it is worth just clarifying that, if an appeal is upheld, there is no payment.
Q136 Lord Lexden: Could we turn to the implications of a move towards greater use of on-screen assessments at GCSE, a move much favoured by the Times Education Commission and the Institute for Government in their recent reports? We would be very grateful to have your views, which will take account, no doubt, of the review that Ofqual conducted in 2020.
Dr Jo Saxton: Like all modern development, absolutely there are opportunities, but there are also obstacles and pitfalls. I am really mindful, as chief regulator, of making sure that we provide the right guardrails so that students are protected from innovation. For example, there were other jurisdictions that went entirely online with their national assessments very quickly, and there have been significant issues with that, so we need to make sure that, as and when there is more use of online assessment, it is safe for students. It is definitely when, not if. As I have already said, some assessments, such as elements of GCSE computer science, are already online.
My personal interest—I have said this publicly before—is that I would love modern technologies to be able to remove the need for tiering in GCSE papers. It is a really tricky thing, as a leader, to have to make those decisions on behalf of the student. I think that time will come. At the moment, though, we are at the place of working out, as a nation, what the national infrastructure needs are and what the appropriate guardrails are for exam boards as they embark on this journey.
Lord Lexden: Will you be updating your review of 2020 and putting forward more in the way of published views and recommendations?
Dr Jo Saxton: We will absolutely publish our recommendations.
Q137 Lord Baker of Dorking: It is my turn to ask a question. Before I do so, you should be aware that 95% of the evidence we have received in this committee, both oral and personal, has said that the Progress 8 and EBacc curriculum is no longer fit for purpose, and that GCSEs are not fit for purpose. One of the professors from the Institute of Education described the fact that we have sit-down, written exams of a severe nature that determine lists of winners and losers, as an outlier in the rest of the world, not only Europe.
The original role of Ofqual when I set it up years ago was mainly to make sure that the exam boards did not fiddle with the downgrades. You have become one of the major bulwarks of the curriculum and of the exam system, which is unique in the world, but there is a huge volume of other support. There have been eight reports supporting the evidence that we have had in the last year—the HMC one, the one from the Youth Unemployment Committee of the House of Lords, the one from the Times Education Commission, the one from the Tony Blair Institute and the one from the Institute for Government. You seem to be rather lonely in the support that you have.
My question is this. What opportunities could AI offer in relation to qualifications and assessments in the 11-16 phase? We have already had examples today of how AI can change the literacy of children more than an exam can. I can give an example from a school that I am supporting. I noticed this term that several of the students were doing their answers on their computers, and that is much better. It is much easier for examiners to read digital replies than rather bad handwriting. Handwriting is not one of the skills that children have today. It is not required. You seem to be opposing all this. You are sticking by what we have.
Dr Jo Saxton: If I can be clear, as chief regulator, my job, unfortunately, much as I am passionate about the curriculum, is not to give advice or to determine what curriculum should be assessed.
On whether GCSEs are fit for purpose, I can assure the committee that they are fit for the purposes that government determined, which are to provide a comprehensive, complex curriculum, to enable progression—we must not forget that more than half of 16 year-olds in this country change institution at 16, so GCSEs are a key tool in their move to the next destination—and for purposes of accountability.
On the importance of computers, I never want to see the end of handwriting, but I completely agree with you on how important they are. No student has to write by hand in exams. In fact, many thousands of students every year are doing their GCSEs and A-levels by computer, so the current regulated system is not a barrier to greater computer use.
Lord Baker of Dorking: I simply do not agree with you, I am afraid, because I have run schools where every student has a computer. There are masses of computers in their schools anyway and they are digital-age. I do not think you have woken up to the fact of how AI will change everything in teaching and learning. It will fundamentally change it in a very clear and, I think, very optimistic way, but there have to be safeguards, which is one thing that we will sort out. I do not believe that there will be a case for the need for written exams at 16 in Britain for very much longer.
Q138 Lord Knight of Weymouth: I have three totally unrelated questions, but one immediately follows on from Lord Baker’s question and Lord Lexden’s question. We heard earlier from Mr Glanville from the IB about how, in the MYP diploma, they use digital for their assessment, and the great gains they get in 3D modelling and candidates being able to change some of the variables. There is huge opportunity attached to being able to assess using the digital form. What are you learning from examples of other jurisdictions in how they do that?
Dr Jo Saxton: Interestingly enough, I visited Mr Glanville’s boss in The Hague to talk about exactly these things. The international baccalaureate is one of the things that Ofqual regulates, so we have very good access to what does and does not work in other jurisdictions.
I mentioned earlier that we are doing a feasibility study about what it would take for high-stakes assessments in England to be fully digital. That is being done jointly with the Department for Education. As I have said before, there are huge opportunities, but we have to make sure that we do not throw any babies out with the bathwater.
Q139 Lord Knight of Weymouth: As long as you are learning from things, that is great. The second question follows up on the answers on national reference testing. Having had a quick look at the government website and the NFER’s discussion of that testing, it is very clear that it is just English language and mathematics. How do you regulate the rest of the subjects when you do not have the reference test to compare against? Do you infer on the basis of the pattern of performance on maths and English language to guide you in how you regulate the performance across the rest of the curriculum?
Dr Jo Saxton: I will give a dual answer. On the national reference tests, between English and maths, that gives us a really good bellwether about the national performance. My colleagues at Ofqual observe awarding and all the exam board processes so that they can see at first hand that due and proper process is being taken—for example, with archived scripts that demonstrate what the quality of work at a certain grade looks like over time. That is continuing to be factored in, but we use data in the fullest sense to regulate. We require significant training and quality assurance on the part of the exam boards to make sure that expertise and human judgment always come into play. Data points are an important check and balance.
Lord Knight of Weymouth: Would it be helpful to you to have national reference testing in other subjects?
Dr Jo Saxton: It would be absolutely wonderful, but the Department for Education and the Treasury would not necessarily want to fund it for every subject.
Lord Knight of Weymouth: It might be that there is some testing that we do in all schools with all pupils that we do not need to do, and that we could therefore afford to do sampling tests across other subjects.
Sir Ian Bauckham: That would clearly be a wider political decision.
Dr Jo Saxton: NRT is a very powerful and really important tool.
Q140 Lord Knight of Weymouth: My final question is about the difficulties of assessing functional skills in English and maths, which is a real obstacle when it comes to apprenticeship requirements for competence in English and maths. GCSEs are required, and that is a difficulty for apprenticeship access. It has bedevilled us for a long time. You also regulate the range of qualifications, including the technical and vocational, so do you see progress in us being able to find something that is more reliable for English and maths at a functional level without requiring GCSE?
Dr Jo Saxton: On functional skills qualifications, particularly the recently reformed ones, we have heard concerns from stakeholders. My team has done a significant piece of research, because there has been a perception that the reformed version is more difficult. We published our findings, which set out that in the pure assessment sense the questions are not harder than in the unreformed version, but we accept that there might be something else going on. For example, could there be something with text density or font that is making it feel less accessible to students?
As with the other qualifications we regulate, this all comes back to what the curriculum that it is built on is asking. The reformed functional skills assessments absolutely assess the curriculum as set out by the Department for Education.
Lord Knight of Weymouth: Ian may have to answer this, because you may feel constrained, but it sounds to me that if we had a curriculum that was more relevant to the world of work and the application of English and maths for these more vocational qualifications, it would be easier for us to come up with an accessible qualification that would then allow those individuals to progress. Is that fair?
Dr Jo Saxton: I will let Ian come in, but I have had this discussion with officials. It was particularly stimulated for me by a visit I made in Canterbury, where I live. I met a young apprentice who wanted to qualify to work with babies. She was continuing to struggle to achieve the functional skills to finish off her apprenticeship. She talked to me about how she knew automatically the ratios of milk and of adults to babies in the room, but she really struggled with some of the mathematical questions.
This comes back to the purpose of the qualification. As functional skills are currently defined, they are meant to enable somebody to function in that skill in any career, not only in one specific domain. Whether to set up employment route-specific functional skills would be a matter for government, but, as they currently stand, they are not meant to silo somebody into a particular route.
Lord Knight of Weymouth: Within occupations, you could define the functional level of literacy, numeracy and digital skill that would be required in order to deliver that occupational standard, and then it would be possible to regulate an assessment to do that.
Dr Jo Saxton: Theoretically it would, absolutely.
Sir Ian Bauckham: But then they would not be cross-domain qualifications, so you would not be equipping the young person potentially to move from one domain to another, because every exemplification that they will have studied of the application of those functional skills would be highly domain specific.
Lord Knight of Weymouth: That is unless we move to a modular approach, which is what the Government’s lifelong learning entitlement implies that we are going to do.
Sir Ian Bauckham: Yes, potentially.
The Chair: Dr Saxton, you mentioned the possibility of using technology to remove the need for tiering in exams. Could you say a little more on that and say whether it could be applicable at GCSE level?
Dr Jo Saxton: It would absolutely be applicable at GCSE, and that is where tiering currently exists in a number of subjects. It is something that digital technologies could deliver at a point when one gets to fully adaptive interactive questions, where the next question changes depending on how you have answered. The difficulty with adaptive tests is that they are incredibly resource-intensive to develop. You need millions of questions for them not to be predictable, so that young people are not able to cheat, in short. It is a thing that I personally care a lot about, but we are some years away from being able to deliver that.
Sir Ian Bauckham: But there is certainly potential for AI in the generation of test questions to address the problem of resource intensity that there has hitherto been in the creation of fully adaptive testing, as Jo has referenced. That is an exciting potential future area.
Q141 Lord Baker of Dorking: It is no longer necessary for any youngsters leaving school to have a certificate at 16, which was necessary when I took the predecessor of the school certificate in the 1940s, because then only 6% left school. They now go on to 18, so a certificate at 16 is largely irrelevant, quite frankly, to what they have by 18. Then they want skills, but skills do not come anywhere in GCSEs. The only skills that are in GCSE are English and maths. There are no other skills. Everybody is coming to us and saying, “We want people who’ve worked in teams, who’ve done creative and critical thinking”, and that sort of thing. Your exam system does not do that in any significant way.
Sir Ian Bauckham: Jo may have mentioned already that a very large proportion of young people change institutions to go on to other courses, including many more specialised courses, at the age of 16. It is only one, but one of the important purposes that GCSEs serve at the moment is to help ensure that those young people transfer to a level and a type of 16-18 education that is suited to their achievements.
Lord Baker of Dorking: But that makes us the outlier. Europe is mainly doing this at 13-14.
Dr Jo Saxton: Again, we come back to the national infrastructure. One of the peculiarities, if you will, of the English system is that the majority of schools do not have sixth-form provision, so a significant proportion of our young people have to change provision at 16. GCSEs help young people to understand their suitability for a future course, and help the admitting institution, particularly colleges and sixth forms, to get assurance about their suitability. Again, this is coming back to our national infrastructure.
The Chair: I would like to thank our two panellists very much for their time. Your contributions are much appreciated.
