Oral evidence: The impact of Covid-19 on education and children’s services, HC 254
Wednesday 10 June 2020
Ordered by the House of Commons to be published on 10 June 2020.
Members present: Robert Halfon (Chair); Jonathan Gullis; Tom Hunt; Dr Caroline Johnson; Kim Johnson; David Johnston; Ian Mearns; David Simmonds.
Questions 453 – 526
I: Dr Zubaida Haque, Interim Director, Runnymede Trust; Kevin Courtney, Joint General Secretary, National Education Union; and Professor Lee Elliot Major, Professor of Social Mobility, University of Exeter.
II: Tom Bewick, Chief Executive, Federation of Awarding Bodies; Sally Collier, Chief Regulator, Ofqual; and Dr Michelle Meadows, Executive Director for Strategy Risk and Research, Ofqual.
Written evidence from witnesses:
Examination of witnesses
Professor Elliot Major: I am Lee Elliot Major, Professor of Social Mobility of the University of Exeter.
Kevin Courtney: I am Kevin Courtney and I am the Joint General Secretary, with Mary Bousted, of the National Education Union.
Dr Haque: I am Dr Zubaida Haque, and I am the Interim Director of the Runnymede Trust, which is a race-equality think tank.
Q454 Chair: Thank you for coming again. The Department for Education’s guidance rationalises the calculated grade and standardisation model on the basis that simply using predicted grades would not be fair to all students, but a survey by TES found that just 39% of school staff in England believe calculated grades will be fair for all. The Sutton Trust said, “Any system relying on teacher assessment risks underestimating the abilities of disadvantaged students”. To what extent are you confident that the grade calculations and standardisation process eliminates the potential for bias against pupils from lower-income backgrounds, those with special needs or disabilities, and boys? What data do you have to back this up?
Professor Elliot Major: I think we are going to have to have some systematic review that looks at how these results play out for different students and how fair they are for different students. There is lots of evidence around human bias in assessment. It is nothing to do with teachers, by the way. We are all prone to bias, and that is around stereotype and confirmation bias. There is a lot of research literature about this. The worry is that, unintentionally, teachers will sometimes underestimate the academic potential of poorer pupils, potentially those from black backgrounds—there is some literature in America on that—and potentially boys as well.
One of the strengths of the system is that it is based on prior results. This is going to be based on a lot of information. The problem with that is that it penalises those pupils—of which I was one—who left it to the last minute. It is those sorts of outliers who are potentially penalised, so I think it warrants some sort of review over fairness after we get the results.
Q455 Chair: Anyone else? Kevin, I should just say, as a welcome, we are going to invite you, or one of your representatives, back next week to talk about school openings, but today is specifically about exam cancellations and grades.
Kevin Courtney: Thank you, Robert. We very much welcome that invitation to talk about school opening. On this question, we should not pass over the extra difficulty that has been caused by the all-eggs-in-one-basket emphasis on terminal exams. Pre the Gove reforms, there would have been more modularity, with people having more results. It is more difficult when we have lost all of the formal assessment because the exam season has been unfortunately lost.
I agree with Lee. Teachers are human beings, and human beings are subject to subconscious bias. I have read some of the research that Zubaida’s organisation has produced. The way I see this is that middle-class people sometimes underestimate working-class people, and black kids are more in the working-class and disadvantaged areas, so there is bias. You have to build a system that copes with that. Personally I would be in favour of a teacher assessment system, but if you had a teacher assessment system you would be providing inset, you would be thinking about reflection and you would be thinking about a system of teacher moderation to try to deal with the bias. We do not have that in this situation.
Our members, teachers up and down the country, are doing what Ofqual has asked of them. They are doing their very best to get the right estimated grade for their students. They know their students well, they are professionals, they are working hard and they are doing a good job of giving estimated grades. They will not be the grades that the child necessarily gets. Ofqual, as you say, will use a statistical model to allocate a grade, which may be different from the estimated grade the teacher has.
Chair: We have a tight session, so if everyone could be as concise as possible.
Dr Haque: Sorry, I disappeared because my internet is unstable. Forgive me if I disappear again.
We have quite a lot of concerns that we have raised with the Department for Education and Ofqual. As of last night, we still have not had any responses from the DfE. We sent them three letters overall from the race-equality sector, as well as from other education academics and so on. I am sorry, Kevin, I missed quite a lot of what you said because my internet was unstable.
This is around a couple of things. One, of course, is that these are exceptional circumstances. We completely understand that DfE and Ofqual are doing their best, and we know that teachers and schools are absolutely trying to do their best to be fair, but these are exceptional circumstances. Right now we have a predicted grade system instead of exams, which has been untried, untested and unevaluated. That is a serious concern.
To be short, the main concerns that we raised were clear. There is a lot of evidence already out there, well documented evidence, about some of the unintentional as well as intentional biases against specific working-class students—we have evidence about higher attaining working-class students—but also particular ethnic minority students and specifically black Caribbean boys, as well as Gypsy Roma and Irish Traveller students. We have seen evidence to show that there is bias in predictions, and teachers tend to underestimate, not overestimate, their final performance.
What we suggested very specifically to DfE and Ofqual was that they should be thinking about using the public sector equality duty. The public sector equality duty is mandatory for all public authorities, as well as for schools and Ofqual. Right now teachers are working in exceptional circumstances in an untried system. They need more guidance to ensure that any unwitting or witting biases do not creep in when they are calculating grades. We suggested that both DfE and Ofqual send guidance to help teachers, because at the end of the day this is not just about students; it is also about helping teachers.
We have not had much of a response. It is important to understand that Covid-19 has exposed or brought into sharp relief preexisting racial and socioeconomic inequalities. That is why we are particularly concerned in this context.
Q456 Chair: Can I specifically ask Lee Elliot Major, given your studies on social mobility, what you think are the effects of an under-predicted, inaccurate grade award on the life chances of a disadvantaged individual, and how far down the line in a young person’s life are these consequences felt? Are they felt more greatly by disadvantaged pupils than by their better-off peers?
Professor Elliot Major: There has been a lot of discussion about learning losses and how we estimate that. One thing we can say is that they are going to be big. However, in many ways, the more important question is about the material impact on someone’s life, on a young child’s live. The thing that worries me is that if you miss out on those threshold grades, if you don’t get that 4 in your English or maths GCSE, you are unlikely to get employed at the moment. If you don’t get into that sixth form—and by the way, sixth-form admissions are a bit of a wild west out there, and every sixth-form college has different admissions criteria—or that university, if you miss out on those grades, that can have lifelong impacts. In many ways that is the bigger issue for me than how big the actual gap will be.
The other thing I want to add to this—I don’t know if we will get into it later—is I know we are concerned with the assessment of current cohorts doing GCSE and A-levels, but I worry that a tsunami of anxiety is hitting schools, with the million children who are going to be taking GCSEs and A-levels next year. Interestingly, I agree with Kevin on this. I think you need a combination of teacher assessment and some core exams. There are lots of inaccuracies in examinations that we do not talk about so much. I would prefer a balanced system. I saw that the Government have announced that there will only be exams, but I would go for both.
Q457 Chair: Zubaida, on the groups that are most affected, you started saying what the statistics showed. The data from BEIS suggests that black students had the lowest predicted grade accuracy at 39.1% compared with 53% for their white counterparts. To what extent is there evidence that unconscious bias or passive racism is the reason for these inaccuracies?
Dr Haque: There is a lot of evidence. There is direct evidence and indirect evidence. If I may, I am also going to talk about the fact that it is not just a question of evidence. We have had conversations with Ofqual where it has been challenging us about the evidence and suggesting that the evidence is mixed. That is not the issue here. The issue is there is a public sector equality duty that all public authorities are supposed to do to ensure that there is not bias. You are supposed to check for it regardless of the evidence.
You asked me a specific question about the evidence. There is considerable evidence from Gill Wyness’s research to Simon Burgess’s research—both academics—showing that specific black students are under-predicted, that GRT students are under-predicted. Can I also say that the Government themselves are aware of that? That was also on the DfE website as a question about how we can ensure that bias does not creep in for working-class students. We know, by the way, that black and ethnic minority students are predominantly from lower socioeconomic groups.
The other thing is the Government commissioned a review last year, the Timpson review, that was based on school exclusions. School exclusions, and particularly in terms of racial inequalities, is about the fact that particular BAME students are being disproportionately excluded from schools. Theresa May was aware of that. That was the whole reason she commissioned the Timpson review. The reason I mention that is the same stereotypes and prejudices, whether they are conscious or unconscious, that teachers have for excluding pupils—and we know that they exclude pupils disproportionately in terms of race—are not going to go away when they mark students’ papers, when they predict their grades, and that is significant.
Q458 Chair: Could I ask all three witnesses, in a nutshell, which groups do you believe could be most detrimentally affected by the calculated grade system?
Kevin Courtney: Zubaida has gone through what the evidence is about that. The question is what systems we are going to build to overcome them. We know that all human beings have unconscious biases. It is not just teachers; it is politicians as well who have that passive racism, those subconscious biases. It is a question of what systems we build, and that is what we need to address. Teachers are now doing estimated grades, but within those estimated grades they have to rank students. That is something they have never done before in any systematic way. They cannot do equal rankings, so there are very fine grade judgments that have to be made there. We realise that there will be problems. Looking at these in the long term needs to happen.
It is also important to say that these discriminatory effects happen every year. The way we run our exam system means that 38% of children do not get a standard pass in English every year because of their comparable outcome system.
Q459 Chair: We are looking generally at left-behind groups here, but this specific session is on exams and grading because of Covid. Lee, can you answer the question about which groups are going to be most disproportionately affected by the grading system?
Professor Elliot Major: For me it always comes back to the most disadvantaged children in society. As we were saying earlier, they are more likely to come from black and minority ethnic backgrounds as well, so you get what the academics call intersectionality. These things combine to make it even worse for those children. Ultimately for me it comes down to disadvantage.
By the way, one thing I would say is that the system is characterised by two gaps. There is the gap of those who are left behind, which I know the Committee is going to look into, which is maybe 20% or 25% of pupils who are really being left behind, and then there is a group of pupils who are pulling away from the rest, perhaps 10%, at the upper end. I urge the Committee to think about two gaps, because I think that is what is happening in this crisis.
Q460 Chair: Can you give a quick response, Zubaida, please?
Dr Haque: Yes. I have already outlined, and I agree with Kevin and Lee, that specific working-class students and specific BAME students are likely to be disadvantaged. It is not a question about which groups are going to be disadvantaged and whether we should do something with those specific groups. This is about fairness. The way you ensure fairness is by carrying out equality impact assessments, whether it is on policies or people’s decisions. That is all we are asking for, just make sure those biases do not affect students, not only in the short term but, as Lee pointed out, in the long term so that gross injustices are not carried out.
Q461 Chair: Please be as concise as possible, because we have so much to get through and we are under very strict limits because of broadcasting issues with the coronavirus. The first stage of Ofqual’s grading process asks schools to provide centre-assessed grades and pupil rankings. Which groups are most likely to be vulnerable to unconscious bias in this process?
Dr Haque: I am sorry, I thought we had already discussed this.
Chair: We discussed it in general terms. I am looking at the different methods that Ofsted is using for the assessment.
Professor Elliot Major: One thing I can add is that if the system penalises, which I think it will do, those who leave it to the last minute—the pupils who leave their revision right to the last minute—there is evidence that boys more than girls will tend to do that. Certainly the poorer pupils might do that and also, by the way, the younger children in the class. There is lots of evidence—and I speak as a summer-born child—that those who are slightly less mature in the year will also leave it to the last minute, so there are lots of different dimensions to that particular question.
Q462 Tom Hunt: This is really important, and it is good there is so much research going into how different people are going to be negatively impacted, potentially, by a predicted grade system, but I would urge a focus on SEND and unconventional learners in general. I say this as somebody who was dyslexic and dyspraxic at school, who quite often got A-Es, quite good results but not trying hard. I was trying hard, I was just a bit strange, a bit different. I think there are lots of children who fall into that category and their brain works a bit differently. They do not necessarily follow process in the same way, but they are bright and they have a lot to offer. A lot of that will be to do with those who have special educational needs, so I would urge, with all the great work that is being done, a focus on that as well.
Dr Haque: That is a very important point. We know that particular black and ethnic minority groups are also more likely to fall under SEND, in particular Gypsy Roma and Irish Traveller students. Yes, we completely support you on that point.
Q463 Jonathan Gullis: I am glad that we are talking about disadvantage, as Lee mentioned earlier, because it is absolutely obvious that BAME communities are going to be harmed by this, but I think that white working class is also going to be a hugely impacted sector. We know that, by the age of five, white boys from disadvantaged backgrounds are already 13% behind disadvantaged black boys in phonics, and we know that disadvantaged white working-class boys are 40% less likely to go into higher education than disadvantaged black boys. It is important that when we talk about these issues we look at disadvantage, full stop. All racial backgrounds are going to be hugely impacted and underestimated. Boys, as Lee and others have referred to, bloom later. We all know that boys will do rubbish in the Christmas time and then suddenly pick up right at the end. What Tom said about SEND is absolutely crucial as well.
It is very important that we do this and really hammer home on disadvantage. We know that gap is going to grow, but we know it is going to grow for all students from all racial backgrounds and it is about how we try to reverse what could already be irreversibly irreparable.
Kevin Courtney: I think those are the groups to focus on. I would say that teachers will be looking at that question and will be reflecting on the fact that some children are last-minute learners and reflecting on the position of SEND children. They are skilled in trying to do that, but that does not mean they will get it completely right. The process that they are going through that is more important than estimating the grade is the ranking process. Each teacher will be trying to rank as well as they can within their class. Then their head of department will be integrating those grades across their schools, but without a defined moderation process. One could not be worked out in the crisis we are in, and there is no moderation process between schools. These things are there for those sorts of reasons.
We should also think about the point that Lee made. It is not just the assessment; it is the impact of the assessment and the impact of the grade. Young people who do not get that standard pass and who then cannot get on to their A-level course or who cannot get into employment is something to think about. If we just have the very strict grade boundaries that we had last year, there will be lots of children who miss out.
Ofqual’s statistical model will, to some extent, have to look at previous achievement in a school, but if you are in that school and you have done better than the previous achievement, you might find yourself not getting the grade for that. There are all sorts of unfairnesses in the system as well.
Q464 Jonathan Gullis: Kevin, thank you very much. I know that the NEU and I are probably going to disagree on much, based on our interactions on social media in the past, but one thing I will say is that you talking about ranking is an absolute godsend. I sat in the kitchen with my partner, who is a head of RE, who spent seven to eight hours ranking children, with other members of her department, kids who she would not have taught herself, and trying to come to some sort of a decision on who deserves to be 1, 2 or 3 and below. I think it was insanity, to be quite frank with you, so thank you for raising that. That has been a huge workload and hugely under-talked in this whole Ofqual process.
Dr Haque: Yes, I completely concur. The first letter we sent to DfE back at the beginning of April—and as of last night, as I mentioned, we still have not had a response—raised this issue, Jonathan and Kevin, about ranking. We raised it with Ofqual specifically in Zoom meetings, that it was an extraordinary ask for teachers to rank students, something that they have not had to do on this scale ever before. Not to give them guidance, first, on how to do it and, secondly, on how to ensure that bias is not creeping in is an astonishing thing. As far as I know, DfE has not. I know Ofqual has listened and is sending teachers more guidance.
We have been focusing a lot—and rightly, to some extent—on which groups are losing out, but that is not the key question here. The key question here is, given that we know biases have not only crept in but have historically damaged particular groups, what measures are we going to take to ensure that that bias does not harm students from this cohort, the Covid-19 cohort, in the long term? That is the first question. The second is: what does that entail? What is the detail? It is those two questions that I feel DfE and Ofqual have not been as strong on.
Chair: That is the purpose of our inquiry and this session, so that is why we asked you, and we have Ofqual coming in next.
Q465 Kim Johnson: Covid-19 and Black Lives Matter have highlighted the inherent inequalities that exist on race and poverty. Panel members have discussed that and talked about the prediction of grades will disproportionately impact on black pupils, working-class pupils and SEND pupils. My question to all the panel is what further research might be needed, after results day, to determine whether these groups have been impacted and unfairly marked down by the process?
Professor Elliot Major: I said earlier that I think there needs to be almost a review around fairness. We talk about this word a lot. This generation, if we are going to call it the Covid generation, is really committed to social justice. We have seen that over recent weeks. I believe that we have to have a system that stands up to the fairness test. I think we should review the results to look at which pupils might have been further disadvantaged by them. We have to make this a fair system.
Kevin Courtney: I agree with that. You can see the research at a statistical level of comparing the achievements of black students, or even more fine-grained ethnicities than that, compared with their achievements last year, the achievements of SEND students compared with the achievements of SEND students last year, so you can look at it at a statistical level. That does not quite get to the level of individual unfairness and the chance for a child to do an exam if they think they have been wrongly graded.
I hope that the Committee would also undertake some research into the difference between comparable outcomes in norm-referenced exams and criterion-referenced exams. We have a stable 38% of children who do not get a standard pass in English each year. It is the language they speak, and every year 38% of them are told that they do not get a standard pass. We are not measuring their achievement; we are ranking them each year. We should look at that for the longer term.
It is important—I do not know whether in this meeting or in further meetings, Robert—to look at the exam system for next year. Lee touched on this earlier. We could face some of these similar situations next year, so we have to learn from this year to improve the situation for the next.
Chair: This is one of the questions I am going to have for Ofqual. They are listening in. Let alone this year, I am very worried about what next year’s problems are going to bring. We will ask that question.
Dr Haque: I very much echo what Kevin and Lee have been saying. For me it is around three broad areas. There is obviously the issue of equality impact assessment of how this entire system has worked. The Government should be doing that in any case. To me, whether you do that through an inquiry or not is irrelevant. In any case, we need to have an equality impact assessment of how this system has worked and whether it has disadvantaged particular groups. That is the first.
The other area of course, and we have not touched on this at all, is around teacher bias and to what extent teacher training has dealt with that historically in any which way or form. Runnymede Trust and other race-equality organisations, as well as education academics, have raised this issue that teacher training at the moment is not fit for purpose when it comes to issues around race equality. Jonathan is right that there is substantial evidence showing that there are biases for white working-class students. We need to think much more about teacher training and to what extent teacher training needs to be improved and expanded in relation to pupils from different backgrounds.
Chair: Thank you. The teacher training point is an important one.
Q466 Kim Johnson: I want to follow that up by asking the question about predicted grades being likely to widen the attainment gap, and looking at how Ofqual needs to look at that and what needs to happen regarding that issue.
Professor Elliot Major: It is a really good question. The other thing I would urge the Committee to do is to look at the divide in private tutoring that is happening outside schools. You need to look at both economic and social inequalities as well as educational inequalities. We tend to silo these issues. What I am observing outside the school gates is huge gaps in which children are getting private tutoring or the extra support. I know you are looking into this, but that also has an impact, of course, on how well the kids do in their assessments. This is so important, and we tend to ignore it when we look at the performance of schools, for example. Increasingly, the education-societal divide is blurring. I would emphasise to look at both outside and inside-school factors.
Q467 Ian Mearns: When it comes to the grading system, those pupils and their parents who are unhappy with the grade that the youngster is given will not be able to appeal their result but will be able to raise a complaint to their school if they have evidence of bias or discrimination. What needs to be done to ensure that this process is as fair, open and transparent as possible and does not just become some sort of preserve of those parents who are more well-heeled and more articulate, with the sharpest elbows and, for want of a better term, are the educationally worried-well?
Chair: I find it extraordinary that the appeals process is based on so-called bias and not on marks of the grade. How on earth is a student going to know whether there has been some kind of bias or discrimination unless they, as Ian says, come from a family that knows how to lobby MPs, work tribunal systems or appeal systems, or whatever it may be? It seems to me quite a system stacked up against the disadvantaged.
Dr Haque: That is excellent, Ian, because this is exactly what I was going to talk about before you came in.
Ian Mearns: I have my moments.
Dr Haque: Chair, I agree with you. There are huge concerns about this exam process. First of all, how do you even prove discrimination? The whole reason the 1976 Race Relations Act was introduced was because it is very difficult to prove direct discrimination. That is why we talk also about indirect discrimination. Especially when it comes to gender and racial discrimination, it is incredibly difficult to prove. That is the first issue. I completely agree with you about parents with sharper elbows. They are more likely to benefit from that system than parents who have not had the advantage or the confidence of working with the education system to get their own way. There is an issue about how you prove it.
The second issue, of course, is we know from the Sutton Trust, as well as from other sources of evidence, that disadvantaged students—and some particular black and ethnic minority students, but they are one and the same—have considerably lost out in remote learning throughout the last couple of months. To say that, if students are not happy with their grade, they have the opportunity to resit when they have not had that advantage from remote learning from the last couple of months, we know they have significantly lost out, but they also are not going to be able to afford tutors and so on. That system will not benefit students from disadvantaged backgrounds. We have raised with Ofqual that we do not think this alternative for sitting exams to improve your grades or to challenge your grades is fit for purpose.
Kevin Courtney: I agree with what Zubaida has just said and with the thrust of Ian’s question. There is false hope for people in the idea of an appeals mechanism that looks at bias. Also, for the reasons that Zubaida identified, there is not much hope in the idea of doing an exam to change it, because the children we are talking about are those who quite often do not have a broadband connection at home, do not have a tablet; they have less.
It is important to look at this situation, which is unique. Everyone is doing their best. Teachers will be doing their best to try to avoid these things, and Ofqual will be doing its best to try to make sure its statistical model does not exacerbate these difficulties, but when we are talking about that, we have to recognise there are these biases in every year. We know that disadvantage at home leads to disadvantage at school, and we have to look at the system every year. We have to take into account that this year, because of the Covid impact on the economy, there might be 200,000 extra pupils who are in poverty.
Q468 Chair: What would any of you do differently to the appeals system to make this fairer?
Professor Elliot Major: I think there is an issue of two cultures in the education system. There are the sharp-elbowed parents that Ian was talking about.
Chair: I want an answer to the specific question from you guys. What would you do to solve the problem that we have clearly identified, you have identified and we have identified? Lee, sorry, I have interrupted you.
Professor Elliot Major: You could do some sort of random sampling of particularly those from more disadvantaged backgrounds to see if their grades have—rather than leaving it to them, you do some sort of case studies to try to do that. It is very difficult. I also think you need to have some sort of campaign to try to empower parents to make sure they know they can appeal, because there is a lack of information.
Dr Haque: The onus should not be on students or parents to prove that they have been given the wrong grades. That is an incredible ask for parents, regardless of what background or advantages they have. That is an unfair system, particularly for parents who come from backgrounds where they do not necessarily have the confidence to interact with teachers and the education system and take them on. That is an unfair system.
I do not know if exams are a good alternative to challenging your grades. One of the things we said is that parents should be able to go back and challenge teachers or schools themselves about those grades, and take it up with the school as opposed to the system. Parents should not have to prove the discrimination, and students should not have to undertake exams in order to prove that they can get a better grade. It is a very unfair system to put that onus and responsibility on them.
Kevin Courtney: The circumstances that we are facing are truly extraordinary, as everyone knows, and it is hard to get the system right this year. I do not think we can blame teachers, schools or Ofqual for that. I do have a suggestion. We have to think for the future. We need far more BAME teachers, black teachers, in our system, we need to look at training, we need to look at moderation of teacher assessment. All those things can be worked on and should be worked on. We need to look at the general level of inequality in society, things like Sure Start.
Q469 Chair: No one would disagree with all those things, but we have an immediate problem. I am trying to understand how you solve the immediate problem.
Kevin Courtney: I think this year—and this will be controversial—Ofqual needs to relax the grade boundaries. There need to be more children getting all of the higher grades. Where there is doubt, somebody should move up a grade mark rather than moving down a grade. That will mean the exams are regarded as not having the same rigour as the previous exam series, but I do not see how else we are going to get through this.
Dr Haque: We had this conversation with Ofqual, and it was alarming for us that Ofqual was more concerned about over-compensating students in grade predictions rather than what is currently going on, which is that particular groups of students are being under-predicted. We did challenge it about that and asked, “Why is it that you are happy to have a system where particular groups are being under-predicted but you are not as happy if they are over-predicted?” It seems that the emphasis is there. I hope, Chair, you will take that up with Ofqual.
Q470 Ian Mearns: I must admit I am finding this whole session is becoming very internally frustrating, because you can see what is mapping out here. We have already, to a certain extent, established that a resit system will probably further disadvantage the youngsters from disadvantaged groups. To get it on the record, is that your thinking?
Kevin Courtney: Yes. It is not a resit system, because the children have not sat the exams. As I understand it, there is no way of moving down, effectively. Both grades will be considered. Whatever you get from the teacher assessment, the Ofqual statistical model process—if you do the exam you might move up a grade but you cannot move down a grade. For some children they will want to do that, but it is not the solution that we are looking for.
Ian Mearns: I cannot help thinking, given the ranking exercise that Jonathan’s wife was doing at home, this seems to build in all of the innate prejudice that exists out there in society, sadly, but that we can do very little about in the short term. The trouble is we are looking at short-term issues here that are going to have lifelong outcomes for children.
Q471 David Johnston: A couple of questions for Lee. The Children’s Commissioner told us last week that if no more children went back to school, 8 million children would have been out of school for six months come September. What do you think needs to happen now to mitigate the impact of that learning loss? What sort of interventions and what role should Government play in that?
Professor Elliot Major: This is a key question. I am really worried about what I call the wave of anxiety. I happen to have two children who are taking exams next year. We need real clarity on what these exams are going to cover. Some teachers are trying to give the kids as much content as possible over the last few weeks, others are just doing consolidation. No one knows. Clarity over examinations for next year is key.
Over and above that, I have been championing—and I know members of the Committee have been as well—the idea of a national tutoring service. This is where you get undergraduates and graduates to provide one-to-one support for disadvantaged pupils. The idea is to tap into that volunteering spirit among many young people. Many graduates will be unemployed, by the way, so this could be a way of helping them at the same time. It would help teachers, because teachers are going to be dealing with all sorts of pastoral issues over the next few weeks and certainly in September. I do not think it is going to be a normal school year next year. That is increasingly clear.
Q472 Chair: How do you scale that up nationally?
Professor Elliot Major: There are tutoring organisations across the country. What they need is extra capacity to do it. We already have brilliant social mobility tutoring organisations around the country. We just need Government commitment to it. The big challenge is targeting, as David will know from a previous career he had. How do you enable the regions of the country that need it the most? That is why it has to be a national programme. There has to be oversight from the Government in some way so it goes to the places that need it most, the schools that need it most and, most importantly, the children who need it most.
Dr Haque: I do not know if you know, but the Independent SAGE group—full disclosure, I am a member of the Independent SAGE group—talked about this quite a lot, about what more we can do for the students who are not going to be able to return to school this summer, particularly being concerned about their learning. More crucially, education is not just about learning for these kids. These kids are also missing out on the social and psychological gaps around interaction. For a lot of students, we know there are mental health issues that are coming out now because they are missing that peer-to-peer contact. I agree with Lee that tutoring could be one solution. The only issue is that it just solves the learning issue, it does not solve the issue around peer-to-peer contact and how important that is and the psychological impact of that.
What Independent SAGE suggested is that there are so many different venues out of school right now that cannot be utilised because of Covid-19, whether that is football stadiums, private schools, fields, golf courses and so on. This is a good time to be thinking about whether we can use those venues and expand the school workforce—we could use some of these tutors you talked about earlier, Lee—so that we can adhere to social distancing, so that more students can have learning and education and so that more students are able to interact with each other and get the social and psychological benefits of that as well.
Kevin Courtney: Let’s hope that the virus is enough in control that we can get lots more children back to school in September, but I do not think that is what civil servants are estimating. We are looking at this disruption carrying on possibly even beyond September, beyond the six months, hopefully with more children in school. That means that we have to ramp up this delivery of broadband and laptops to every child who needs one. I do not see why that could not work. Jeff Bezos and Amazon have made a lot of money out of our country. Why are we not delivering reading books to children who are disadvantaged, to children on free school meals? Why are we not delivering art materials to those children?
I agree with Lee and Zubaida. Expanding the workforce—I am not quite sure about the tutoring thing, but getting those people—we need to have summer holiday clubs to deal with some of these emotional questions. Then if we have to have social distancing, if that means reduced class sizes, we need more adults to help with the teaching, and that should be another question that we have to look at as a society.
Chair: I think the book delivery idea is an important one. We must put that in our recommendations. I have also proposed a national broadcasting service by the BBC and others that could broadcast on TV, because we know most homes have a TV if they do not have the internet.
On your point, Zubaida, about wellbeing and pastoral care, I agree with you. I have met with Place2Be and I think that should be scaled up in schools—charities like that should be scaled up—as the kids return to school. I am sure you agree with that, Lee, as well.
Q473 David Johnston: Your idea is an interesting one, Lee. How do you think these sorts of interventions should be measured for effectiveness? Without putting words in your mouth, one of my concerns here is destinations. We are talking an awful lot about grades and what happens to those. I have a real concern about the destinations of the young people who have missed all this schooling. Even if they are getting online teaching, there is no substitute for being in the classroom. What are your thoughts about how we can measure the effectiveness of what we do here?
Professor Elliot Major: Ultimately, I think you have to measure whatever you do in terms of lifelong material impacts on the children who we are trying to support. I totally agree with you on that. We do get obsessed with grades, and they are very important.
Kevin touched on it earlier. I think the Committee should look, in its inquiry, into the left-behind, into the very basis of how we do assessment. There should be a basic school certificate that every pupil should be expected to pass. That would be basic English and maths that you need to survive and prosper in life. You need to look at it systematically to do that.
The other thing I would say is the tutoring is not just about getting the basics. There are many incredibly academically bright kids who just happen to come from poor backgrounds or black backgrounds. When I am thinking about tutoring, it is about getting everyone over the basics but it is also about enabling those who are academically orientated to get into the universities and so on. For me there are two gaps that we need to think about. There is getting everyone so they leave school able to prosper in life and then there is a different issue about whether you go into academic or vocational. Ultimately it has to be about the lifelong impacts on these children.
Q474 David Johnston: My question on the social aspects of this was covered by Zubaida. Lee, what do you think the other non-school actors should be doing here in how they assess young people? Thinking about universities and employers, how should they be adjusting what they usually do in view of this loss of learning?
Professor Elliot Major: That is a great question. Universities are going to have to think about lowering grades further for those students who can show they have been particularly disrupted by this—and employers. You will know, David, that employers often use crude proxies for selection criteria. It is absolutely crucial that it does not all fall on schools, that we challenge both universities and employers to be far more creative and radical in how they judge talent.
Q475 Tom Hunt: I very much appreciate that there is going to be a different session on this, but I want to touch upon something that was said earlier, which is the possibility that even in September we might not be in a position for all children to go back to school—I find that a horrifying prospect. Frankly, I am concerned enough as it is about the time until September.
This is to Kevin primarily. I know there is not complete agreement between your union and the Government about when it is appropriate to go back and so on. If in your view you did not have any more confidence than now regarding safety, and it got to September and October, would there ever be a point where you thought, “This is just getting ridiculous. This is terminal for the life chances of many children. We just need to go back”? Just out of interest
Kevin Courtney: The question is to be led by the science on this, isn’t it, and to be led by SAGE and other scientists looking at these matters? We want to get back to schools as soon as it is safe to do so. We recognise the point that Robert made on the radio this morning. It is not children who are affected by this, fundamentally, and the chance of any individual child being badly affected is very remote. It is a question of the impact of schools on community transmission, and it is a question of the effect on vulnerable adults. If a child is living with a grandparent, there is the vulnerability of that grandparent. All of this turns on the science. I really hope that the science will develop into a place—I think this is a faint hope, though—where children do not transmit. If children did not transmit, it would be unproblematic to open schools, but that is not what the scientists are telling us.
We hope to be back with wider opening. We have to prepare for the eventuality that we are not back at wider opening, but we are not being obstructive to wider opening. We have talked about five tests that are tests that can be met, and we want the Government to work with us on how to meet them, as the Government in Denmark is working with the Danish teachers, as the Scottish Government is working with the Scottish teachers, on how to reach those milestones.
Chair: We are going to have the bulk of this discussion, a big discussion, next week.
Q476 Jonathan Gullis: Kevin and I can spar over the NEU and its response next week. I am sure Kevin does not know I am a former union rep, so I look forward to having a discussion as a schoolteacher.
The point that has been made about summer schools is absolutely common sense to me. Zubaida has made some excellent points about, first, the school building being probably one of the most underutilised buildings in any community, but we also know there are plenty of other spaces. I speak to my head teachers in Stoke-on-Trent North, Kidsgrove and Talke and they keep saying, “We need a clear plan for September.”
I understand the anger that some parents feel about why are all primary students not returning now. I have tried to explain, and had some backlash on Twitter, that I do not think it was achievable after seeing what schools would look like while we have the 2-metre rule still in place, until we go down to 1 metre like they do in Europe. We have also seen the impact—it is a logistical nightmare and schools need time. You cannot announce something today and expect by next week for that to be the case. I think it was right, the cancellation of all year groups coming back apart from the key ones, although I would have brought back year 5 rather than year 6 because I agree with Lee that it is the next year group that is going to be more traumatised than the current one.
Therefore, I would absolutely urge you guys to keep pushing for the summer schools programme. As a former teacher, I am prepared to come out, work over the summer holidays with students in my community and volunteer in any way I can, because if we do not we are going to fail an entire generation of people. We have to use the school building for mental health wellbeing, health checks and education during the next 12 months definitely, and make sure that third sectors can come into schools, use the building for free after school to do further engagement for mental health, socialisation and education. I want to thank you guys for backing the Committee’s calls on that.
Dr Haque: I will be very quick because I am conscious of time. I absolutely agree with you, Jonathan. The reasons Independent SAGE proposed the summer schools programme are, first, we consulted with parents. There was an overwhelming response from parents about the fact that their kids, their particular year groups—other than reception, year 1 and year 6—were not going to go back to school, so what about all those children who are being left at home? Then there are parents who were saying, “My child was already struggling in school, and I am particularly concerned.” The option or the choice should not be whether to be in school or whether to be outside of school, whether to be in school and get some learning or to be outside of school and lose out on learning and lose out on social interaction. That is why the summer school programme is important.
The other reason the summer school programme is important is, even when we return in September, as the Independent SAGE group we have said that the risks are still there. We do not know if there will be a second wave. If this is planning ahead, this is planning ahead by testing and trying the summer school to make use of all the non-school venues. If in September we have a second wave or the coronavirus cases are still high—and they will be in some areas more than others—we already have a system that we have tried. We need to start now. The Government need to be much more creative than they have been.
Q477 Chair: Kevin, before you answer the question, if the social distancing rule was changed, would you be more willing to have schools open earlier? That is the only question I am going to ask on that this week.
Kevin Courtney: If the science suggested that you could reduce the social distancing distance without that impact on R, you clearly could put more children into a class, so that is a key determinant. But at the moment the Government are not saying that we have to follow social distancing rules in schools, and that is a problem to us. We do not understand how you can have social distancing everywhere else and not have it in school situations.
I understand that with early years’ kids you could not expect them to be socially distant, so the idea of putting them in a small bubble where if the virus is spreading it only spreads in that small bubble is a workaround. The question of how the social distancing works is a question where we have to turn to the science for answers.
Q478 Chair: We will have a very, very vigorous discussion. We have all the union representatives coming next week, so I look forward to next week. Thank you all for really important evidence. It seems there is pretty much consensus on the summer schools, as I understand it from you.
Professor Elliot Major: One thing I would add on summer schools is that the big challenge, whenever I have looked at evaluation of summer schools, is getting the kids that you most want to turn up.
Q479 Chair: Can you answer that? How do you do it?
Professor Elliot Major: You have to have them in areas that are most disadvantaged. You have to write to the parents. It is very difficult. Every evaluation of a summer school programme I have seen—
Dr Haque: It has to be done in consultation with parents. One of the astonishing things with the whole schools reopening issue is that parents, schools and head teachers have not been consulted, let alone GPs and so on. You need that local consultation. Lee is absolutely right that you need to ensure that you have the children who need the most help getting that help.
Kevin Courtney: Students coming out of training to become teachers would be a good source of people who could be employed on summer play centres. I agree with Lee and Zubaida about concentrating on getting the kids there who will most benefit.
Chair: Working, presumably, with social workers and teachers to get the parents to come in.
I really appreciate it. Thank you very much for your time. It was really valuable.
Examination of witnesses
Q480 Chair: Good morning, everybody. Thank you very much for coming. Could you kindly introduce yourselves for the benefit of those watching and the tape?
Sally Collier: Good morning. I am Sally Collier, and I am the Chief Regulator at Ofqual.
Dr Meadows: I am Michelle Meadows, the Executive Director for Strategy, Risk and Research at Ofqual.
Tom Bewick: I am Tom Bewick, the Chief Executive of the Federation of Awarding Bodies, which is the representative body for the regulated community in England.
Q481 Chair: Thank you. We quoted some stats in the recent session, which you were kindly listening to, that suggested that the grade predictions usually disadvantage pupils from BAME and disadvantaged backgrounds. You have produced surveys saying that, according to your own equality assessments, this is not the case because this is a different situation. Could you explain why, please?
Sally Collier: First, in terms of the systems this year, of course we all wish that exams had been able to go ahead. As your previous witnesses said, these are truly unprecedented times. The systems for both general and vocational qualifications have had to be redesigned at incredible speed and we think the systems this year are absolutely the fairest possible in the circumstances. As I will throughout this hearing, I will ask Michelle Meadows, who is our lead on this—she understands the research and is overseeing the development of the standardisation model—to take the specific questions about understanding the research.
Dr Meadows: As part of our equalities impact assessment, we did a rapid review of the literature for the evidence of bias in teacher predictions of A-level grades used for university entrance. Teachers’ estimates are grades that are used to inform exam boards, setting grade boundaries, and teacher assessment in the round. What you see when you look across this literature is that the evidence for bias is mixed. It depends very much upon the context in which teachers are making the judgments. There is some evidence of bias. For example, in A-level grades, there is evidence that bias with regard to ethnic minorities interacts with the ability of the students. For the most able students, there tends to be under-prediction of the grades that students go on to get. At lower levels of ability, you get the reverse effect where there is some over-prediction. These effects are nuanced, they are complex, and we are very aware of them and have been putting in place safeguards in our guidance to teachers.
Q482 Chair: Can we bring in Tom on this as well? How confident are you that the grade calculations and standardisation process eliminates the bias against pupils from low-income backgrounds, special needs, disabilities, BAME, or any other disadvantaged group?
Dr Meadows: The most important thing is to try to ensure that the judgments that come into the exam boards before standardisation are as unbiased as possible. Our guidance to centres emphasises the need for looking back over evidence and data. We require more than one teacher to be involved. It is not just left to an individual teacher. Teachers must work with the subject lead or head of department. There is head of centre sign-off. They need to declare that they are fair and honest representations of students’ likely grades. We have put in place additional guidance to teachers about how to avoid some of the human heuristic influences on judgments, how to avoid contrast effects, recency effects and so on, those ways in which bias can creep into judgments.
Q483 Chair: What modelling have you done—I appreciate this may be very difficult—to show that what you are proposing will work, or is that not possible to do?
Dr Meadows: The centre assessment grades will be in by the end of the week. As you can imagine, one of the first things we will want to do is look at what the outcomes would look like from those grades for students with different protected characteristics and how they compare with previous years. Then there will be a standardisation process by which we will adjust outcomes for schools and colleges to set a fair standard, a level playing field. Then we will want to look again at what those differences look like compared with what we see in historical data.
Q484 Chair: To what extent will extenuating or mitigating circumstances be taken into account in the calculated grade process? Sally, as the head of Ofqual, I hope you will answer some questions today. We do want you to answer questions. That is why we invited you.
Sally Collier: Yes, of course. I am very happy to answer this one.
You asked how the processes this year will take extenuating circumstances into account. Of course the predicted grade process asks teachers to predict what their students would have achieved, and they can take into account some of those circumstances when they make their predictions. Of course, they have to have evidence of some of the work that has been completed, but they know their students and they can take into account some of their circumstances.
The other point in the calculated grade process is that we ask teachers if their students would have had reasonable adjustments, for disabled pupils for example, and that they should give their prediction on the basis that those reasonable adjustments would have come into effect. There are safeguards in the prediction process itself, and then there are all the measures that Michelle outlined that can be taken once the grades are in, in the standardisation process, and then, of course, in the evaluation afterwards, as many of your previous guests talked about.
Q485 Chair: Tom we will come to vocational qualifications later, but how do you feel the new system will affect disadvantaged pupils? What can be done, and what mitigating circumstances should be taken into account?
Tom Bewick: Thank you, Committee, for the opportunity to come and give evidence today.
To the central question about whether a teacher-led calculated model copes with bias in the system, the empirical evidence, which you discussed with your previous guests this morning, shows that it is far from ideal to have a model based purely on teacher-led assessments. What you have heard from Ofqual, and what we hear from working with our members, is that the mitigating circumstances are being put in place, where at least there is external moderation, and ultimately the regulator will look at longitudinal data, past generations, to ensure that we are fair to them.
But ultimately it is worth making the point for the public record—and I think the Chief Regulator hinted at this in her response to you—that, in the circumstances, this is effectively the least worst option that the Secretary of State and the regulator have had to introduce. Does it deal with all the issues of inequity in our education system, the attainment gap, educational disadvantage and all the things you were discussing just a few moments ago? No, it does not. We have to be careful here that we do not conflate the process that is under way to deal with this particular cohort of learners this summer to ensure that they are not disadvantaged, that they achieve their results on both 13 August and 20 August for the GCSEs, and these broader issues about educational disadvantage in the system.
Q486 Chair: Sally, can I understand the resources? As I understand it, you will be looking at every single school in the country, in essence. What extra resources are required to do this? It is obviously a huge effort that you have to undertake.
Sally Collier: We have set the rules for the system. The Secretary of State sets the policy on how exams should be run this year. We turn the policy into implementation. The first stage is working. Exam boards have developed new IT systems in a small number of weeks, which went live last Monday, as planned. As Jonathan pointed out, tens of thousands of teachers around the country are inputting their grades. That is done by the schools—some very hard work goes into it by Jonathan’s wife and many others—and the exam boards. We have small numbers of experts who will be looking at the data and monitoring and regulating all the boards for vocational qualifications. We have our usual amount of resource for doing this work, and of course it replaces what we would normally be doing in any normal summer.
Q487 Chair: In The Sunday Times it is suggested that schoolchildren may have disruption in next year’s exams and that Ofqual has been drawing up contingency plans for next summer’s exams, which may be delayed from May to July, or that you might have teenagers not sitting exams for a second year. Is this report correct? What are the contingency plans that you are looking at?
Sally Collier: As you would imagine, 2021 is at the forefront of our minds. There is this summer, there is autumn 2020 and then there is 2021. Schools will need to know at the earliest opportunity what they will have to teach next year, what Government want schools to teach and how it impacts on exams. Our role is to make sure those exams are fair, that they work, that they are rolled out and that results are issued on time. Of course, Government need to consider, and are considering, the impact of lost teaching and learning. We are looking at what that means for the exam system and what can be done. We will be coming forth, as with all the other policies that we have implemented so far, with a full consultation, a full equalities impact assessment, on 2021 in the coming weeks.
Q488 Chair: The previous consultation—understandably, by the way, this is not a criticism—was quite short. I think it was a couple of weeks. Is that right? Presumably, if this does happen, the consultation period will be much longer.
Sally Collier: Schools, teachers and exam boards need to know very quickly, ideally before the summer break, what will happen in September. For exam boards it is a bit of a longer time, but papers are prepared many months in advance, so we do need to get on with it. I cannot promise, Chair, that there will be a very long consultation, but we will give the maximum time possible. We have had a tremendous response to our consultation so far. We had almost 13,000 responses on the general qualifications.
Q489 Chair: How much extra is being spent on Ofqual to deal with the current situation? You mentioned that there are extra resources. How much extra?
Sally Collier: There is no extra being spent. We have our usual budget.
Q490 Chair: You will be able to manage all this work under your existing budget?
Sally Collier: Yes.
Q491 Kim Johnson: Michelle, in your introduction you mentioned that there would be safeguards in place for teachers in the changing process. What are those safeguards and who has oversight?
Dr Meadows: I was referring to the guidance we have put in place to help teachers protect against bias in their judgments. We have made some very practical recommendations about how teachers might do that.
For example, an organisation, Race on the Agenda, made a very good suggestion to us, that we suggest to heads of centres that they look back at previous data, such as A-level predictions for university entrance versus actual grades attained by different protected characteristics, and use that data to challenge themselves about whether historically there had been any evidence of unconscious bias. They would, therefore, be able to reflect on the judgments they are making this year.
We have put in place as much support and guidance as we possibly can, given that all the schools are operating in quite different contexts this year, to support those unbiased objective judgments.
Q492 Kim Johnson: This is a question to both Ofqual and Tom. Your current proposals do not protect students from being systematically disadvantaged. Zubaida in the first panel raised the need to utilise the public sector equalities duty. Your equalities impact assessment states that the evidence of the likelihood of bias in centre assessment grades is mixed, as you have just mentioned. Should it not be a priority to establish either way the likelihood of bias and to identify which groups are most at risk? Is more research needed, when should it be undertaken, and by whom?
Sally Collier: We take our public sector equalities duty incredibly seriously. Equality has been at the forefront of our thinking since the first thought of this new system. We have engaged with Zubaida’s organisation, the Runnymede Trust, with the Sutton Trust, the Social Mobility Commission. We have been talking to many groups about how to minimise bias. Michelle has set out all the safeguards in the process, as far as possible, that we can put in place to minimise bias. I will ask Michelle to talk about the particular research issues and whether there is any more that can be done.
Dr Meadows: The previous panel talked about the importance of evaluating what has been done this summer, and we are absolutely committed to that. I have already talked about how we will look at how attainment gaps vary this year compared with what we normally see under an examination system, but we will have a full programme of evaluation that we will publish in the autumn.
Regarding attainment gaps, we are surveying teachers about the process through which they are coming up with the centre assessment grades in different types of schools and in different scenarios, to understand the issues from their perspective, how they used our guidance and whether they found it helpful or not. We are doing case studies with teachers and focus groups with students to get under the lid of this year’s process, because of two things. It is really important that in the autumn we are able to show both the strength and the limitations of what we have done this year, and there is also learning for the future. Whenever something like this happens, we want to make sure that we learn everything we can for future examination series.
Q493 David Simmonds: This may be a question particularly for Tom. I have had quite a bit of feedback from further education institutions and schools with concerns about consistency in the way awarding bodies are approaching the allocation of grades. It would be useful to hear about the measures being taken to make sure that we are not seeing different approaches, for example in the use or not of work that has already been done and in the different cut-off dates by which those assessments can be made or not made.
Tom Bewick: It is worth highlighting and distinguishing consistency around the general qualification, which we have had a lot of discussion about in relation to GCSEs, A-levels and applied generals. A small number of boards are involved in delivering those qualifications, and they work closely with Ofqual and the Department through the Joint Council for Qualifications. Arguably, because of the sessional nature of those general qualifications and the fact they are more homogenous, it is possible to have more consistency and standardisation, and the relationship with schools and colleges in that regard tends to be more consistent.
I think what you are hinting at is some of the feedback you have had from colleges delivering the vocational technical qualifications. There are nearly 15,000 of those qualifications active, and 10,000 or so of those qualifications are within scope of this particular extraordinary regulatory framework. It is the nature of occupations in the labour market, and indeed of the niche awarding organisations that deliver those qualifications—they are not part of one homogenous organisation—that they will have different relationships historically with FE colleges and with centres. There has been a huge amount of work behind the scenes with Ofqual and with groups of awarding organisations who have looked carefully at their qualifications and put them into the relevant categories with regard to whether they are calculated grades, adapted assessments or, in a very small number of cases, are delayed. Awarding organisations have been issued with guidance from Ofqual and, in turn, guidance has been issued to centres.
I do not want to play it down and say that inconsistency and lack of standardisation is not a challenge—the centres were all working to incredible timescales—but it is an issue that has been addressed throughout this process.
Q494 Ian Mearns: I asked a similar question to the first panel about students only being able to complain about their grades if they have evidence of bias. Is that not an unfair threshold for individual students? If it is the only recourse that students have, how will we prevent it from becoming the preserve of the well-heeled, the most articulate and the worried-well educationally?
Chair: As I came in on Ian’s questions in the previous session—I hope Ian doesn’t mind—I find it incomprehensible to understand how a student, especially one from a disadvantaged family, is supposed to know if there is bias or discrimination in their grading results and then undertake some kind of appeal.
Sally Collier: The appeal system is very important and we have spent many hours with very many people trying to come up with the fairest possible appeal system in the circumstances. It is clear on the first issue that if a student could appeal on the basis that they simply did not like the grade the teacher was giving them, the whole system could have ground to a halt. The teaching unions, including Kevin’s, very much supported a formal appeal process based on something being wrong, if something had gone wrong with the system, which by the way is the case in any year.
Obviously there needs to be a route for a student who feels that they have been discriminated against, but as regards all the issues you raise about how you prove that, I do not claim to have any easy answers. However, I do know that the route needs to be available and I know that we need to work with the Department and with everybody in the system to make sure that students who feel that has happened to them are supported, that they know the route. Clearly it is an issue we have been focusing on and will continue to focus on as we run up to results days and beyond.
Q495 Ian Mearns: I would like some reassurance about how this will be done in a completely open and transparent way so that everybody can have a clear understanding that everybody is getting a fair crack of the whip. I am not convinced at the moment, Sally, I am afraid.
Chair: Yes, I don’t understand how it will work. If you are a well-heeled family, you are going to lobby your MP, you are going to know how to navigate the appeal system—speak to lawyers, possibly—and you might choose to do the resits, where you will have a better chance. A disadvantaged person is not necessarily going to have those advantages, is not going to be able to navigate the system, and they will not know. The system will be completely biased against them. Even if they do decide to do their exams in the autumn, who will be looking after them and teaching them, especially now we know that schools will not be back until September, if then? I just do not think enough has been taken into consideration by Ofqual in looking after these disadvantaged pupils and making sure there is a level playing field.
Sally Collier: This is an important issue, supporting those students to know the routes open to them to appeal. It is clearly a responsibility of Ofqual, as is what happens in schools when students get their results, that the process is as fair and as open as possible. Ultimately Ofqual is the regulatory arbiter. If an appeal goes through the system and students do not feel it has been fair, they can appeal to us. The process is in place. We need to make sure it is as accessible as possible.
Q496 Chair: Okay, so why don’t you explain it? If I think of a constituent in my area of Harlow, they feel they have been hard done by, they do not have access to lawyers and MPs and so on, what will happen in practice? Just explain how it will work, practically.
Sally Collier: If a student opens their results and they think, “What on earth has happened? I didn’t expect to get this result,” they go to their school and ask their teachers. The teachers can check what information they submitted for the student. The first port of call is: has there been some kind of mistake, is the result wrong? Of course, as your panellist pointed out, this is new, it is novel, there are over 5 million grades coming in and there could have been a mistake. There is a route through which the school—as in any other year—would put in their appeal to the exam board, the process would be checked, the evidence would be looked at and a mistake can be corrected.
Q497 Chair: Who helps them do that?
Sally Collier: The schools would need to help the students do that. What we need to look at is who else can help them.
Q498 Chair: Who in the school? Is it going to be the head teacher? Is it their form teacher? Who?
Sally Collier: I cannot answer for every school, but as in any year—and I do appreciate there is much more emphasis on the predicted grades issues this year—there will be different processes in schools to assist students who want to challenge their results.
Q499 Chair: Did you design this appeal system, or is it from the Department?
Sally Collier: We designed the appeal system.
Q500 Chair: What consideration did you take in trying to think how this would translate in reality to a pupil from a disadvantaged background?
Sally Collier: The whole reason for emphasising the route that students can take to challenge bias is why we allowed that optional route. These are live issues that are being worked on at present.
Tom Bewick: I share the Committee’s frustration around this. I think it is important to look at this as an appeals process that, in many ways, discourages an appeal because it is an appeal against a process as opposed to an appeal against a grade.
That said, you can obviously imagine a situation where there was mass appealing against actual grades, how that would blow a complete hole through the important role that Ofqual, as a regulator, has to play in trying to statistically moderate these calculated grades so that there is not rampant grade inflation, for example, but equally is fair to previous generations who have taken these examinations. It is a bit like when you get a parking ticket. You know that, in a sense, when you fill out the appeals form you are not appealing against the decision; you are having to appeal against whether or not the process of the parking ticket being levelled at you has been fair or not. That is not unusual, is it, in the public sector?
Q501 Chair: But what happens if the processes you have set in order for this grading to happen are fine but they have arrived at an unfair estimate? Where are the external checks? Should the exam board not be able to look at the evidence used to arrive at the estimated grade, for example?
Tom Bewick: Of course.
Sally Collier: Michelle, do you want to pick that up?
Chair: Tom, you go first, because you were speaking and I interrupted you. I apologise.
Tom Bewick: I can only speak for the exam boards and the awarding organisations. As the Chief Regulator said, this is a system that has been devised by the regulator. We are in the business of implementing it.
Where an appeal comes forward and, therefore, one has to look at the audit trail of these decisions, the examination boards of course will share with the regulator and with the centre concerned the evidence that they have used and how the calculated grade has been handled from the point of it being delivered from the centre to the examination board and onwards to the regulator. However, my point about ultimately in a calculated grade in the system, if you are going to be fair to previous generations who have taken these exams, and future generations as well, there is a requirement on the regulator to do that statistical moderation. That is why, I suspect, this appeals process is not one that is expansive, that enables students effectively to appeal the decision of the grade; it only really enables you to appeal the process.
Q502 Chair: Sally, when people appeal, what do they do? Fill out a form? Ring you up? How does it work, exactly?
Sally Collier: Not through us. Each exam board has its own process for taking an appeal, and they, as we speak, will be designing the new appeal process for this year because, as with everything else this year, it is brand new.
Q503 Chair: Each exam board could do it completely differently?
Sally Collier: Well, they have to follow our structure.
Q504 Chair: Yes, but does someone have to fill out a 20-page form, write a letter? What happens?
Sally Collier: That detail is being worked out as we speak.
Q505 Chair: Should there not be a fixed system across the boards?
Sally Collier: Exam boards will want to make sure, as they have throughout this process, that where they can, the look and feel of their systems is consistent. They will all want to make it as easy as possible for this to happen.
Chair: I do worry that this is a system for the professional classes and not for people from disadvantaged backgrounds. It will be interesting to see the stats on how many people from disadvantaged backgrounds know how to do the appeal and get change.
Ian Mearns: Chairman, I think we need absolute clarity on how this system is going to work, and it needs to be mapped out so that every parent, if they want to access some sort of complaint or review procedure, will know exactly how to do it. Do they go to the teacher? Do they go to the subject lead, the senior management team, the deputy head or the head within the school, or do they go to the chair of governors, the local authority, the multi-academy trust or the diocesan bishop?
Q506 Chair: Would Ofqual consider setting up a helpline so that any parent could ring, freephone, to ask what the procedure is for appealing against grades that they think are unfair?
Sally Collier: I am very happy to take that away and look at it. These are live issues, and we want to do everything we can to support all students, but particularly those groups of students that you mention, to access the system.
Q507 Chair: Is that a genuine takeaway, or is it just to make us happy today?
Sally Collier: The former.
Chair: I think a genuine helpline, where somebody could ring in and say, “What do I do?” would make a big difference. Ian, have you finished on this?
Ian Mearns: No, I have hardly started.
Chair: I know you have some questions on historical data and so on. Fire away.
Q508 Ian Mearns: As I have just alluded to there, we have a very fractured landscape in how schools are established nowadays, and there are concerns that some of the newer types of school will be disadvantaged by the use of historical results as part of the standardisation process. How will you ensure that the system is fair for the newer types of institution?
Sally Collier: I am going to hand that to Michelle, because the detail is very much in her bailiwick.
Ian Mearns: I am quite happy to have takeaway or home delivery. I don’t mind either.
Chair: We have had a letter from the free schools network, for example, asking how on earth they are going to be looked after and have a fair playing field.
Dr Meadows: What we are doing at the moment is testing a variety of different approaches to standardisation. We are using 2019 data and seeing which of these approaches most accurately predicts the grades that students actually got.
What we want to do is choose the approach that gives the highest level of accuracy, and of course we want that for all students. We are testing the impact it has on different school types, including free schools, independents, state schools, all of those different classifications—schools that are relatively new and schools that are more established. We are making sure that the approach we take to standardisation is fair across those different types of school, so that we do not inadvertently choose the approach that is best on average but that has an adverse consequence for a particular school type.
Q509 Ian Mearns: Are you confident, though, that that can be robustly done just by using the 2019 data as opposed to earlier years?
Dr Meadows: We are looking across lots of different qualifications and subjects. This is big data. So, yes. What we are seeing so far is that particular approaches that work well on average also work well for these different school types. This decision will not be in tension.
Q510 Ian Mearns: Is it a fair concern that has been expressed by the New Schools Network that the statistical model you are using might result in results being driven down?
Dr Meadows: There is concern, just as there is for students this year that—if I can use the analogy of students—there will be students who, frankly, would have pulled it out of the bag on the day and there would have been students who unfortunately would have had a bad day. Equally, there will be some schools who would have improved this year and there will be some schools who might have had a bad year this year, and of course we will never know. It is impossible to reliably identify which schools they would be. All we can do is make sure that the standardisation does not in any way systematically advantage or disadvantage a school type. What we cannot predict is the unknown, those changes that may well have happened if exams had gone ahead this year.
Yes, we too have heard from schools for whom this year would have been the year, but trying to evidence that, of course, is incredibly difficult.
Q511 Ian Mearns: When it comes to the resit situation—although, as Kevin Courtney pointed out, it is not actually a resit because they would not have taken an exam in the first place—how are we going to mitigate against the inherent unfairness of some youngsters getting more lesson time and tuition than others?
Sally Collier: This is obviously a big issue for any exam series that is going to take place in the autumn. I come back to the role of Government in ensuring that all those students who want to take exams in the autumn are given that opportunity. I think that is just as important as the appeals issue, because those students who feel they want to take an exam should be offered that opportunity. We agree with that, and that is why in our consultation on autumn 2020 we are requiring exam boards to offer exams in all subjects in the autumn.
It is for Government, working with schools and with us—the lead is for Government—to say how, in these extraordinary circumstances, those students who want to take an exam are supported to do so. Of course there will be different levels of support required for A-level students versus GCSE students. That is not just an Ofqual issue. It is an issue for Government in supporting those students to be able to take those exams. The Secretary of State has been very clear in his wish to have that autumn exam series.
Q512 Ian Mearns: When you are making all these considerations, is there anything that you are specifically doing to make sure that diversity is also part of your decision-making process in including people who have a much more deeply ingrained diversity perspective?
Sally Collier: All groups of students, particularly those groups of students with protected characteristics, whether they be disabled students or special needs students for example, are absolutely at the forefront of our minds in every part of this process, including the autumn. One of the particular considerations in our proposals for the autumn is that we are consulting on removing some of the non-exam assessments. That is specifically because students may have had a differential experience in what they have been able to do for the non-exam assessments so far—some will have completed work, some will not have completed work. We are also trying to look at what schools can actually do. For example, in PE, you will not be able to run assessments on team competitions. In drama, you will not be able to run assessments.
In all of those things, at the forefront of our minds is which groups of students will that impact most and how can we mitigate against it. Also at the forefront of our minds is how on earth schools are going to manage this.
Q513 Dr Caroline Johnson: You were talking about the appeals process and the Chair was making the point that the professional families may find it more accessible than other families might. What information do you have on the timeframe of these appeals? Will there be deadlines? Presumably there will be. If there are, how will you communicate those to pupils and parents, and how do those timetables relate to students’ needs to start either applying to repeat the exam, to start their university courses or to start their A-level courses?
Sally Collier: As in any year, the appeals process will need to run very quickly, prioritising those students who need the grades for progression, so prioritising A-level and other similar qualifications. It is not us who will be doing this, it is the exam boards. They have well established procedures for the tens of thousands of appeals that they run in any normal year. They will clearly be using those mechanisms. They have communication channels into every single school. We will be looking at what they are doing. We will be looking at their timescales and deadlines, and we will, of course, be monitoring the appeal situation and producing national statistics on appeals, as we do in any other year.
Q514 Dr Caroline Johnson: How will you communicate those to parents and children? What resources do you have available? Do you expect there to be, for example, more appeals this year than there would normally be? Do you have the resources to deal with them in the timeframe that is necessary?
Sally Collier: It is not our resources. It is the exam boards’ resources. They have considerable resource to do this because in any normal year they will have tens of thousands of appeals, and we will be ensuring that they attribute sufficient resource to this to be able to put those arrangements in place.
Tom Bewick: I think it is a really important question. There is a lot of detail required in the answer, so I will check, take that away and commit on record to come back with a note to the Committee from a collation of the exam boards involved in the qualifications concerned outlining what their appeals processes and timelines are and how it all works.
Q515 Tom Hunt: I am sorry to sound like a broken record, but I continue to have concerns about those with special educational needs. I make no apology for mentioning my own experience, because I was somebody who really struggled at school—dyslexia, dyspraxia—and I spent a lot of time struggling in the classroom because my brain operated in a slightly different way. I was not a very good classroom learner, I did things in my own way, and time and time again I surprised my teachers with my exam results. I am very concerned that, however much you do, many children with special educational needs will get predicted grades that do not reflect their true academic ability. Also, we have brilliant teachers who do their very best, and the majority of them do have an understanding of children with SEND, but there are concerns raised by some SEND specialists that there is not enough of an understanding of certain special educational disabilities within the education sector. I think it is incredibly important that we have SEND specialists potentially feeding in to predicting grades for those children who have those special educational needs.
Sally Collier: On that subject, we have specifically said in our guidance that that should be the case and that the special educational needs individuals should be involved in the calculation of those grades.
Q516 Tom Hunt: I also have a quick question about private pupils. I have had a couple of cases in my constituency—if this is the case in my constituency, I am sure it is the case in many other places across the country—of individuals, and one in particular wants to be a doctor but did not quite get the grades he needed last year. He is repeating the year, has been getting private tuition and has not been able to find an assessment centre anywhere that is capable or feels able to give him predicted grades, so he is going to be looking at having to do exams in the autumn. He has decided that having another year out is not going to be practical, so he is probably going to have to give up his dream to study medicine, certainly in the short term. What would your advice be to schools and individuals trying to find a way round that kind of situation?
Sally Collier: This is a very difficult issue. We have spent many hours with many people trying to come up with solutions for those individuals who are not in a school or college environment. There are many types of private candidates, from an adult who takes a GCSE as an interest to the candidate that you have just talked about.
First, our guidance to schools said that where heads can reliably get some evidence for those candidates, but it has to be reliable evidence, and maybe it is a pupil who has been at their school or maybe the pupil can come in and present some evidence—and the vast majority of respondents to the consultation agreed with this, that there has to be a level playing field and that there cannot be one system for private candidates—where schools can do that they should. But then we were aware of particular candidates who were not being accepted by a school, so we then worked very quickly with exam boards to get some additional centres, just specifically dealing with private candidates, doing really rapid assessments of the evidence, talking to the students’ tutors, where they had special educational needs talking to those, to get some more candidates through. The figures have just started to come through. I very much hope that the majority of private candidates will be able to progress as they intended.
Tom Hunt: Thank you for that very detailed response. I am sorry that I was not aware that you did have SEND specialists feeding into the prediction process. Perhaps I should have been aware of that. I am glad to hear that is the case. I really hope that in practice it does lead to avoiding a situation where we have those with special educational needs getting grades that do not reflect their true academic ability. Thank you for your responses.
Q517 Chair: Before I bring in David Johnston on the vocational side, I just want to understand on the vocational issue—Tom, perhaps you can answer this—in these exceptional circumstances, who has the authority to make decisions and ensure there is transparency and consistency between awarding organisations? If you just take the example of my own college in my constituency, Harlow College, they have come across the same qualification being assessed in different ways by different AOs and are unclear as to who has oversight to ensure that decisions are made effectively. I point to Ofqual. What adaptations are planned for vocational assessment in the next academic year, and when will the information be available? Will you have the authority to make the decisions that ensure consistency and transparency across the awarding organisations?
Sally Collier: Shall I start?
Chair: Tom, if you can come in, please, and tell me whether the Harlow College view is a general view.
Tom Bewick: It comes back to the point I raised earlier about the lack of homogeneity in vocational technical qualifications. We are talking first not about 50 or 60 A-level or GCSE subjects; we are talking about many thousands of qualifications. There are well over 140 awarding organisations that deliver those, sometimes because they are very niche, occupational sectors—think about veterinary practice, health and beauty—and that is why these niche organisations exist. So, of course, they have the individual relationships with the likes of Harlow College and other FE colleges. That is the system of awarding that we operate in England. That tries to explain why there is complexity in the system.
As for the point about the approach to this summer for vocational technical students and ensuring that they are not disadvantaged in receiving their results, we were involved in an extremely intensive exercise over the last few weeks with Ofqual and the awarding organisations, sifting through thousands of these qualifications. Unlike the A-level and GCSEs where it is a straight decision around, for example, going down the calculated assessment route, there are three types of approach to vocational technical qualifications.
Obviously where they lend themselves more to the GCSE and A-level models, such as for example the BTECs and Applied Generals, there is a calculated approach. Where they are more about competency and being able to show competency through assessment, that is where you get into models such as the adaptive assessment. Indeed, there are still some qualifications in vocational technical—and Harlow College and others will perhaps be responsible for delivering these—that are really about licence to practise and are about the safety of an individual to go out there in the workforce and practise their particular skill or trade. With the best will in the world, it is not always possible to estimate, for example, whether somebody is safe to go out and drive on our roads or whether someone is safe to operate some equipment. One needs to be able to assess that skill practically, to be able to observe it visibly and report back to the awarding organisation.
Chair: Thank you. That is very comprehensive.
Sally Collier: I will start with your question about who oversees this. As the regulator, we oversee all of those 140 bodies. We will be taking a risk-based approach, as you would expect, on 140 bodies to ensure that our rules are being followed and that any mistakes that happen are put right as quickly as possible.
Our overriding principle for vocational qualifications, as with general qualifications, is that as many students as possible progress, whether that is to further education or into employment. Of those 14,000 qualifications, we think about two-thirds will see students progress either because they are getting a calculated grade or because they can have an adapted assessment where the method of assessment changes, maybe moved online for example. The vast majority of students will progress. There will be a minority, as Tom says, who, mainly for safety reasons or because employer professional bodies say we need to have a practical assessment, will have to be delayed.
In response to your question about when they will know, all of this is happening at the moment, as well as what is going to happen in the autumn.
Q518 David Johnston: Tom, you said that Ofqual should make the case to the DfE for a capacity building fund to help those awarding bodies for vocational qualifications. What exactly is it needed for? Do you have a cost in mind?
Tom Bewick: Yes. The Chair asked the question of Sally earlier about whether Ofqual had had to expend additional resources, and she replied saying that within Ofqual’s budget, which is set by the Department for Education, she is able to deploy resources to deal with the exceptional situation we find ourselves in. Of course, awarding organisations are not arms of the state; 60% of them are charities and around a third of them are for-profit or commercial organisations. They are already covered by Ofqual’s recognition of general conditions, so this is an additional regulatory regime that has been brought in for the circumstances that we are completely aware of. Effectively, awarding organisations, which are not taxpayer grant funded by the Government, essentially earn their keep by providing these important services to the education and skills system as part of an overall ecosystem of qualifications and examinations.
What we have said to the Department, and also to the regulator, is that there are additional costs of implementing the Secretary of State’s two ministerial directions, one for general qualifications and one for the vocational technical qualifications. We have put in an estimate to the Department, based on a focus group of mass market, mid-sized AOs and niche AOs, as to what that amount would be, and we came to a ballpark figure of £16.3 million. It is important to stress that this is about putting in an estimate. It is not a bill. It is not a demand of the Government to pay our members that amount of money. It was a piece of work that was requested by senior officials within the Department for Education. We shared that work, as Sally knows, with the regulator.
We would expect some kind of grant fund to be made available based on the evidence we have submitted, but ultimately we accept that it is for the Secretary of State to decide on what criteria he would pay out against a claim for these additional activities. In any case, it may be that some awarding organisations or exam boards decide not to make a claim. Charities, for example, may decide to absorb some of these costs within their own resources. That is a decision they can make, of course. However, I do think the principle here, that the Government are essentially directing our members to deliver a parallel regulatory regime alongside the existing regime, will incur additional costs.
Q519 David Johnston: Is it your position that if they do not get those additional funds, they will not be able to meet the timeframes for assessments and calculating grades?
Tom Bewick: I would not want to conflate those two issues. My members are committed, public-spirited bodies. They want to ensure, as does the regulator and the Department, that learners this summer are not disadvantaged, and they will continue to be professionally focused on that piece of work.
Just one more quick example of why this request has gone in and why it is additional: some of my members would have availed themselves of Treasury schemes, such as putting some of their staff on furlough. Obviously if you are called back from furlough, or indeed if you cannot go on furlough because you have to burn the midnight oil—literally, in some cases—to ensure that we are meeting these incredibly tight timescales, financially that is a hit for my members, which is arguably not a hit for other businesses, organisations and charities in the economy that have been able to avail themselves of the Chancellor’s Treasury support schemes, such as the job retention scheme.
Q520 David Johnston: Turning to the autumn resits, is it your view that exams should be offered in all subjects by all boards?
Tom Bewick: That is a decision that the regulator, talking to the examination boards, has already made and we support that.
Q521 Chair: Sally, do you want to comment on any of that?
Sally Collier: Perhaps just on the financial landscape. Of course we appreciate that the awarding bodies are facing the same financial challenges that many others in the country are facing, and there are some costs in the new regulatory regime. However, this is necessary to get fair grades for as many students as possible this summer.
Q522 Chair: Does anybody else have any questions? I have a question from Kim, who sadly had to go because she has a question in the House of Commons Chamber. She has asked: what is the diversity of both your organisations, at a senior level particularly?
Sally Collier: At a senior level—and I am very happy to write to you with the breakdown—we are very good on gender; we need to do much more on diversity and representatives from ethnic minorities. We launched a diversity and inclusion plan earlier in the year. I am very happy to come back at a later hearing.
Q523 Chair: In rough figures. You must know roughly the percentages among your senior employees. Is it more than 5%, 10%? What?
Sally Collier: I am not going to quote a figure at you because I will probably get it wrong. What I do know is that as a whole in Ofqual, representation from ethnic minorities reflects the local population in Coventry, where we are based, but it dips at senior levels. That is why we have had our diversity and inclusion strategy and I can talk a lot about what we are doing on recruitment, what we are doing across the organisation, but maybe now is not the time.
Q524 Chair: Kim Johnson asked specifically about senior levels.
Sally Collier: I am very happy to write to you with the specific breakdown.
Q525 Ian Mearns: I think that what Sally has just told us about the operation and the work that needs to be done has a particular bearing. I wonder if Ofqual, therefore, needs to reflect on the answers that were given earlier in consideration of what really is an admitted weakness of diversity in the management structure.
Sally Collier: From day one in this process, we have sought to engage the relevant experts throughout. We have had incredible representation from organisations representing ethnic minorities, from the Sutton Trust, the Equality and Human Rights Commission, so I do not accept that we are somehow deficient in the arrangements we are making this summer. I think we have been incredibly engaged with those organisations. We have taken on board many of their comments, and we continue to engage with them to ensure that this system is as fair as possible for all students this summer.
Q526 Chair: I worry that it is just not off the top of your head—and I mean this very politely—the figures for diversity at your senior levels. That you cannot tell us, that you have write to us, is a little bit surprising, and I do mean that very politely because I respect the work you do, as you know.
You said you were going to write to us about that. Could you also write to us to say whether you can genuinely, seriously consider the setting up of a helpline for every child or parent who needs to contact you to advise them on how to appeal and to make sure that they get a fair deal from their various exam awarding bodies?
Sally Collier: Yes, very happy to do so.
Dr Meadows: I think it is worth the Committee knowing that we do have a public helpline that has been open, including across weekends and bank holidays, since exams were cancelled. We have taken thousands of calls, as you can imagine, from parents and students, and we will continue, obviously, to have that helpline open. But as Sally says, we will take away whether there is more we can do, through the helpline, to make sure that people understand the appeals arrangements.
Tom Bewick: I can answer that question if it is about the Federation of Awarding Bodies, the organisation that I lead. There are six members of staff, three are senior managers. I am the only male in the organisation, so the gender balance is in favour of women. Until recently we had one BAME member of staff, but I think all six current members of staff would classify themselves as white British.
Chair: Thank you. I very much appreciate the work you are all doing. I know you have had sustained questioning, and I know it is an incredibly difficult time and you are working against the odds, so please don’t take the questioning as any underestimation of the tasks you have. I thank both of you, Michelle and Sally from Ofqual, for answering the questions. Tom, it has been great to have you as well. I should declare an interest. I think I am on your skills radio show tomorrow afternoon, if I am not mistaken. I wish you all well and thank you for all that you are doing to help at this time.