Education Committee
Oral evidence: Exams for 15-19 year-olds in England: Follow-up, HC 143
Tuesday 10 June 2014
Ordered by the House of Commons to be published on 10 June 2014.
Written evidence from witnesses:
– Ofqual (EXA006)
Members present: Pat Glass (Chair), Neil Carmichael, Bill Esterson, Caroline Nokes, Mr David Ward, Craig Whittaker.
Questions 1-84
Witnesses: Glenys Stacey, Chief Executive and Chief Regulator, Ofqual, Amanda Spielman, Chair, Ofqual, and Dr Michelle Meadows, Director of Research and Evaluation, Ofqual, gave evidence.
Q1 [Pat Glass (Chair)]: Good morning and thank you for coming on this lovely day. We wanted to meet, as we do periodically, with you, Glenys and Amanda—and welcome, Michelle, because it is your first Education Select Committee—and just touch base with you about where we are. It is a very difficult and tight timetable that the Government has set and what we wanted to do was to feel with you where you are in that. This is a very tumultuous time, is it not? There is an awful lot going on—an awful lot of changes. A lot of this is not in your control; it is in the control of awarding bodies or within the Department for Education, who are setting content at the moment. Do you have a plan B if this slips—if something goes badly wrong in this very tight timetable of massive change?
Glenys Stacey: First of all, you are right; there is a lot happening, and we have been very busy indeed in overseeing GCSE and A-level reforms. You will know that we were able to be extremely influential in determining the rate of change, particularly on GCSEs, where we are simply proceeding for the moment with English language, English literature and maths. We would say, in retrospect, that that was a good decision and it has enabled us to focus and to get those qualifications and the rules for those qualifications right. We have real strengths now in programme planning and overseeing such matters and we hold the ring, if you like, with DfE and with exam boards. We have a monthly management board where the chief executives of exam boards come around the table with us, with their senior advisers, and we are making sure that the right things are happening at the right time. If we had any significant slippage there, we would be the first to put our hands up and say that there was, but we have not.
At the moment, we have specifications in from exam boards for accreditation, as we expected. Our aim is that, should those pass muster, they will be in schools at the start of the autumn term this year. I suppose the risks now, for that first tranche of qualifications for first teaching in September 2015, are whether they pass muster and whether they will get through the accreditation process. That is now underway. In a sense, we will wait to see what comes out of that, but certainly we have got to that stage being very clear with exam boards about the quality of the qualifications and the assessments that we expect.
Q2 Chair: If they are going to be in schools in September 2014, what if they do not pass muster? Are you checking these things as you go along? Is there a contingency plan? Do we fall back on existing exams?
Glenys Stacey: First of all, we are aiming to get them in schools by the beginning of the autumn term 2014—that is why we have got accreditation happening now, starting in June—but they are not for first teaching until September 2015, so we are trying to get them in 12 months in advance. It may well be that some but not all of the specifications get through this first round of accreditation. We have got four exam boards, each applying for each of these subjects, so some but not all may get through that first round, but there will be repeat opportunities for boards to submit revised specifications for us to consider again. It really rather depends on the nature of the problem. It might be a small problem in the guidance or something more significant—for example, a revision of the mark scheme—but it should still be possible for boards to re‑submit a month hence or two months hence.
Q3 Chair: So the contingency plan, in a sense, is getting it early.
Glenys Stacey: Absolutely. We are getting it early because we think that is the fairest thing for schools, so that they have as much time as possible to prepare. We would expect, as we have said publicly, to get these in the autumn term. We would like to get them in when people come back from the summer vacation, but if they do not get through their accreditation then it may be that some tail-enders, if you like, come in during the autumn term.
Q4 Chair: On Monday, you launched a survey to gather teachers’ views on their experiences of the strategies used by schools to maximise students’ results. Why have you found it necessary to do that? Why was that?
Glenys Stacey: We thought long and hard about it and there are a number of drivers. One is that the number of straight malpractice cases that we are notified about or become aware of is increasing. That is absolute breaking of the rules. We suspect that there is more malpractice than we see. Much of it will not be reported. We have a concern about straight malpractice, but equally we are concerned about some things that happen that are not straight malpractice but it is not clear whether they are good or bad things. The teachers and, indeed, head teachers that we speak to make it plain to us that there is a good deal of confusion around where lines are drawn and also a good deal of concern about what might be happening in the school down the road. We are concerned about those issues because if things happen on a large enough scale, experience tells us that that can affect the validity of results and the educational value of qualifications for students. Thinking about it, we are not being condemnatory and we are not waving a big stick, but what we are saying is, “Can we have a discussion about it?” To aid that discussion, it is helpful to know more as to what people have had experiences of so it is less anecdotal and a bit more evidence. We have got some but we are getting more.
Q5 Chair: Is it an anonymous survey?
Glenys Stacey: Yes.
Q6 Chair: So if you get a worrying amount back saying, “My head teacher is putting pressure on me to break the rules”, or “I am not sure which side of the line this falls”, and it is significant, where would you go with it from there?
Glenys Stacey: There are two things. I will ask Michelle to speak in a minute, as she is leading on it, but first of all, just to be clear, if someone comes back saying, “My name is X, my head teacher’s name is Y at this school and this incident of malpractice has happened”, then at that point we would investigate. It rather depends what material we get. Michelle, would you like to explain the approach?
Dr Meadows: Yes, that is right. The survey does point as well to our whistleblowing policy. The survey is really about trying to get some firmer evidence about the scale of issues within schools and also teachers’ views about the acceptability of different strategies. We appreciate this is not hard science; we are not at the end of it going to be able to draw firm conclusions that, “In 50% of schools, X, Y and Z goes on”. However, at the moment there is so much anecdote and noise out there about what happens, as Glenys says, in the school down the road, so it will enable us to get a better picture of what is happening. This is also an initial step. What we need to do as well is, on the basis of this work, some more in-depth qualitative work—interviews or focus groups with teachers—to really lift the lid on the issues that come out of the responses to the survey.
Chair: Interesting. Thank you.
Q7 Caroline Nokes: We learnt from the University of Cambridge that the current reforms are characterised by a very “hasty rush”. Is it your view that there is too much happening too soon?
Glenys Stacey: Not in qualifications, no. If that was the case, we would be saying. Indeed, as you know, we have said before where we thought the timetable was too pressing. What is happening in the first year at GCSE is that just three subjects are being reformed. They are being reformed for first teaching in September 2015. They are the most significant changes, because the content of GCSEs is more demanding, the structure is completely different and the assessment will be of a better quality. So, the changes at GCSE are significant, but only three subjects are happening in the first year. At A-level, the main changes are moving to linearisation, de‑coupling AS and then some refreshing of the subject content, but there is no change in their performance standards, so those changes are of a different scale.
What I would also say is that when I go out to schools, I am quite rightly challenged by schools about how long the reforms might last—the totality of the timescale for reforms. People want them completed. There is this tension between not doing it all at once and getting it completed within a reasonable timeframe. We have set that to be three years, with most of it happening in the first two years, and schools seem to be accepting of that. There is a tension there to be managed, which we think we are managing reasonably well. The first tranche is relatively small. We are learning from that. The second tranche bites off 90% to 95% of student sittings, and then we have got a tail end, if you like, for the third year. It is a phased reform, trying to balance the need for reform against the time for doing it.
Q8 Caroline Nokes: You said earlier that you expected accreditation to be completed this summer and the information to be in schools from September, but do you have sufficient evidence that schools will be ready to teach from September 2015 across the board—so, GCSE, AS and A-level?
Glenys Stacey: It would be a bit early to get that at the moment, because we are in summer 2014. Our first effort is to get those qualifications specifications into schools very early so as to give schools the opportunity to prepare. It is right to say that we will wish to watch over the course of the year to see that schools are preparing, and we will be very keen to hear from DfE of the work that they will be doing to check that as well. Having said that, these are not new subjects. English literature, English language and maths have been taught in schools for a good long time and there are a good number of experienced people there. So, yes, we will be watching it, but our part is to get those qualifications in soon enough for schools to see the detail. We are discussing with ASCL other ways in which we can assist. For example, they will be running an event later this year looking at how one prepares to teach students a linear qualification and we will be providing informational material for those sorts of events. We will be playing our part, with as much information provision as possible to help schools really get to grips with the new specifications.
Q9 Caroline Nokes: I think it would be fair to say way back in 1988 there was a cohort of 16-year-olds who felt like they were guinea pigs when they took the first GCSEs. Do you think it is fair to say that children starting Key Stage 4 in 2015, with the introduction of the first stage of all of the changes, are also guinea pigs? Do you think it is fair on them?
Glenys Stacey: Forgive me, but that is an expression you could use for any change. I suspect that the question that we are addressing is: are these qualifications at the moment good enough or could they be much better and produce better educational outcomes for children? If they can—and we believe they can—then, in a sense, the sooner we are able to introduce those qualifications the better. There is no avoiding, in any change, someone being in the first year of the change. We have to satisfy ourselves that these qualifications are the right ones. Secondly, what we are consulting on at the moment is an approach to the setting of the performance standards for those new qualifications to recognise the fact that there is a tranche of students in the first year, and there is an ethical imperative to make sure that in the setting of those standards they are not adversely affected by that. We recognise some of the issues for the first-year cohort—particularly that we do not want them to be unfairly disadvantaged by the fact that they are in the first year. We are hoping to get good responses to our consultation on how those standards are set.
Q10 Caroline Nokes: Under the current timetable of reform, how are you going about guaranteeing coherence between Key Stage 4 and Key Stage 5 study? Do you think that this indicates that there is a disconnect between GCSEs, which are primarily used as a means of school accountability, and A-levels, which ultimately are the real school‑leaving qualification?
Glenys Stacey: You have got a couple of things wrapped up in that question. The first one is the coherence between GCSEs and A-levels. It is a difficult question. We know the subjects for the GCSE first and second tranche and we can recognise why they are selected: because they are critical subjects, EBacc subjects or other subjects that usually attract a large cohort. We can be satisfied of the rationale for selecting those first subjects and happy with that. Then, in the first and second tranche of A-levels, we have got similar sorts of subjects. The difficulty that you might be homing in on is that there is not necessarily a straight sequence of seven years of reform, where you are starting with Key Stage 3 and then Key Stage 4 and Key Stage 5. I suspect that that length of time would be very problematic for schools as well.
Given that some of these qualifications are very needing of reform, it would be quite difficult to wait. For example, in maths and English, we would be jumping up and down if we were not able to substantially improve those qualifications as quickly as possible. The cost of that is that there is a risk of a lack of continuity—some subject break, if you like. We know that exam boards, who are trying to protect their market share at both GCSE and A‑level, will be looking to smooth out the rather bumpy bits to make sure that their product range, if you like, is as coherent and cohesive and as attractive to schools as possible, but there is no doubt that there is a risk of a lack of coherence if we are not careful. That is fair, is it not?
Amanda Spielman: Yes.
Q11 Caroline Nokes: Do you think there would have been any value to pre‑testing or piloting the new-style qualifications?
Glenys Stacey: You would need to think about what that would involve. Is that a large-scale pilot for an entire cohort of students? What would that qualification then be worth? The days of large-scale piloting perhaps are gone. There are aspects of these qualifications that we can test and model, and that is a very helpful way of looking at it—for example, looking at and deciding on the maths tiering model and where the overlap would be; much statistical modelling helps with that. The other thing is if you pilot things on a small scale—and we have some experience of that—you find that in a pilot environment, things do tend to work rather well. One of the factors we are aware of there is they are not being set in the context of the accountability measures. Trying to think how qualifications will work in schools seems to us to be really important. Piloting how they will work in schools is enormously difficult, but thinking through, as we decide the design rules for these qualifications, not just what is good assessment practice but how it will play out in schools seems to us very important, and we have been doing that earnestly.
Q12 Bill Esterson: On the question about whether there should have been piloting, the DfE found—after the problems with the introduction of A2 exams in 2002, and following an inquiry by the predecessor of this Committee—that trialling should have been carried out and piloting should have been used. Were they wrong to say that? They seem to be indicating it was possible and it was a big mistake not to do it. Are we in danger of repeating the same mistakes?
Glenys Stacey: My first thought is that that was in 2002. GCSE English and maths were hard-wired into the accountability measures in 2005, if I recollect. Indeed, the EBacc measure now hard-wires other subjects and the new proposal for Best 8 hard-wires them as well. The world has changed, or at least the way qualifications play out in schools has changed, quite a lot since 2002. That would be my first thought. As I have said on piloting, it is rather difficult to work out whether you could pilot a whole qualification and in any way replicate the real world in which it is going to sit. In the real world, schools are judged on those qualifications and, as I say, we find that when things are piloted with small groups of students, it tends to be the willing, the able and the earnest who do that and there are no consequences for them in doing that. When you try to pilot something more large scale in the real world, it is much more difficult to get any reliable outcomes. I do not know, Michelle, whether you want to add anything to that.
Dr Meadows: I absolutely agree with that. We have had experience of piloting qualifications such as the functional skills, but from a research perspective I am not entirely sure that we learnt that much from those pilots that led us to change those qualifications in a significant way. We have got some more experience now of piloting. I would like to say, though, that those people who have been working in qualifications for a long time and have got a lot of expertise to bring, and also the research literature on how qualifications play out in schools—those folks with that expertise—have been very much involved in these developments. While we may not have piloted, we have not been doing this in isolation, not drawing on past experience.
Q13 Chair: When I visit schools—and schools can be quite contradictory—they tell me, “If there is a change of Government in 2015, we need stability, so we have to let these things settle in, but you must change this, this and this”. It is not always the same thing in the schools that you go to. There is a demand for stability and a demand for change. When these things were first announced, there was a great deal of concern and a lot of noise about the pace of change and the fact that this was all happening at the same time, the real danger being so many changes at the same time. The Secretary of State, rather cleverly, changed the name of “EBCs” to “GCSEs” and everybody sighed with relief, but the same change is happening. It is your job to balance this pace of change with the dangers, and those dangers still seem to be there. Do you still have concern about the range of changes and the pace of change?
Glenys Stacey: We have given an absolute priority to managing these reforms at Ofqual, as you would expect. Much of our business has been running these reforms. We have got 40 or 50 staff doing it. We are delivering daily against the requirements in our plan to make sure that these things happen and happen well. We have done that and the specifications are now in for accreditation. I am less—and I think Ofqual is less—concerned now that we have done it.
Q14 Chair: You have managed the risk through putting resources into it.
Glenys Stacey: Yes, that is right, and bringing a lot of expertise in but also drawing in exam boards and their expertise in a way that we have not done in past reforms, so that we can be as sure as possible that we are getting things right. The other thing to bear in mind is that, whereas in the past one might have done this and then launched qualifications and just left them, that is not our intention for these qualifications. We want them to work and to work well for students, and we will be tracking closely that they do. We do not want to wait until 2017 to find that we might have a problem with any one of them; we want to see how they work in schools to make sure that they are as we think they are. Looking at them periodically as they run seems to us to be enormously important and something that has not really happened in the past in a systematic way.
Q15 Chair: So you have the ability to tweak changes if things are not working well.
Glenys Stacey: Yes. If, for example, the assessments are not good enough year on year, we would be requiring them to be good enough. If we find that the balance of exam and non‑exam assessment is not producing sufficiently good educational outcomes, we can re‑visit that as well. It is about introducing very significant changes to improve these qualifications and then nurturing those qualifications. Like you and like those that I meet in schools, we do not want to see persistent change to these qualifications for anything other than the best possible reasons. We do not want change unless it is required for the best possible reasons. It seems to us that schools and the regulator have something in common there: that we want these qualifications to be good and then to be accepted and to be delivered in earnest. The impression that I get from schools and what people tell me is that once they have got the qualifications, they—
Chair: They are just going to get on with it and make it work.
Glenys Stacey: Yes.
Q16 Mr Ward: Before we move on from this issue of accountability measures, which has been touched on a couple of times now, your own investigation into the 2012 English GCSE found that “insufficient attention was given to the totality of the changes and incentives on schools” during the qualification’s design phase. We talked for a long time when you met us before about ensuring this sort of thing never happened again. Are we doing that? How is that being achieved when the accountability measures themselves are changing? You mentioned Progress 8. How do you hit a target where you are not sure what the impact of that target is going to be?
Glenys Stacey: If I just answer the piece of your question that relates to designing qualifications, and then we can look at Target 8. That is right; we did say that in 2012 on GCSEs. There, we had in mind particularly the place of non-exam assessment and the balance of non‑exam assessment to exam assessment. If you recollect, in that qualification, 60% was non‑exam assessment. When we have looked now at the new qualification, we have found, comparing it with the old, that there was quite a lot of overlap in what was covered in exam and non-exam assessment in English, which was not helpful and is not good assessment practice.
Also, we had particular issues with speaking and listening, as you know. We have thought long and hard about that. We have substantially changed the position on the type of assessment. We have taken out speaking and listening as well for English. Equally, we have thought long and hard in maths about the nature of the assessment; whether the qualification is tiered and the way it is tiered, where that tiering is and how to make that tiering equate where it needs to equate. Also, we have thought long and hard about how schools will make choices for less able, moderately able and more able students, whether it will be foundation or higher tier; we have tried to think through how schools will react to the choices that they have to make.
Then we have thought long and hard about how to best pin down, if you like, the quality of assessment, whether it is lower tier or higher tier. We have really strengthened the assessment objectives and tightened them up, and we have set in a number of what we call industry standards that all exam boards must meet so they have got a common, agreed approach to designing those assessments. Those are things like what type of calculator can be allowed or not allowed in a maths exam; what proportion of the assessment should be calculator or non-calculator; how you target different parts of the maths specification to less, moderately and more able students; and really tightening the assessment objectives. All of those things are really designed to make sure these qualifications are as good as they can be. We are predicting the choices schools might have to make, how they might make them and what they might take into account and, yes, much of that is very significantly informed by our experiences in June 2012 in GCSE English.
Q17 Mr Ward: This is a little bit like the Treasury and the tax specialists, is it not? There is a constant battle going on about what we can do to ensure there will not be avoidance, and then somebody finds some clever way. On the assumption that you will not get it perfectly right because there will always be innovation and creative ways, how quickly can that be identified and dealt with?
Glenys Stacey: This might help. Our approach is threefold, really. One is designing good quality qualifications that recognise the pressure schools are under. Secondly, we are trying to encourage a more open dialogue with schools about some of the choices they make. Our ethical review is the start of that. Thirdly, we are very much trying to advise and influence Government on the accountability measures. It is all coming back to the accountability measures. Because you asked initially about that, I wonder whether we could just deal with that.
Amanda Spielman: To take half a step back, the Committee commented previously that warnings had been raised within the QCA on previous GCSEs and we very much accepted that recommendation. It is worth saying that the board review of these reformed qualifications is quite broad. We cover the options rejected, the pros and cons of the things that are not being taking forward as well as the things that are, and the reasons for rejection to make sure that the risks associated with what we are deciding are very much understood, known to everybody involved and carried forward in risk registers. Then we work with the people who have raised concerns. Glenys visits many schools. We discuss very openly. With A-level science practicals, for example, we are continuing to work with Wellcome and Gatsby. We invest very significantly in continuing those feedback loops.
The effects of the accountability system are effectively one part of that. We have studied the new accountability arrangements—Progress 8 and Attainment 8 and so on—and it is clear that they will create a new and slightly different set of incentives. There may be good things—it will reduce pressure on the C/D borderline that has been so dominant in the last few years—but pressures will come in in other places. We have seen this year, for example, there is a new trend to enter students early for English literature in year 10, which seems likely to take advantage of the double weighting that is available where students sit both. There are a number of likely effects, and we will be monitoring those carefully and talking to schools about whether the changes that we see genuinely reflect desirable educational trends or things that people are less comfortable about so that we can, from our perspective, advise and adjust where necessary.
Glenys Stacey: Can I just add to that, David? You were saying that as you close one door, if you like, another opens. We suspect that as we tighten up on the more traditional ways in which exam boards competed that you have been concerned about as a Committee and we have been concerned about and these qualifications within each subject become more directly comparable, schools will be looking at their choice of subjects for students—looking in the Best 8 measures about the choices that they would make. They are also looking at and changing their entry patterns. That is something that we are keeping a very close eye on, most definitely.
Q18 Mr Ward: Moving to a new area, if I can, the National Children’s Bureau and the Council for Disabled Children have raised concerns about the effects of the reforms on particularly vulnerable groups, for example children with mental health problems, Travellers, refugees and other groups. I just wondered what, if anything, had been done to take into account the needs of these groups—the groups who do not get GCSEs.
Glenys Stacey: I am going to ask Michelle to speak on that in a moment, because it is an area of her expertise, but just to say to start off with that you will know that in all of the changes that we have made, we have done—I think very well-regarded—quality impact analyses to make sure that, for all recognised groups, we consider that carefully and make sure that our board consider them as well. Also, we recognise, as I am sure you do, that some of the known difficulties are dealt with in the way that individuals are able to apply, for example, for special consideration. There are mechanisms in the system to allow in some circumstances for those, but not necessarily dealing with all of the things that you have expressed a concern about there. Can we just look in a bit more detail?
Dr Meadows: In making any changes to qualification design, it always has an impact on the rank ordering of students—who does well and who does not—but there are some factors that people are very concerned about, such as gender differences and differences between social groups and ethnic groups, that seem to exist whatever the design of the qualification. I know, for example, there has been a lot of concern about whether the removal of coursework will impact upon on girls. Looking at the history of changes to qualifications, for example in English, where there was quite a dramatic reduction in the amount of coursework in the 1990s, the gender difference between boys and girls in results was remarkably unchanged; it was absolutely steady. Equally, the same has happened but in the opposite direction for mathematics. So much of what leads to differences in outcomes between these groups occurs far before it gets to the point of qualifications.
I would say, though, that there are some things that have changed that are quite positive. For instance, the fact that we are only going to have tiering in qualifications where it is absolutely necessary is a really positive step. There is a lot of research evidence that seems to suggest that tier entry decisions are not just based upon prior attainment and general ability, although that is the predominant factor; there are, indeed, social factors in those decisions too, so certain groups are more likely to be entered into the foundation tier than the higher tier, in a way that simply cannot be explained by their past performance. Removing tiering wherever we possibly can is a positive thing. There is some room for optimism.
Q19 Chair: David talked earlier about one door closing and another one opening. Head teachers are very clever people and they will, in my experience, simply move the radar around to wherever the latest accountability measure is. That is not just about exams. It is not just about what you do; it is about the behaviours that are created by the way in which we inspect schools, etc. It goes well beyond what you do. How often do you meet with Sir Michael Wilshaw and look at the impact of Ofsted and the exam system on changing behaviours within schools?
Glenys Stacey: I meet with Michael Wilshaw periodically—perhaps about once a quarter.
Q20 Chair: Is this on the agenda?
Glenys Stacey: We discuss our mutual worries.
Amanda Spielman: Yes, and we consider the totality of what happens in the education system and what should be overseen, and consider the points where our respective responsibilities touch and how to make sure that they neither overlap nor does a hole open up in between.
Q21 Chair: There is the threat that you can be a fantastic head in one school but, if Ofsted walk in tomorrow, you can suddenly become a failing head of a failing school. When those kinds of things hang over head teachers’ heads—that Ofsted can walk in tomorrow and you can be out the door—then they are going to change their behaviour, are they not, to match the accountability system?
Glenys Stacey: First of all, we absolutely understand the primary power of Ofsted; we are but nothing in the schools’ minds compared to that. What has helped us is a relatively recent move in Ofsted to regionalise and to establish regional operational leads and, indeed, a chief operating officer as well. We have those operational contacts, which are really important to us, and we are discussing at that level. For example, we are very keen to see our A-level arrangements in science for practical assessments to be delivered in schools and we are talking with Ofsted about how they can assist us in giving us any early warning of any difficulties with it. There are increasingly, as these reforms play out, opportunities for us to work at an operational level with Ofsted to make sure that there is the right exchange of information early enough. We cannot be the canary down the mine for schools, but Ofsted can play that role for us and it is really helpful.
Q22 Craig Whittaker: I just want to ask you about the changes in the entry patterns going into summer.
Glenys Stacey: For this summer?
Craig Whittaker: Yes, this year. Can we expect a car crash when the results come out?
Glenys Stacey: That is not an expression that we use, but there are a number of changes happening. Again, Michelle can help us with the detail, but in summary, on the entry‑pattern side of things, we have got much less early entry than we had last year—so more mature students and fewer year 10 students than we had last year—in the key subjects, which is interesting to us. Also, we will have some moves across from GCSE to IGCSE, and that is quite significant for us as well in looking at the cohort that is left taking GCSE. In terms of entry patterns, they are two big drivers. I am not aware of anything else to particularly alert the Committee to.
Dr Meadows: What this will mean is that results and headline figures might well look very different this year, but legitimately so. It might well raise concerns because this cohort coming through is simply very different to last year’s cohort, so we cannot expect to see very steady grade proportions; that would be worrying in and of itself.
Q23 Craig Whittaker: How are we going to deal with that? Last year there was an uproar about what happened in the English scenario, for example. How are you going to stop that scenario happening this year?
Glenys Stacey: Just to be clear, it was the year before last that we had an uproar: 2012.
Craig Whittaker: It was, yes.
Chair: It just feels like last year, does it not?
Glenys Stacey: 2013 was interesting for us. There were different changes in the cohorts again and moves to IGCSE. We were able to do a number of things, and we are doing just the same this year. For example, we were able to provide information ahead of results as to the sorts of changes that we were seeing that might affect how results would look on the day. We keep standards steady, but if there are changes in the cohort we can show it. So, the first thing is providing information. We have got early material on our website now. As we did last year, we are writing to each school to set out the main drivers that we think will affect the way results look at a high level. That is really quite important, and it was very welcomed by schools last year; they could begin to understand why things would look a bit different.
Secondly, we are very careful to work well in advance with exam boards to make sure that we have got a good understanding of what is happening. We have had our first formal meeting with exam boards already. We have set out what we expect by way of how they will oversee awarding. We have a data-exchange protocol set up so we get data and information in daily, so we can track what is happening, and that is very helpful. Indeed, we have a stats team who last year were working long shifts, and they will be doing the same this year so that we get to the point ahead of results day where we have got a pretty clear understanding of how these different drivers have played out. But until we go through awarding we cannot be absolutely sure, because there are some contradictory drivers this year. Less early entry means that you have got more mature students; on the other hand, we have got linear qualifications. You have got these things that may be driving results in different directions. What we can be clear about is that we will hold standards steady. It is our job to do that.
Q24 Craig Whittaker: So you are quite happy, then—or confident—that it will not be a car crash, because it has been managed much closer and in a much better way.
Glenys Stacey: Yes. I am very clear that we are doing the best possible job to make sure that standards are set and set right. What I am expecting is that there will be a greater variation in results school by school, because there is a combination of things happening and it really depends how the individual school, and often the individual teacher, has reacted to those things. Some of it will be schools’ policies in early or late entry or moving to a different qualification, and some of it will be, in geography, for example, whether the geography teacher has understood the detailed changes to the geography specifications, with more geography in them—whether he has got that and taught that. If you think that through, we can expect to see some variation school by school depending on their readiness and some of their strategies now and before.
Q25 Craig Whittaker: But no car crash.
Glenys Stacey: I have not anticipated a car crash.
Craig Whittaker: No, it is alright. That was unfair.
Glenys Stacey: What I am saying is, again, it will not be an easy summer, but I have not had an easy summer yet.
Q26 Craig Whittaker: Let me just ask you about IGCSEs. Pat alluded to schools changing the radar—I will not go as far as saying “playing the system”, but that is what happens. We have seen a 96% increase in IGCSEs this year and a 4% decline in GCSEs. Is that because the weaker schools are working the system?
Glenys Stacey: That is an interesting way of looking at it. I suppose the first thing to say is that you are right; these are relatively recent increases. A couple of years ago, IGCSE was just 1% or less of the cohort, so it was not a player, really. It is only in the last couple of years it has come up. Secondly, when we talk of IGCSEs—and I know that you will know this—it is not one qualification; there are any number of them provided by all of the exam boards plus CIE. They do not have a common rule set in terms of their content or design, so an IGCSE in English literature could not have any Shakespeare in it and could have a lot of coursework in it. There will be any number of reasons why schools choose IGCSEs. Some of them will be laudable from our perspective, because they can provide an opportunity for quite creative and interesting study, so I can understand why schools would choose them for those reasons. Also, it is true to say that for some schools and for some students having a good proportion of coursework and being able to submit and re‑submit coursework, or re‑work coursework, will be a particular attraction.
It seems to us that one of the things that we can do to aid schools in decision-making as we get new qualifications coming through is to provide more information in terms of the differences. There is a risk that some schools are making choices without a good understanding of whether something is similar or different and without a good enough understanding of how standards are set ultimately anyway, so getting more information is important. Whether or not IGCSEs are in a performance measure in the future is a critical issue, we believe.
Q27 Craig Whittaker: Yes. We have gone from, I think you said, 1% and, if my maths is correct, we are currently running at just over 6% in IGCSEs, which is a huge jump. Particularly with the huge push around rigour and standards, will they get pulled in if it gets too big?
Glenys Stacey: First of all, I cannot question your maths, but if it is wrong we will be able to tell you.
Craig Whittaker: Yes, I am sure you will.
Glenys Stacey: There is certainly a significant enough interest now in IGCSEs, particularly in English and maths—not so much in the other subjects, but they may well become of more interest with the new accountability measures, so we need to watch out for that. Secondly, as I have said, they are all different shapes and sizes. We could regulate to box them more and they would look more like GCSEs, but then they would be less attractive to those schools that choose them for laudable educational reasons or to those schools that choose them for other reasons. There is quite an interesting issue there for us. A first question for us, frankly, is: are they going to account for performance measures in future? If they are, then we will have very serious concerns about the simple range and variety, because we cannot assure exact equivalence. We are watching that with interest.
Q28 Chair: One of the things that the Committee noticed when we went and looked at curricula in East Asia was the lack of creativity within curricula. One of the real strengths of our curriculum has always been our creativity, the fact that it drives entrepreneurialism and ideas, etc. Is that in your thinking when you are looking at specifications? Are you looking at creating GCSEs, or agreeing GCSEs, that are going to have that level of creativity and ideas, that are going to lead to great businesses in the future?
Glenys Stacey: There are a number of dimensions to that and Michelle might want to comment, but just to start off with, for most of these GCSEs the subject content is agreed by DfE or in a process overseen by DfE, and our interest is in making sure that that is of sufficient demand. Generally speaking, there is more subject content in these new qualifications than in the past, which is not necessarily a bad thing. Secondly, we are very interested in seeing—and I suppose it is one of the benefits of the exam board competing model—how they then interpret that. We want that to be interpreted in a way that still demonstrates sufficient demand. Exam boards will be trying to interpret it in ways that make it an attractive and exciting product for schools to teach and students to learn. That will lead them towards looking at it creatively.
Chair: The ideas curriculum.
Glenys Stacey: We want to see good quality assessments. There is no inhibition on creativity or innovation there, but we do want assessments to be valid and to be of the right standard and to continue to be of the right standard, and we do not want exam boards competing on, “Well, our assessment is a bit easier than another assessment”. We do not stop any innovation there, but we do want to be sure that it is of the right level of assessment demand.
Q29 Chair: I want to ask a point of clarification, Michelle. Those of us who have worked in education for a long time have this view—I do not know whether it is scientific or not—that there are occasionally rogue cohorts. We do not know why; it is just something in the water that particular year. It is the same right through primary and secondary, and it is the same across schools. Did you suggest that about the 2014 cohort? Were you suggesting that it was a particularly difficult cohort?
Dr Meadows: No, I did not mean to suggest that. I meant to suggest that those who are certificating this summer are different in the sense that they will be predominantly year 11 students, whereas last year we had a mix of year 10 and year 11 students. There are structural shifts, rather than a suggestion that there is something particularly unusual about the pupils themselves this year.
Q30 Bill Esterson: I have a number of questions that come out of the previous two batches of questions. Just picking up the point about creativity, Glenys, you made the point that there is more subject content. It might be argued that by students having to learn more information for exams, there is less time available to develop the skills that encourage creativity and innovation.
Glenys Stacey: What perhaps you need to look at, then, is the assessment objectives. It is quite a technical area, but for each of these qualifications there is a stated purpose, which is common, and then they each have a set of assessment objectives that are set out in the curriculum. It is three or four bullet points. They are what we are seeking to assess and what a student should be able to almost do and demonstrate if they have completed that course of study. Those assessment objectives are specific to the individual subjects. They do not generally say “demonstrate creativity”. They will require the student to demonstrate a knowledge and understanding of the subject, but, as I say, they are subject-specific. In history, for example—I do not have it on the top of my head—there will be something around the analytical skills and the ability to join the dots and see themes.
Q31 Bill Esterson: Sorry to interrupt. Tell me how, in a written linear exam, you demonstrate creativity.
Glenys Stacey: The student will be asked to be creative in art, but, as I say, in, say, history, he is not being asked to be creative; he is not getting marks in a mark scheme dedicated to creativity. He is being assessed on the totality of the curriculum for that subject by reference to those assessment objectives. It is not just what he studied; it is whether he can demonstrate this particular knowledge base but also demonstrate some skills and attributes that are different for different subjects.
Q32 Chair: Would they be asked to provide illustrative examples and the long and short-term impact of these things—that kind of thing?
Dr Meadows: One of the things I was thinking as Glenys was speaking was that one fairly significant change with these qualifications is that there is now a greater requirement for students to give extended responses. We are moving away from quite short‑answer responses to an expectation of longer, more essay-like responses. If anything, that will require a little bit more creativity in responding and will no doubt wash back in the way in which students are prepared for the examinations.
Q33 Bill Esterson: I am sure we will return to that topic. I wanted to ask some more questions about IGCSE. In your submission to us, you made the point that one reason for the shift from GCSE to IGCSE in English language, for example, might be the changes made to remove speaking and listening from the final grade and to rebalance the assessment from 40% to 60% written exams. Is that a result of schools wanting to continue to use familiar teaching methods? Do you think that is the reason that is driving this?
Glenys Stacey: As you know, we made changes to the speaking and listening assessment because of the issues with the current GCSE that came to fruition in June 2012 and some of the evidence we had there of some strategic approaches to the marking of it in schools. Now we see more schools moving to IGCSE English and we have to speculate as to the reasons why that is, but certainly, if IGCSE English still allows speaking and listening to be part of a grade and GCSE English does not, one of the factors might be that there is still the opportunity to ensure that your student does well on speaking and listening.
Q34 Bill Esterson: Do you regard IGCSE as a good qualification?
Glenys Stacey: That is an impossible question to answer, because they are very different. Talking about IGCSE as one thing is a bit like talking about cancer as one thing. It is not. They are very different across the subjects and across the individual providers. That is the first thing. It is a group—a family, if you like—of qualifications that can be very different.
Q35 Bill Esterson: Are there good IGCSE qualifications?
Glenys Stacey: It depends what you mean by “good”. They can be very varied. You can potentially have one with a very rich subject content and a type of assessment that, if it is a well-motivated and well-taught student, he will really enjoy it, he will be stimulated, he will learn more and he will do well. Some of them have that potential.
Q36 Bill Esterson: I suppose what I am driving at is: if you have got parents listening to this discussion, should they be concerned if their school has made that change?
Glenys Stacey: I will give you one small example. As the Chairman said, all schools are different and do different things. We had someone at work with us recently who was on work experience, in the way that some of you have work experience people here this week. She was telling me—and, indeed, told our board—of her experience, which was that she started studying GCSE English literature. She is very keen on it; she loves Shakespeare. After the first term of study, her school moved her off that to IGCSE English, with no Shakespeare in the IGCSE that she is now taking, and the main text being a book that is a couple of years old that is described by this student as “easy”. That student would love to study Shakespeare and is not getting the opportunity to do so, yet the whole class has moved to this IGCSE. You do question why really, do you not? It is very different, according to not just the qualification but the reasons for the choices made by individual schools to go for one qualification or another. The motivations behind that are so important. Some schools will do it for very good, laudable reasons and will teach their students extremely well.
Q37 Bill Esterson: I shall move on to the new GCSEs rather than staying with IGCSEs. Talking to employers—this builds on the point about creativity—they want certain things, including creativity but also teamwork and interpersonal skills. This is a point you make in your own submissions as well. How can a system of GCSE be a strong foundation for further academic and vocational study and for employment? How can it do all those things?
Glenys Stacey: I would say it can play its part. If you are going to study A-level maths, it would help if your GCSE maths prepared you well for that. At the moment, there is a general consensus that it is a step too far; as you step into A-level maths, you are simply not prepared for the extent of the challenge and the hike up. A GCSE would have a key role to play in making sure that student is ready to take that next step and hits the ground running as they do so. It is also right to say that GCSEs, like any other qualification, cannot be all things to all men. They cannot do everything. An assessment is simply an assessment. As I have said before, we are looking to make sure that assessment assesses what is in the curriculum and what the assessment objectives say. Those assessment objectives do not say “teamwork”—there is not a GCSE in teamwork—and they do not cover interpersonal skills either. Indeed, it would be very difficult to design a national standardised assessment of teamwork or interpersonal skills that we could set any standards for reliably or validly. We have to recognise that a qualification is just one thing and an assessment is just one thing. What is really important is the education of students in schools.
Q38 Bill Esterson: Yet many employers rely on GCSE as a guide, do they not?
Glenys Stacey: Many employers, from our research, would use a number of GCSEs together as a good reason to call a student to interview. They will then recruit or not recruit the student according to the view they take at interview. Again, it is about how much weight you can put onto one assessment, really. As I say, we are working hard to make sure that those qualifications and those assessments do exactly what they say on the can. It is really important that we are very clear what its purpose is, what the assessment objectives are and what you can expect if a student comes out with a good grade in those qualifications, but it does not make him this fully-rounded human being who is absolutely employable in all respects. That is a bigger job for education and, indeed, for parents. It will do what it says on the can. That is the important thing.
Amanda Spielman: We have consistently said that qualifications are a partial mirror of education; they are not the totality of the curriculum. If the curriculum reduces to being seen as just the qualifications, it is not a good thing for education.
Q39 Bill Esterson: That potentially opens a debate about whether we need them to do something other than what they do. In some countries there is a school‑leaving qualification at 18 rather than the system we have, which is a partial 18 and a partial 16 system. With the changes in things like school-leaving age, does it imply that perhaps we should be considering that as a change?
Glenys Stacey: That would be a matter for Government.
Amanda Spielman: It is a matter for the Government, yes. We regulate the system we have; we do not re‑design it.
Q40 Bill Esterson: We have talked about the fact that we are only making changes to a number of subjects initially. This will have an effect on schools and on students. Would it have been better to do this all in one go, which is the way that things have been done in the past?
Glenys Stacey: The first thing is it is pretty inconvenient for people but it would not be possible to do it in one go.
Bill Esterson: We have made wholesale changes in the past. I seem to remember the move from O-level to GCSE was done all in one go.
Glenys Stacey: We made wholesale changes in the past when there were many more exam boards than there are now—tens of exam boards—and there were fewer subjects as well. What we have got now is about 90 subjects at GCSE and about the same number at AS and A-level—I have not got the exact figures, but it is somewhere around that—and just four exam boards. There is not enough assessment expertise in the country to do it all at once whilst at the same time running the existing system. We still want the existing system to run, with 15.5 million scripts being marked as we speak, 100 million individual questions, and all the rest of it. We are closely watching 5,000 exams being sat. We do not have in exam boards spare capacity waiting for the opportunity to reform everything at once. As the practicalities are, we simply could not do that. There is also a debate to be had—and we have started it—about whether we want as a society all those 90 subjects, because if we do, there is quite a hike up for some of them. It is right to have that debate as well. The last thing I would say is it is also right, in our view, to plan for a reform so it is completed within a reasonable time. I have every sympathy and empathy with schools’ views that it is to happen and they want it to happen within a reasonable time, which is why we are saying it will be completed within three years. That is still quite a task for us and for exam boards.
Q41 Bill Esterson: While all these changes are happening, we are going to have different gradings on certificates; we are going to have numerical gradings on some and letters on others. How are you going to make sure that the public understand this, that employers understand it and that students are not disadvantaged?
Glenys Stacey: First of all, we will not have it until 2017. That is the first awarding, so we have got time to get information out and we will be doing that. Secondly, the research that we have done with employers suggests that they are not going to find it too difficult to understand that a C is a C and a 4 is a 4. It is not seen to be a big issue with them. Thirdly, it seems to us quite important that we do signal that they are different in that way. It would be more confusing if they were using a letter-grading scale. It is a signal, really, that this is the new qualification, if you like.
There is a fair amount we need to do over the next couple of years to keep on pressing and getting to people about how different these are and what assumptions they can make from the different grades. As part of that, we have got a consultation on at the moment about how grades are set for the future and we are running regional events at the moment, as we speak, including breakfast events for local employers. We have started the work that we need to do to make sure people are as aware as they need to be of the changes.
Q42 Chair: Can I ask why 9 to 1 and not 1 to 9? For those of us who have been around long enough to have gone through the GCE, a grade 1 was the highest qualification. Would it not just make things easier for grannies at least if we had 1 to 9?
Glenys Stacey: It is one of those things where there is no agreement. People disagree, so they have different views. We did not have a preference at all, so we consulted on it—it seemed to be sensible—and in the consultation responses, the majority of those who responded saying that they would like a number approach as opposed to letters went for 1 to 9 in the way that we have described. It was just as simple as that, really. We have consulted on it. A decision had to be made, so the balance of views from the consultation was quite influential.
Chair: Thank you. Can I just say—this is obviously down to poor chairmanship—we are about quarter of the way through and have less than an hour left? Can we have really quick questions? It is rather interesting to have a philosophical discussion about it, because it is really interesting, but if we are going to get through to post-16 qualifications, etc, we need to move things along.
Q43 Bill Esterson: Without the background we do not get the full evidence, Chair. The other question I had was about those students who sit GCSE English and maths in summer 2016 and need to re‑take in summer 2017. Are they going to be assessed under the current specifications?
Glenys Stacey: There is one more sitting, is there not?
Dr Meadows: Yes. They will have a re-sit opportunity.
Glenys Stacey: Yes. Is it November or June?
Dr Meadows: I thought it was November.
Glenys Stacey: Yes. We can come back and—
Bill Esterson: Yes, that would be helpful.
Glenys Stacey: There is one re-sit opportunity for those students. I am sorry; I cannot recollect whether it is November or June. I know we have debated it, though. Can we come back to you and say which it is? I think it is June, actually, but can we come back and tell you which one it is?
Chair: Yes, you can. Thank you.
Glenys Stacey: There are pros and cons to either of them, but we have got somewhere with it, so if we could. But there will be a re‑sit opportunity, yes.
Q44 Mr Ward: We can have a philosophical discussion now on the assessment of maths. I understand the tiering issue and the cap on ambition. I cannot pretend to know too much about the issue on maths, but I know that it has been raised as an issue. This is to do with the higher tier and the overlapping of the tiers. For my benefit, if for nobody else’s, what is that problem that has been identified and how has it been responded to?
Glenys Stacey: The problem—there is quite a lot of research on it, which is quite helpful—is that, first of all, if you have got an overlapping-tier model, with a foundation and a higher tier, schools make choices about which students go into which, and they tend to make those choices on the basis of the setting they have already done. They will take a whole set into the foundation or higher tier and a student and his or her parents are not necessarily engaged in that decision. There is a risk with that—and some research from Jo‑Anne Baird demonstrated this—that occasionally a student is disadvantaged by that, because they will be put into the foundation tier, where they can only get to a C grade, when they could have done better on the higher tier. That is the capping, if you like. A student’s aspirations are capped because they are in the foundation tier and it only goes up to C.
Q45 Mr Ward: The general policy was to move away from tiering, but within maths my understanding is it is to be retained. What is the justification?
Glenys Stacey: We set out, again, and consulted on a proposition that, wherever possible, subjects would not be tiered. Indeed, there are quite a number of subjects that are now moving to non‑tiering, English being a very good example. In maths, there is no way in which we can design a GCSE that would be accessible to all students and produce a good, valid outcome without it being tiered. The issues are really twofold, in the main. One is the nature of maths, from your basic four functions through to quadratic equations and all the rest of it—the scale and range of maths, even at GCSE—and then, secondly, the scale and range of the abilities of students taking maths. In some subjects, students will be quite close together in ability. Forgive me, I do not remember which piece of research, Michelle, but I think there is a seven-year age difference in their age capacity for maths by the time they are taking it. You are trying to squeeze into a qualification an enormous range of maths and an enormous range of ability of students taking it. To create it into one qualification, it is inevitable that you will need some sort of structural tiering in it. The real debate has been about the nature of that tiering.
Q46 Mr Ward: Is that resolved?
Glenys Stacey: Yes. Qualifications are in for accreditation now with a tiering model.
Q47 Mr Ward: The note we have here is that there are “concerns about the proposed tiering model offering two routes to the very high-stakes grades 4 and 5; the two routes cannot be guaranteed to be of equal demand”.
Glenys Stacey: This is where the overlap is. We have got a foundation and a higher tier and there inevitably is an overlap—there has to be—and it is at 4 and 5. It is quite interesting; the research that we have looked at shows, oddly, that students in the higher tier can be disadvantaged because markers mark more harshly the higher tier. I do not know why—just human nature—but they do. That is of interest to us. Also, where the grades overlap—at C at the moment or 4 in the future—people want to know, so far as possible, that that represents broadly the same thing and the student has the same ability certified whether they are on foundation or higher tier. To some extent, that is impossible to achieve because they are studying slightly differently, although there is a lot of overlap, but we can do more than we have done in the past to secure it. We are doing that in a couple of ways. One is, for the higher tier, making sure the assessment reaches right back into the foundation tier maths so that there is a good range of assessment there; and then secondly, a rather technical piece of work where the two tiers overlap to ensure proper equating. Forgive me, but in the main it is about making sure you have got anchor questions in both. You might want to deal with the technical side.
Dr Meadows: Yes. We expect boards to have 20% of the marks across the tiers on the same questions. By having that information, you can statistically align the tiers so that where you draw the grade boundaries is informed by this information to ensure that the standards—
Mr Ward: Commonality between—
Dr Meadows: Yes.
Glenys Stacey: This is not perfect, because there is no way you can create a perfect assessment, but it is much better than where we are now. We think it is as good as we can get and the right balance to strike.
Q48 Chair: I understand about the issue between foundation and higher tier, but different countries have different content and they have different methods of assessment. How are you also going to align this with qualifications in other countries, particularly high-performing countries?
Glenys Stacey: If we take maths, interestingly, there is quite a lot of commonality of view as to what maths is—what subject content is maths. There will be different views about the balance to be struck between statistics and geometry, or something, but the basic components are agreed. I say that because it is not like that in history. We know broadly what maths is and, indeed, we have got a curriculum set out with a lot more maths than we have ever had in GCSE maths before and a lot more specificity of it as well. That is really quite useful. Our job is to make sure that that maths curriculum is transposed into the individual qualifications that exam boards offer.
Secondly, we have done some interesting work this year to be more specific about the requirements of the assessment. We require assessment strategies from exam boards to show how they are going to assess this maths content. As I said earlier, we have linked the content to the assessment objectives and considered how it should be assessed for more, less and moderately abled students. We have a lot more specificity. Then, we have a consultation out at the moment that finishes at the end of June with some propositions about linking the general performance standard: what a grade 4 actually means in terms of learning. There, we are linking grade C to grade 4 and we are proposing that grade 5 should equate to achievement in other jurisdictions. We are using analysis from the PISA results, which happen every few years, to make that link. We are making that proposition. The idea there is, yes, that when your child leaves school with maths GCSE or any other GCSE, he ought to be as well-educated and able to compete with his equivalent in France, Poland or whatever. That is not necessarily to say that that is selecting Singapore as the model, but it is those jurisdictions that perform better than us that we ought to be able to aspire to.
Q49 Mr Ward: On non-exam assessment, in geography there are going to be a couple of field trips and there will then be an assessment of whether you have passed or not. Will that be the same for science in terms of practicals?
Glenys Stacey: Geography is different to science in lots of ways. In geography, we have got non-exam assessment at A-level but not at GCSE. If we stick with A-levels, we have got practical science assessed in certain ways at A-level and in geography it is integrated into the one grade. Is that what you are interested in—those different arrangements and why they are different?
Q50 Mr Ward: We are now moving to exam assessment being the rule, apart from geography, which has an element of non-exam assessment through the field trips. What I am saying is you have got a strong practical element to the sciences as well. Is a similar observation of completion by the member of teaching staff going to be used?
Glenys Stacey: There is non-exam assessment in a range of subjects at A‑level, not just geography. I always come back to art or the performance subjects, for example. Our general approach is that if something can be validly assessed by exam, then it ought to be assessed by exam. That general principle has reduced the amount of controlled assessment across a range of subjects but not eradicated it completely. Indeed, in geography, it is not possible to separate out the work that you need to do at A-level neatly in that way. In science, we are in a different position.
Q51 Mr Ward: This is referring to GCSE.
Glenys Stacey: At GCSE we have not yet decided how we will assess science practical skills.
Q52 Craig Whittaker: I just want to ask you about the national reference test. You have said that it is an “unusual” thing to do, so why are we doing it?
Glenys Stacey: It is an unusual thing to do. As far as I know, only a few countries do it—five or six—and they use it for different reasons. I am tremendously pleased about it. It is a great thing and it is a real endorsement of this country’s desire to get results right and to have a fair, open, transparent and objective process for setting standards each year and building confidence in that. The reasons for it are pretty clear from our perspective. It is another set of information and data that will feed into awarding. Here, we are particularly interested in any information that can tell us whether students are doing better, because it is quite difficult for us to see that at the moment. There are different ways you can do it. We have gone for a national reference test. We are absolutely sure that it will make a positive difference in awarding; that is why we are doing it.
Q53 Craig Whittaker: Have you considered other options as well? Are you convinced that the national reference test is the best option to give you the best outcome?
Glenys Stacey: The national reference test is one way of doing it. We have been to see how it is done, for example, in Hong Kong. There are other ways of doing it that we have considered. Michelle, you might want to speak a bit about those.
Dr Meadows: Yes, for sure, there are different ways of doing it, each with pros and cons. One other way of doing it would be to include questions within the GCSE papers that are kept the same year on year. That has some advantages, in the sense that all the students would take them so you would have quite reliable information, as opposed to the reference test, where what we will be doing is taking a relatively small sample of student performance. There are big downsides to doing it, not least that it is crucial that the reference test is kept secure, as you can imagine; we cannot have the items leak into the general public knowledge. You can imagine that if we were to include the items within the papers, that would really be quite difficult to administer in an extremely secure way.
Another difficulty with it is that we estimate that the reference test is likely to be about an hour long for students to take. If a candidate has just sat a GCSE paper, the last thing they want to do is take more questions that are not going to impact on their end grade. There has been discussion about different methods and approaches. There are pros and cons, but the reference test ought to be a well-designed instrument that is the best way—
Q54 Craig Whittaker: So it is all about being open, honest and transparent.
Glenys Stacey: It is also about getting the best assessment expertise and advice on these things. We have a standards advisory group that has helped us. We have also got a sub‑group to advise us specifically about the national reference test. That includes Mike Cresswell, who is now one of our board members, and Professor Jo‑Anne Baird. We are looking to get the right advice. We have specific assessment expertise within the organisation devoted to the design of the test as well. We are about to run a supplier day, where any of our potential suppliers for the test will be able to come to us for the day, go through all of our propositions and consider and test our thinking. It is rather tricky trying to avoid falling foul of the European procurement rules, so it is not that we can go and talk to one exam board about it; we cannot. But we can get as much involvement as possible by having a supplier day and getting people in. We have put quite a lot of effort into thinking it through and getting the design, we think, as right as possible ourselves as well, with advice not just from our own assessment experts but from those we can have ready access to without breaking the procurement rules.
Q55 Craig Whittaker: Can I ask you what progress you have made in deciding how information about individual student performance could be improved? I know Michael Gove asked you to have a look at that. Has there been any progress made on it? Is it going to happen?
Glenys Stacey: That is an interesting question. I am not sure that we have got a good enough answer.
Amanda Spielman: I do not think we have a particularly detailed answer, but the essence of it was that qualifications are designed to report reliably at qualification level; to start reporting more detail would have required significantly more assessment.
Q56 Craig Whittaker: No progress has been made on it, then. Is that what you are saying?
Glenys Stacey: Well, not so much that. We have thought it through. For example, I was talking with Professor Rob Coe the other day about whether an English qualification could say whether or not the student was able to use the apostrophe correctly. If we were trying to do that, two things would happen. One is that you would have a much lengthier assessment, with the cost associated with that. Secondly, you would substantially revise the nature of assessment. There has to be a recognition as to what you can safely ask, again, of a grade and a qualification without hugely distorting it.
A more positive thing, though, in terms of the progress, which I hope you will be pleased to hear, is that on maths, we have been determined in pinning down from Government what the primary policy aim is. If a student comes out with a 4—or a C, as it is now—what should that represent? People want maths to mean all things to all men, but pinning it down, your average student, if you like, should be sufficiently confident and competent in core maths. That has driven the design of the qualification and driven the pinning down of the assessment objectives and the linking of the content to the assessment objectives. We have been able to target what that should mean. What it will not say is that he is equally good at geometry and quadratic equations, because he could be very good at one and not so good at another, but it is a compensatory qualification. If you wanted to simply say, “He is good at quadratic equations”, you would have a qualification that is quadratic equations. You have got a lot in there and it is a compensatory qualification.
Q57 Craig Whittaker: Is it likely to happen or not?
Glenys Stacey: We have not given advice back to Government yet. It is right to say the Secretary of State asked for advice on it a while ago, so you are right to pick us up on it. We do need to go back, but I do not think it is squarable with the way qualifications are at the moment. You would need a hell of a lot more assessment than we have got now.
Q58 Craig Whittaker: Let me just ask you about the evidence base behind your proposals on the assessment of practical work in science A-levels. What expert advice did you take? I know there was a consultation, but there was fairly poor uptake on the consultation, from memory.
Glenys Stacey: We ran a full 12-week consultation exercise and we met with, as far as we know, the entire range of respected interest groups—SCORE and others. We met with some of them several times. We had a good amount of discussion. I met with the Chief Scientific Officer as well. Indeed, those discussions did influence the board’s final decisions. In summary, there was a general agreement that things are not as they should be. There were not many propositions about how it could be much better and at the same time we could secure valid outcomes, but some of the propositions that were made we were able to incorporate—for example, ensuring that a good range of practical skills are covered in real practical exercises on the bench, and also making sure that more of the understanding of experimentation is assessed in examination. We have been able to embed that as well.
Q59 Craig Whittaker: What has the reaction been to your assessment proposals in other subject areas, specifically for A-level English, for example? What options have you explored that would support more valid and reliable non-exam assessment in this subject?
Glenys Stacey: You asked first of all what the reaction had been.
Craig Whittaker: Yes.
Glenys Stacey: Correct me if I am wrong, but the most adverse reaction we had by far was in relation to sciences. I am not aware that we have had any particular difficulty in terms of our proposals for the other subjects. I am not sure that there has been any particular concern from those other subjects.
Secondly, we have applied the same principles, really. If it can be assessed by exam, assess by exam; if not, it can be done in non‑exam assessment. There is some of that in English, because the creative writing side of it is not possible to do in any sensible assessment in an exam, so we have still got it in there. I am not sure whether you were expecting something different in English.
Q60 Craig Whittaker: No. It was just specifically in English. I do know that, for example, you have reduced the weighting for non-exam assessment in that area. I just wondered whether there was—
Glenys Stacey: An outcry against that?
Craig Whittaker: Yes.
Glenys Stacey: Not that I am aware of, no.
Q61 Chair: Can I ask you about the uncoupling of AS-levels from A‑levels? You said that you had done some impact assessment. What does your impact assessment suggest?
Glenys Stacey: I do not think we have said we have done any.
Dr Meadows: No.
Q62 Chair: In that case, it is about what you think, then. Is the default position going to be very quickly that students will take A2s in their chosen subjects anyway? What do you think is going to happen with disadvantaged students?
Glenys Stacey: What will schools do in relation to AS and A2?
Chair: I guess it is a little bit of guesswork.
Glenys Stacey: Yes, it is a bit of guesswork. I was at an event recently of sixth-form colleges and this came up. We had a, “Put your hands up if you are still going to teach AS”, and the majority of hands went up. It is not scientific and it is not a piece of research, but it is interesting. In individual schools, I get the impression that they are very keen to hear from higher education as to how they are going to value and rate AS-levels. That is really quite significant.
Amanda Spielman: Yes. The Pearson submission you have had for today draws attention to that point: that it is perhaps universities that could most helpfully clarify how they will regard and treat AS in their admissions going forward.
Glenys Stacey: Can I just say it might be helpful to recognise that AS can be designed to be co-taught with A2 now? I do not imagine an exam board is going to be designing an AS that cannot be co-taught with A2. If a school is looking at an A2 with an AS it can co-teach, a school will no doubt wish to do some sort of assessment at the end of the first year of study to see how the student is getting on. If it is not that AS assessment, it is going to have to design another assessment or use previous years’ papers. There are some advantages, or it is not so disadvantageous to take AS and A2.
Q63 Chair: The advantage I always found was that AS subjects gave young people the opportunity to try things that they did not feel confident in, particularly in my area of maths. I saw so many young people who did okay at GCSE and who tried maths at AS and discovered A-level maths is incredibly different from GCSE and then went on to do pure maths and whatever. It brought in students who would not normally have studied those subjects and it was really good for disadvantaged students, to get them a taster of something.
Glenys Stacey: There is not anything that I can see that would stop people still taking that approach. I suppose inevitably, schools are chasing the best possible A2 result and no doubt giving students advice on where their most likely prospects are to get the best possible A2 result. That is an increasing driver, I would say, in schools that we see. It is whether, studying that subject, you are more or less likely to get your A* or A. You can see that advice being promulgated at GCSE level in terms of what you are aiming for, rather than whether you should take this or that at AS. A second driver will be around funding and what teaching resource you have available in a school. You will set out to offer A2 and possibly AS if you have got the teachers to support it.
Chair: That is the biggest criticism in schools: that they do not have the resources.
Glenys Stacey: We see those two drivers to be quite prevalent.
Q64 Bill Esterson: This was the point that was made in evidence to us: would both AS and A-level be funded for the same student? Alongside that, the NUS and others made the point that many of their students said they would not put in for A-level if they could not start with AS and then move on. That was the very big concern, particularly in STEM subjects, where it is perceived to be really tough. Given that is such an important area, that is a very real worry.
Glenys Stacey: We have made it very plain that the decoupling of AS from A2 is a Government policy decision.
Bill Esterson: You can still comment on Government policy decisions if you want to.
Glenys Stacey: We have made the comments that we wish to make.
Q65 Chair: Can I ask you one quick question on the content on GCSEs and then we will move on to Neil? Can you clarify this for me? At the moment, the DfE are making the decisions on content. That will continue until 2016. What happens after 2016?
Glenys Stacey: That is a very good question, if I might say. We are currently consulting—we seem to spend a lot of time consulting—on propositions that we are making about the revision of subject content and the arrangements for those remaining subjects. The indication we have had from Government is that neither Government nor ALCAB are interested in taking responsibility, taking ownership and leading on that, so we have thought about alternative ways of doing it. It seems to us really important that the remaining subjects are revised and updated, and that the weight of the content compares to the ones that have already been changed. We are proposing in our consultation that exam boards take the lead on that but that we set requirements for how they do that. Those requirements would include requiring exam boards to consult with and involve stakeholder groups so that they can make sure, so far as possible, that that content is fit for purpose.
Q66 Chair: Would that include exam boards being responsible for revisions?
Glenys Stacey: The idea is that we would then have one set of content requirements that is agreed in the subject for whatever exam board and that would not change until you got another process running for that. It is pretty much the same as where we are now with content: you agree it and then the curriculum requirement stays. It can be interpreted by exam boards, but the core content stays until another round of revision.
Chair: Thank you. Regulation of exam boards, Neil.
Q67 Neil Carmichael: Thank you. Good morning. The key theme through my questions is the danger of over-regulation.
Glenys Stacey: Right. We are happy to step back.
Neil Carmichael: You have implemented a lot of changes and I suppose the natural instinct of those being regulated is to wonder just how far this is going to go. Do you feel you have got the balance right?
Glenys Stacey: We are striking a different balance. Our focus is very much on regulation that ensures quality. That is not about the amount of regulation; it is about the nature of regulation. It is about regulating intelligently, is it not?
Neil Carmichael: Absolutely.
Glenys Stacey: Again, we have learnt a fair amount from GCSE English and other experiences. Ofqual is now four years old as a regulator. That is quite young. It normally takes about 10 years for a regulator to mature. We have reflected on how we have regulated in the past and found ways in which we can improve it. We are in a transition from the way of regulating that was quite an extensive rule set to a way of regulation that is, we believe, more intelligent and more likely to ensure good-quality products.
During a transition, there is always a danger of a bit of overlaying; you are bringing in new requirements that put the onus more on exam boards to demonstrate quality but you have still got some elements of the old way of regulating as well. We have got a transition happening. We are going very much in the right direction. We are just starting a review of the code of practice that will eventually result, I think, in getting rid of the code of practice, but it serves a purpose at the moment as we develop this alternative way of regulating.
Q68 Neil Carmichael: To what extent are you using regulation to effectively introduce improvements?
Glenys Stacey: Very much so. I will not use the expression “Trojan Horse”.
Neil Carmichael: No, I would not.
Glenys Stacey: We are certainly using these reforms as a stalking horse really for changing the way we regulate in order to ensure quality.
Amanda Spielman: Improvements in what?
Q69 Neil Carmichael: That is a very good question. I think basically to allay the fears that we were discussing a year or two ago and have discussed since. That is really what we are getting at here. For example, in connection with marking: are we satisfied that we have got marking across the piece in a fit state? That would be my next question.
Glenys Stacey: Just a tail-end point on your last question: I would say that in the past a lot of effort has gone into changing qualifications—for example, GCSE English and 60% controlled assessment for curriculum reasons and all the rest of it—but not enough effort went into inter-board comparability and making sure those qualifications were of sufficient quality whatever the exam board and whatever the year. Recognising that and making sure that happens has been very important to us, and we are in a much better place on it. That is what this way of regulating is all about, really. Coming on to marking, what was your question?
Q70 Neil Carmichael: Are you satisfied with the overall standard of marking? Do you think there are any worries about variance of marking?
Glenys Stacey: There are several things to say. First of all, we produced an extensive report on the quality of marking in March this year. If you have not had the opportunity to see it, I very much recommend that you do. It is on our website. It is the definitive assessment of quality of marking as it stands at the moment. Broadly speaking, what we found there is that over the last 10 years there has been a significant transition from paper‑based to on-screen marking and also a transition—to a slightly lesser extent but still moving in that direction—to item-level marking, so you are given your 100 answers to question 8 and you mark those.
Item-level marking and on-screen marking have immense potential to enable exam boards to monitor and deal with the quality of markers. Online, immediate evaluations take place and a lot more controls happen. Our summary is that the quality of marking is good and the public can have confidence in it, but it can get yet better. We made propositions in order to make it yet better. Just to run through those, as we are taking them all forward, first of all, we will agree with exam boards common metrics for the quality of marking—not markers; marking. There are different metrics; there are pros and cons to all of them. Just one would drive adverse behaviours, but having a basket of metrics that they all report on seems to us quite important.
Secondly, we will require exam boards to ‘fess up when they get something systematically wrong. If exam boards are not absolutely open as quickly as possible about what went wrong, why, what the effect was, what grade changes there were—if any—and what they are doing to stop it happening again, then it undermines confidence more than ever. Getting absolute transparency and openness about that seems quite important.
Thirdly, we will improve fundamentally the appeal system, which we have said publicly is labyrinthine and does not build confidence. It seems to us that we simply want a fair and effective appeal system, and we have got that work under way. Those are three significant moves.
Lastly—and please do not discount this as just a techy thing, because it is not—we are focusing on mark schemes. All of the research shows that the more we can get good mark schemes, the more we will get good marking. Certainly we are looking at mark schemes very closely in the current accreditation process and we have got some research about to start about how we improve them. What we do not want to do is be so prescriptive that we get precise but not good mark schemes, but getting that balance right is so important.
Q71 Neil Carmichael: Have you been focusing on OCR in connection with this?
Glenys Stacey: Yes. It is a separate piece of work, but if you are asking whether we have shown a stern regulatory interest in OCR’s record of marking for the past 12 months, the answer is yes.
Q72 Neil Carmichael: Moving on to the question of examinations and teaching to examinations, AQA have talked about unintended consequences, because now teachers really cannot, according to them, necessarily learn about the exams sufficiently in order to teach them. The pendulum, according to AQA, in that respect may have gone too far. What are your thoughts?
Glenys Stacey: I have two immediate thoughts. One is that AQA have not said that to me, so that is interesting. If they have that concern that somehow things have gone too far, I would like them to say it to me so that we could consider upon what basis they think that and why. I am surprised that if that is the case it has not been mentioned before now in the many meetings that we have had. Secondly, I do wonder sometimes whether we will get to the point where there is nothing to do for schools but to actually teach. I do not quite understand the point that AQA are making, but I would be very happy to speak with them.
Q73 Neil Carmichael: What I have gleaned from this is that AQA point to unforeseen consequences of action taken by Ofqual in regard to training.
Glenys Stacey: Oh, training.
Neil Carmichael: Yes.
Glenys Stacey: Right, okay.
Neil Carmichael: Chiefly because they cannot necessarily be trained.
Glenys Stacey: This is really coming back to the problem we had with seminars a little while ago. Do you remember?
Chair: Yes, we do.
Glenys Stacey: We took some firm action then to control seminars and to put in various controls—for example, requiring them to be recorded and keeping those that had an involvement in the setting of an exam away from seminars. That is where we had seen some really quite appalling breaches—although not very often. We think that firm approach was right at the time. With these new qualifications coming, we have moved those controls slightly. We have taken a review and we have looked at it and we have softened our controls to some extent, but we are still asking examiners to keep away. We do review controls periodically based on the risk as opposed to the benefit of allowing things. If the time is right for us to take another look at whether we have got that balance right, we would be very happy to do so. Again, I am not sure that AQA have raised it with me, but I would be very happy to have a look at it.
Q74 Neil Carmichael: Regulation is one of those vexed issues; one man’s regulation is another man’s problem, and vice versa. Do you sense that the direction of travel you are in is being supported by those you are regulating, or do you think you are having to push and push and push?
Glenys Stacey: I am not sure that everyone that is regulated loves regulation or loves being regulated. What those that are regulated generally want is a level playing field. They want to know that we are being even-handed and that we are regulating in the right territory in broadly the right way. I honestly think that in the work we have done with exam boards in the last nine to 12 months to get these qualifications in for accreditation, exam boards have played their full part, they have behaved responsibly and they have respected and appreciated what we have done as well. In the reform work, there is a general acceptance that we have got the balance right. There is a different question about whether every aspect of how we regulate the current day is right, and it is always right for those being regulated to question whether we do or not, and there are some things where we listen and think, “Yes, we could make that easier” or whatever. That is always the way, but generally speaking I do not get a sense from those we regulate that we are over-regulating.
I was at an event about a month ago of all awarding bodies that we held. It was not a compulsory event, but there are about 160 of them and there was a very good attendance. It was 9 o’clock on a Monday morning. I said to one of those attending, “I am quite surprised at the level of attendance. Why do you think so many have come?” She said, “Well, you are regulating now, and we have noticed it”. What is wrong with that? That is good, is it not? We are regulating and they have noticed it and they have come to hear what is expected of them in terms of the quality of all qualifications. I was quietly very pleased about that.
Neil Carmichael: So you should be. It is a really good point to end on. Thank you very much indeed.
Q75 Chair: Can I ask you just a couple of questions about the role and administration of Ofqual? We have talked quite a bit about transparency this morning. Why do you not publish a full list of external subject experts on your website?
Glenys Stacey: Thank you for asking that. It is something that my Chairman has been questioning as well. The position is that we have just gone through a review of our experts and a big recruitment drive. We have now got 650 experts. In that recruitment drive, we set out new terms and conditions that require experts to agree to their name being published as and when they do work for us, so that was a step forward. I have reconsidered that, because it has been suggested to me that without the publication of the full list there is the sense of a lack of transparency and a concern, and I understand and accept that. I, with the team at Ofqual, re-thought that and we are now intending to publish the full list. We are not waiting for people to do the work and then publish. To do that, we have to approach again these 650 experts and ask for their permission, or at least notify them that we intend to change their terms and give them the opportunity to choose not to work for us. It is right that we go through that process with them, out of respect for them and to give them the choice. That is our intention and that is what is now underway.
Q76 Chair: Have you got any idea when we are likely to see that?
Glenys Stacey: Those letters are going out now. We would wish to publish that list pretty quickly, but we need 650 people to have the opportunity to reply.
Q77 Chair: “Soon” is a word loosely bandied about.
Glenys Stacey: Within the next two months.
Q78 Chair: That is great. Thank you. You have got a lot to do and you have done a lot. Do you have sufficient resources to deliver all the things that you need to? If you do not, what is going to have to give?
Glenys Stacey: There are two ways to look at resources. One is about the expertise—the nature of the resource; not the money, but the people. There is always a shortage of expertise in the area we work in. We have been very fortunate in recruiting some good people. We have Michelle here. Paul Newton is coming to join us later in the year. We have had a number of good recruitments. Expertise is building, but we always will want more. There is also money—the totality of the money. What we have had is our baseline and then additional money given to us year upon year for reform or, for example, for the national reference test. We have got yet more work to do now on vocational qualifications, and BIS are being very responsive to our approaches there.
We have certainly had enough resource to date, and do try to manage our money extremely carefully. We have made some changes, for example, to our IT provision this year to save a good amount of money. We are earnest in our application of it. The discussions that we are now having with DfE and others are about a consolidation of our funding, because we are getting bits for reform and bits for the national reference test and there is a point now where we have to consolidate our resources going forward. The CSR exercise is our opportunity to do that so we have got some more certainty around the provision.
Chair: That leads us well onto our next question, which is about your relationship with the DfE.
Q79 Craig Whittaker: I think you have asked the question. How would you characterise your relationship with the DfE and officials?
Glenys Stacey: I will ask Amanda to answer that one.
Craig Whittaker: Are you pleading the fifth amendment?
Amanda Spielman: No, we are not pleading the fifth amendment at all. Ofqual has an organisational responsibility. It is not a personal office; Ofqual is an organisation. The relationship is pretty well considered and balanced. On the one hand, we have the constructive working relationship in relation to reform. We seem to be progressing that and running well-structured processes while at the same time maintaining the real clarity that is important about our independence in terms of our statutory responsibilities for maintaining standards. There is good respect on both sides for that.
Q80 Craig Whittaker: Are you afraid of, or are you quite comfortable with saying “no” when something that is unrealistic is being asked?
Amanda Spielman: We have said “no” when we have needed to in the past. We never do so without full and careful consideration of the options. We would approach things in exactly the same way in the future.
Q81 Craig Whittaker: Can I just ask you about your relationship with fellow regulators and the result? The relationship seems to be better than it was. Is that as a result of acceptance on all parts that the three-country regulation of GCSE and A‑levels is no longer tenable?
Glenys Stacey: With devolution and different policies that seem to be increasingly diverging, it is the regulator’s job to recognise that and accept that, which we do. Secondly, even with policies diverging and with qualifications being in many ways different, there is still a lot of commonality; you are still regulating the same set of exam boards, for example, and you do not want to duplicate or replicate the administrative or regulatory burden. Even with qualifications being quite different, there are a lot of things that we need to do together in order to make regulation work. We were very keen to recognise that and to have good working relationships around those things. We have moved from an era of gentlemanly agreement with nothing really set down as to how you do things or when you would not do things into a more considered era, I would say. We have a working protocol now agreed with the Welsh regulator so we each know what to expect, and we particularly know where we need to collaborate and where we do not.
Q82 Craig Whittaker: There is still a way to go, then.
Glenys Stacey: No. I think we are where we need to be. Qualifications will increasingly diverge. Wales is intent on having its own regulator as well—Qualifications Wales—and we wish them all the best with that. Things are going to continue to change, but we need to make sure that, where we need to regulate jointly, we do and we do it in a way that does not have any surprises.
Q83 Craig Whittaker: You mentioned Wales. They still use the original AS qualification. Should we therefore re-name ours, because ours is not the original?
Glenys Stacey: Re‑name the AS?
Craig Whittaker: Yes, because it is different to the original. Wales has the original format; ours is different. Should we re‑name it?
Glenys Stacey: The titling of qualifications is enormously controversial. I am sure if we proposed any changes we would get a lot of push-back from all sorts of quarters, including schools and higher education, so that is not something that we would wish to do lightly. What I do think, though, is that it is incumbent on us to be very clear about where things are the same and where things are different. We certainly could do more there, not just for AS and A-level but for GCSEs as well. As these qualifications and policies diverge, we need to be transparent about how they differ.
Q84 Craig Whittaker: Are we not in danger, then, of devaluing the AS qualification, for example? That is a scenario we had a couple of years ago, when you had the Welsh, for example, saying they were going to re-mark the exams and we were not, so you got a big disparity between the two. If people perceive it as being the same qualification but it is very different, why call it the same thing?
Glenys Stacey: As I say, there are a number of approaches to that. One is re‑titling. We do not automatically discount that; I am simply saying that that would be a very big step and there would be some strong views either way about it. Most immediately, it is about how different they are and whether we can explain those differences. That is the first step. We have not got these new qualifications yet. How different are they in practice? We will want to do a considered piece of work about that. We do not write off re‑titling but we are not jumping to it.
Chair: Having a full discussion at the beginning managed to get us right through the brief and by 11.31, so maybe I am not such a bad Chairman. Thank you for coming. As you will have picked up, the Committee does clearly still have some concerns around things like curriculum content and the balance of accountabilities. Not all of those things fall within your remit and we understand the tensions that are there. No doubt we will meet again on a regular basis, but thank you so much for coming this morning.
Oral evidence: Exams for 15-19 year-olds in England: Follow-up, HC 143 2