Science and Technology Committee
Oral evidence: School practical science, HC 329
Wednesday 3 September 2014
Ordered by the House of Commons to be published on 3 September 2014.
Members present: Andrew Miller (Chair); Jim Dowd; Stephen Metcalfe; Stephen Mosley; Sarah Newton; Graham Stringer; David Tredinnick
Questions 1-41
Witnesses: Janet Holloway, Head of Reform, Ofqual, Dr Steven Evans, OCR, Darren Northcott, National Association of Schoolmasters and Union of Women Teachers, Nicole Morgan, Royal Society of Chemistry, and Steve Jones, Consortium of Local Education Authorities for the Provision of Science, gave evidence.
Q1 Chair: Thank you, everyone who participated in the round table. Thank you for coming earlier. I appreciate that one or two of you had difficult train journeys. I hope the west coast main line is now sorted because it is of interest to several of us.
We are now going to go on to some formal questions to each of the panels. We are starting with panel 2 who looked at the SCORE approach to assessing science practicals. May I ask this panel—I appreciate there may be different answers from different organisations—what you see as the strengths and weaknesses of the SCORE approach to assessing science practicals?
Steve Jones: Everyone is looking at me, not that I agreed I would do this.
Nicole Morgan: I think it would be slightly strange if we represent our own—
Steve Jones: Yes, it would be odd for you to do your own. Certainly the observation, or the concerns, about the decoupling were not easily addressed in the discussion. A number of the organisations present sign up to the view that decoupling the practical assessment would be unhelpful at this point, and we could not resolve that across the piece and come to a satisfactory solution. A lot of the discussions led back to the time line, so one of the strengths is the suggestion that some of the things need more time to resolve than is available in the time line as proposed. Clearly, it is not within the remit of Ofqual to determine the time line, so we cannot say that there is not time to do this. Some of the suggestions, in terms of developing questioning and so on, were greeted positively but there were concerns about whether it could be done in the time available. That is a start. Does anyone want to add to that?
Janet Holloway: There are three parts to the proposal from SCORE, though I look to Nicole to smile at me when I say that she took the opportunity to express that it was not necessarily a proposal; it was a reflection of views of members and a collection of those views. Two of those parts are actually part and parcel of the proposals for the new qualifications: the use of a record of laboratory activities that is an active record kept by students of their own work; the second is a test of students’ technical and manipulative skills. The part that was proposed that has not been covered is that of an extended investigation, though we did discuss the opportunities available through another qualification called the extended project, which is gaining hugely in use and availability to students and is a qualification highly regarded by higher education.
Q2 Chair: Anybody else?
Nicole Morgan: I want to reinforce the headline of the proposal that everyone agreed on, which is that practical work is an essential part of the A‑level experience and therefore must be reflected in the assessment. Obviously, there are differences of opinion as to how that assessment is formed, and there was a lot of discussion and agreement that the circumstances within schools at the moment—the high pressure, high stakes assessment environment, league tables, the pressure on resourcing for practical work and the availability of confident specialist teachers—were all contributing to a situation where we think it will be difficult to introduce the changes in the time scales because so many things need to be addressed. While we all agreed with some of the sentiments behind the separation of the practical work, we felt there were too many risks associated with it for the reality of where schools are at the moment. There were lots of discussions about the resourcing and preparedness of schools, the manageability; those were the kinds of things we talked about, obviously all coming from a slightly different perspective, but we all agreed on many of the same issues.
Steve Jones: There was a strong measure of agreement that what is current practice is not very successful—everybody would sign up to that—so there is a need for change.
Q3 Stephen Metcalfe: I am pleased to hear that there is agreement that things should not stay the same, despite the risks. What we are trying to tease out is this. There has been a proposal made by Ofqual, there has been quite a lot of criticism of that proposal and we are trying to explore whether there are alternatives. Yes, it is great that people have thrown some ideas on to the table, but what we need are some practicable or workable alternatives, to put back to Ofqual to say, “This is an alternative,” and we are not really hearing that yet. Having looked at the SCORE proposal, do you think that it is better than what is currently proposed or is it just that elements of it might help?
Darren Northcott: I think elements of it might help. You are looking for a package of overall proposals and, as our colleagues have said, that is not what SCORE provided. To be fair, they provided a response to Ofqual’s proposals, but I think in developing a potential alternative approach one of the things SCORE highlighted, on ours as well, was the issue around time scale, for example. Whatever you think of these proposals, they are an incredibly complex series of changes being introduced in a very short period of time. One of the objectives of the proposed change to A‑level is to encourage more practical work in schools and colleges. Everyone would share that objective, but I think there are some practical barriers. It is not just a question of changing a qualification so that you get more practical work. We have to understand that some schools and colleges are better placed to achieve that than others. In developing an alternative, there are some things we need to consider, one of which is, “Would an alternative package of proposals introduce change over a longer time scale?” That seems reasonable. I know that falls short of your very reasonable question, which is, “Give us an alternative package of proposals,”—I haven’t got that—but what I think perhaps a slightly more effective package of proposals would contain is a longer time scale and perhaps more of an audit of the state of preparedness of the system to deliver more practical science learning.
Dr Evans: Picking up on what Darren said, one of the things we discussed quite a lot was the unanimous acknowledgment that the current assessment is not delivering what people want it to do; it is not actually delivering a breadth of practical experience, so we had a bit of discussion around an investigation. Lots of people say that is great, and I would not disagree with them, but it does not necessarily underpin that breadth. I think the acceptance across the group was that, based on what may be the norm at the moment—in terms of how many practical activities people are undertaking—the endorsement involves a step up, raising the bar on that requirement, quite significantly perhaps in some cases. But I think from what Darren was saying, it is about how that can be better supported. If that is where we need to go, I think everyone acknowledged that we need to make sure there is breadth of experience on a range of practicals—not just a few, but “How is that better?” It comes through in some of the SCORE comments around making sure schools can be prepared for that. I do not think that is covered at the moment, and something we were all very concerned about was how schools can resource doing more practical work.
Nicole Morgan: I think it also comes down to the fact that there is a recommendation in the way the awarding organisations have approached it. They are developing different questions that are about to be trialled. Obviously, trialling the questions is very welcome, but there needs to be time to learn from those trials. Is everything going to work? Will schools have enough time to be able to prepare themselves to make sure that students are ready for a new way of being tested? We hope, and I am sure there is a degree of confidence, that the new questioning will work, but there is still a certain risk associated with that. One of our concerns in the time scales is that there is not much time to learn, to gather the evidence that is needed to support the new way of testing.
Q4 Stephen Metcalfe: Does anyone else want to add anything? No, okay. I hear what you are saying, but, as I said, we have to try to tease out some alternatives. Someone had to make a decision, someone has and now we need to present alternatives.
Looking practically at some of the elements that make up not necessarily SCORE’s proposal but some of the ideas, one of the criticisms—if that is the right word—is that it is particularly open to malpractice. Is that something that you would agree with and recognise? No?
Steve Jones: I do not think it is necessarily open to malpractice any more than any of the other things that have been proposed. There is almost a sort of contradiction within the argument for the proposal from Ofqual that the current high stakes environment is driving malpractice. There is a very valid point in the SCORE observation that the solution to that appears to be to lower the stakes on the assessment so that there isn’t a driver, but that would inevitably mean less emphasis on practical work. When you raise that point they say, “Oh, yes, but it’s still important to have practicals.” Therefore, it is still high stakes and there is still a driver within the system for malpractice. I do not think this is opening the door to malpractice any more than any of the other solutions that have been put forward. Malpractice needs to be tackled, but this is a bit of a blunt instrument to deal with it: “Get rid of the practical altogether. Therefore you can’t have any malpractice.” It is not terribly useful.
Q5 Stephen Metcalfe: Okay, and the other—
Janet Holloway: May I add something on malpractice? For us to have a complete understanding of the proposal and the opportunities for malpractice, it needs to be worked up further; it needs further consideration. However, I think it takes us back to something that we discussed at the beginning of the meeting—once I had joined it after the rail difficulties—which was, “What do you want from the practical work undertaken by students?” If it is about supporting and furthering teaching and learning—giving life to the learning—then the pressures of qualifications, grades and so on, applying with it can drive perverse intentions and can lose the purpose of having the practical and the emphasis on practical work in the first place. I think there was agreement round the table that indeed that is what has happened, which is why we have the situation that we have at the moment. It is the tension between what you want to get from this and its incorporation into a qualification and the consequences and the activities it drives to maximise the results in those qualifications. It would be the same with any proposal. You would want to think about those. You would want to work up the detail to understand what kind of unintended perverse activities might arise from it.
Q6 Stephen Metcalfe: I have a final question; I realise we are tight for time. With regard to an extended investigation or an extended project, are there practical problems with doing something like that, perhaps particularly with large class sizes?
Dr Evans: Investigation is a very good thought exercise to do. In terms of people who do it at the moment, teachers and centres choose to do an investigation; they go into it with their eyes open and put a lot of resource against it. For it to work well, it is an individual investigation, something the student wants to explore and do. When you open that up to saying that the whole national cohort has to do an investigation, you go down to quite a formulaic approach. Within a centre, there may only be one, and it is not student choice any more; it perhaps loses a lot of the actual value that an investigation has, which is one of the key concerns we were discussing.
Steve Jones: On that point, there is a bit of a risk, because if you design the system within the constraints that are already present—for example, availability of equipment, laboratories, technicians, and so on— and you take those as read, they limit what you can then do, and you end up with a potentially very unaspirational solution. If the intention is to drive better practice, you have to be very careful about taking as read that nobody has any glassware and labs, because before you even start you limit what you are trying to achieve. There is a balance between having something which cannot be delivered because the resourcing is not there but on the other hand being so constrained by the consequences of the last 20 years of decisions around resourcing that you do not end up with an aspiration or something that you really want. That got quite an airing as well. It generated the idea that this is an interesting discussion about one small aspect of the issues around practical work in science. You very quickly run to the edge of this bit and you end up thinking about labs, technicians, equipment and all the other things that enable or disable good quality practical work.
Nicole Morgan: It is also about the coherence of the science experience, whether it is individual science subjects or science with maths, and seeing how they are all developing alongside each other. There were some concerns that SCORE raised earlier on about maths being developed independent of sciences, and I think practical work is another element and why there is an issue. You have to look at the whole.
Q7 Sarah Newton: I have a very quick question on the practical side of this. What consideration was given to enabling children or young people with physical disabilities to participate? It seemed that in this particular methodology there was going to be examination and actual physical manipulation, which could disadvantage some otherwise very able students? What consideration was given to that?
Darren Northcott: That is an incredibly important point. Obviously Ofqual is under a statutory duty to undertake a quality impact assessment of its proposals. People are thinking through what the consequences are, but one thing is very clear: whatever assessment arrangements we establish they cannot be allowed to disadvantage unreasonably candidates with a physical disability. Clearly the integrity of the qualification has to be retained—that is important—but I think you raise really important questions that have not been emphasised across the debate. Perhaps they need more thought, because that would be a terrible consequence of any reforms that were introduced.
Nicole Morgan: But I think Ofqual and the awarding organisations already have procedures in place to support those students, because those students are already doing A-levels with a practical within them, so in that respect there is no change. Obviously you always need to be mindful of what you are introducing being accessible to all, but I think there are procedures.
Q8 Chair: A final word.
Janet Holloway: I presume you were asking that question about the SCORE proposal.
Sarah Newton: Absolutely.
Janet Holloway: The Ofqual proposal goes into some detail with an equality impact.
Chair: Unfortunately, time is against us, so thank you very much to the first panel for your presentation. You were actually panel 2 when you were in the room, but we will now go to what was panel 1 in the other room, the York University and Salters Horners proposal.
Examination of Witnesses
Witnesses: Michelle Meadows, Director of Research and Evaluation, Ofqual, Stella Paes, Head of Science, AQA, Max Hyde, National Union of Teachers, and Dr Elizabeth Swinbank, University of York, gave evidence.
Q9 Chair: If I may, I will start with the same question: what are the strengths and weaknesses of the York approach, and what do you think it brings to the table that Ofqual’s does not?
Dr Swinbank: What we propose is very similar in quite a lot of respects to the Ofqual proposal, but I think it goes further than that. We have said that not only do we want the breadth of practical skills to be specified, so that over the course of their A‑level students experience a lot of different practical activities and techniques, but in our proposal we have built in the fact that we want to develop an investigative inquiry approach to practicals, so it is not just about learning to use specific techniques and instruments; it is to develop the whole approach, from asking questions, designing investigations, analysing and evaluating. A strength of our approach, or our proposal, is that it is realistic and manageable. It is based quite strongly on what already happens in the Salters-Nuffield biology project, which is directed by my colleague Anne Scott. In that course we have core practicals; we have an investigative approach built in. Students and teachers self‑report; they record instances in which students have demonstrated particular skills and approaches, so it is manageable in terms of the record keeping that students and teachers are asked to undertake.
I think a difference between our proposal and the Ofqual proposal is that it is not simply a pass/fail model. We want to reward students who have gone further than merely doing what they are told, but actually we have—
Q10 Chair: I want to come back later to your distinction proposition. Does Ofqual have a comment?
Michelle Meadows: What was interesting—I hope you agree—was that there was more that was similar than different in the two proposals, and actually the assessment criteria that are being developed for practical assessment emphasise investigative approaches. As I say, I think there is more that is common than different about the proposals.
Q11 Chair: Ofqual are seeking to improve skills while addressing issues around malpractice, or alleged malpractice. How does this solution address those problems, or perceived problems?
Dr Swinbank: One of the things that is very important is that there is a verification built into this, in that centres are visited by representatives of awarding bodies. That is something that awarding bodies are already discussing in relation to the Ofqual proposal. Centres are visited, teachers are asked to provide evidence that practical work is built into their scheme of work, that they have practical facilities available, that they are up to date and that their students are keeping records appropriately. That is very important.
Max Hyde: Something that is attractive to me—because as well as being from the National Union of Teachers I am a teacher of physics and chemistry and have been since 1977, and I am also a science ambassador—is the professional approach; it is introducing a very wide range of skills that are hard to measure unless you are professional about it. In other words, you do not have an exact check‑list for some of the very important things, such as how somebody has approached and solved a problem, but it is recognisable by teachers. It is something that is important to industry and to academia as well: “How do you approach these things?” I really like the idea of a teacher saying at the end of the two‑year course, “Yes, this young person has demonstrated that.” As you are well aware, a lot of really important skills are very hard to measure in a tick‑box way. It is about using professional judgment—I think teachers will welcome that—and also the wide range of skills that you are looking at. The expectation will be that young people have this wider range of skills. Decoupling it from high stakes testing is a controversial issue, but I think there is a lot of merit in actually judging practical performance separately. The trick will be to make sure it is treated with equal seriousness, because it is not coming out with a grade.
Michelle Meadows: The Ofqual proposal for separating the practical and science is not just about teacher malpractice. There has obviously been a lot of discussion about that, but the fact is that at the moment the way practical skills are being assessed means that the assessments do not discriminate well. The reason for that is that most candidates can master the kind of direct manipulative skills and so on that are being assessed, and that is quite appropriately so. That means that, when you then combine it, as it is at the moment, with more written extended theoretical answers from candidates, the spread of marks on those assessments is such that the grade boundaries end up being very close together because all the candidates achieve a certain proportion of marks. When you have close grade boundaries, you can misclassify students. It is unreliable, bad assessment practice. There are lots of different reasons why we think that our proposal is better than current practice. It is not just about teacher malpractice.
Q12 Chair: Okay, but moving on to the point about the distinction in this proposition, as I understand it, there would be a certificate that says that I have a grade A in my chemistry and a distinction in my practical. I presume I could have a grade C in my chemistry and not a distinction pass, and I presume you would not get a certificate if you had a creeping over the boundary pass on your written work and a failure—that would be a rare person, I suspect. That is not unique—I was in your group when you were discussing it; there were in days gone by certificates, say, in French that distinguished between passing the exam and passing the oral, so this is not an original idea. How would you see that working and do you see it formally being on the certificate in the way that I have described, in your proposition?
Dr Swinbank: I think, as with the current Ofqual proposal, that, yes, it would be on the certificate alongside the grade that was awarded on the basis of the written papers. Our pass/distinction proposal is, as has already been said, based on the fact that most students can be taught to do most of the things that would enable them to get a pass grade. I would be very surprised if anybody, or very many people, who went through an A‑level course did not pass on the practical skills, because most students, given reasonable teaching opportunities, will master those skills, and I have no problem at all with that. It is absolutely right that students should do that as part of their A‑level. The distinction grade would be to reward those students who are not just going through these things, more or less mastering skills and doing as they are taught, but who have gone a bit further: they are actually bringing their own choices and decision making; they have that extra level of experience and practical know‑how. If those students can be rewarded with a distinction grade, that will say something about them, and I hope that employers and university admissions officers would recognise that. For example, they might say, “In order to do a degree in biology at this particular university, yes, we want you to have a distinction in your practical skills as well.”
Q13 Chair: So they have designed, conducted and evaluated an experiment rather than simply following a recipe.
Dr Swinbank: I would not just say “an experiment.” It is very important that over the course of the two years—it would not be just on one occasion—we want the teacher to be able to say, “I have evidence that on several occasions Joe, my student, has shown me that he has been able to do this, this and this. He has produced the evidence and I can confidently say he is worthy of a distinction grade.”
Stella Paes: The York proposal, in referring to the distinction, is different from the Ofqual proposal in that I think the bar is higher, so the pass as proposed by Ofqual is equivalent to the distinction proposed by York. That is quite interesting. The expectation in the Ofqual proposal is that youngsters demonstrate independence and confidence in the competencies at the end of the two years. In a sense it is a hybrid of what York are proposing, in that youngsters will probably be at a pass level as they work their way through the course because they will have been taught the skills explicitly, but the teacher makes that holistic judgment at the end of the two years and says that what a pass actually represents is a real achievement for the youngster in that they are independent, confident investigators. It is very interesting and very encouraging that an august university has generated very similar recommendations to what Ofqual suggested.
Q14 Chair: Does Ofqual see any merit in this line of discussion?
Michelle Meadows: Using a distinction as well as a pass?
Chair: Yes.
Michelle Meadows: We made the judgment that teachers would most reliably be able to distinguish between pass and fail, given the kinds of skills and abilities that were being assessed.
Q15 Chair: I am not asking you historically about “we” in the past. You listened to the discussion this morning. Is there any merit in what you heard from Dr Swinbank’s proposition?
Michelle Meadows: I think it is very interesting. I would want to see evidence that teachers can reliably distinguish what would be pass criteria and distinction criteria, that the kinds of behaviours that we are looking for and assessing actually can be split into the three groupings in a valid way. I would want to see evidence of that before changing what we have proposed.
Q16 Chair: Presumably, on the basis of your past evidence to us, you would also want to see a rigorous method to ensure that malpractice did not occur.
Michelle Meadows: Indeed, but I think that the kinds of approaches that have been suggested in the York proposal—log books, talking to students, visiting centres, actually seeing the teachers assessing—are very similar, and appropriately deal with that kind of issue as best one can.
Chair: Thank you very much to the York panel, for want of a better word. Thank you for coming. We move straight on to the Wellcome Trust group, please.
Examination of Witnesses
Witnesses: Dennis Opposs, Director of Standards, Ofqual, Professor Peter Main, Institute of Physics, Hilary Leevers, Head of Education and Learning, the Wellcome Trust, David Britz Colwill, Sixth Form Colleges Association, and Helen Thorne, Director of Policy and Research, UCAS, gave evidence.
Q17 Chair: Again, I start with the same question: what do you see as the strengths and weaknesses of the Wellcome Trust approach?
Hilary Leevers: At the Wellcome Trust we did not propose a specific approach in our response. We referred to some work we had done with the Gatsby Charitable Foundation suggesting ways in which we could move forward. What we considered today was something that was like the Ofqual proposal, but included teacher grading—so we retained grading—and we discussed the extent to which we thought teachers were able to discriminate among their students’ abilities. That grading would be incorporated into the actual A‑level grade. We felt that that connection was very important. In order to support the teacher assessment and its quality, we also talked about using a system of peer‑to‑peer moderation, where teachers were more involved in working with schools around them to improve the quality of their assessment, but also that they would have external input from the awarding organisations.
Professor Main: I strongly support the point about peer teacher assessment. We should be looking at treating assessment in a different way from what we have been doing until recently—professionalising the teaching work force so that they can learn to be assessors and to use teachers from different schools to go into other schools to act as the assessors. That in fact gets rid of many of the issues that Ofqual identified with malpractice, and possibly compression of marks.
The other great strength of the proposal is the philosophical point, which I think is at the crux of the discussions: by separating off practical work in terms of grading, one sends a very strong message that practical work is an add‑on to science. This is not the case. Practical work is absolutely essential. It is what science is; it is the core of science. One of the major problems that some of us have had with Ofqual’s proposal—and there are many good features in what they proposed—is the message being sent to some schools, where perhaps costs are important, that practical work is somehow an option, and that it is not at the core of what science is. I think the Wellcome Trust proposal, along with others, covers that point, and that is important.
Helen Thorne: From the admissions perspective, I think the Wellcome proposal offers some significant advantages over the Ofqual proposal. One of the challenges with the Ofqual proposal is that the pass/fail option means that when students are applying to university they are all likely to be predicted to pass by their teachers, so it does not help admissions staff to select between applications. The reality is that universities offering science courses are probably going to say that they all need to pass the practical element as well in order to be accepted on to the course, so it does not address one of the key issues about discrimination between students who are good and those who are most able.
Q18 Chair: Would you like to comment from Ofqual’s point of view?
Dennis Opposs: There are many things in the proposal that are sound, certainly. I think the cluster moderation suggestion, which is a good one, has been shown in other circumstances to be something that is valuable and will work. There is a big question about whether, in the circumstances that currently schools and colleges find themselves, we can create a system that would allow that to work. It requires teachers to visit each other’s schools, and to be out of schools and colleges during the time when they might otherwise be teaching. I think there are some manageability or practical problems in that part of the proposal certainly.
Q19 Sarah Newton: I represent a part of the country where we do not have many schools teaching A-levels. We have unbelievably good sixth form colleges and that is where most students go, so I have a real issue with the practicality of how this cluster would work, because most young people in remote rural areas, which I represent, will be in a handful of institutions. If you want a cluster effect to work, people will have to travel hundreds of miles from other counties to be able to do the sort of cluster peer review that you are talking about. Have you really thought about that? It seems to be a very urban‑type model that you are advancing.
Hilary Leevers: We see the idea of peer‑to‑peer moderation as a fantastic professional development opportunity for teachers, as in a very general sense we are talking more about teachers having a certain amount of time dedicated to their professional development, particularly in their specific subjects—in this case, in science. Some of that time could be allocated to peer‑to‑peer assessment, and it could be that those teachers get some sort of accreditation; it could help lead their career pathways. It is something we have thought about, and an idea like this is being trialled in Ireland at the moment. We are very interested to see how that takes place, noting that Ireland is engaged in a very long process of working out the best way to assess practical skills in science and not trying to rush something in over the time scale that we are.
Professor Main: If I might chip in, another issue which cropped up in the discussion was that some of the points that have been made about malpractice and lack of compression are due not to teachers misbehaving but to them not having enough experience or training in issues of assessment. Part of the idea of peer assessment is raising the standard of assessments by people professionalising themselves.
Q20 Stephen Mosley: The proposals that Ofqual have put forward, they say, have the intention of trying to raise practical skills while addressing issues such as malpractice. How do you see your proposal improving those practical skills?
Hilary Leevers: One thing I would say about the Ofqual proposal is that a lot of what they are doing is side‑stepping the issue. If you look at questions of validity, discrimination or malpractice, rather than seeking to improve the system, they are sort of side‑stepping them by saying, “If it is pass/fail, we are not really looking to discriminate in any fine‑grained way.” In terms of the practical experience that teachers and students would have, we are concerned that, if there is a drop in the status of practical science by partitioning off to one side the practical grade as a very low pass/fail, the teaching of practical science will not be well resourced, in terms of both financial resources and time. If it continues to be part of the A‑level grade, it will be seen as a core part of the science teaching and learning experience and will be something teachers dedicate time to—also students. We talk a lot about the teachers, but students are very good at working out where they need to expend their efforts and time, and if they decide that they just need a little bit to get over the pass, or that it is not particularly pertinent to their ongoing education, they may decide not to engage with the practical work, which means they are undermining the whole of that A‑level qualification because it has to have that element for it to be a valid assessment of the science.
Dennis Opposs: I cannot accept that we are side‑stepping things. We need to appreciate that when we are talking about this separate grade it is not a grade that is all practical science. The proposal is that 15% of the final A‑level grade will be based on assessments of practical science in the exam, and then some of those skills will be assessed separately. We still have assessment of practical in there, in the main grade, as it is now. In fact, at the moment, some of the other skills that we are now talking about reporting separately are not actually in the A‑level at all. One of the things that the whole proposal is trying to do is to raise the status, and raise opportunities for teachers to provide good A‑level courses that will enhance the experience of the students taking them. At the moment one of the problems is that teachers are rather bound by trying to chase extra marks for their students in the part of the assessment that they undertake.
Q21 Chair: Have you produced any model questions yet that would illustrate how that would work?
Dennis Opposs: There are questions that we have from the boards.
Q22 Chair: Could we have some?
Dennis Opposs: I am sure that would be possible, yes.
David Britz Colwill: On the issue about a certain percentage of practical work still being assessed during the exam, the debate we had at the table was that a student who is very competent at a particular practical skill in reality is very different from someone who can theoretically approach that. It is almost like the theory of practical, and I do not think the theory of practical and being good at that is the same as being practically competent. I like that style of questioning and it has a place in an exam; in fact, many awarding bodies do that anyway. AQA, which our centre uses for sciences, do that and we enjoy those questions. But to take away the actual grade value of physical practical work is the thing that we disagree with, and I do not think 15% of questions is enough of a judgment on a student’s competence.
Q23 Stephen Mosley: We were talking about cluster moderation, but the other thing you put forward in the proposal is students maintaining a portfolio and log book. Could you explain how you think that would improve things?
Hilary Leevers: What we discussed was that it was more to evidence that the practical work was taking place and that teachers could use it to discuss the performance of students in the peer‑to‑peer process, but really it was also giving teachers more responsibility for being able to evidence that that has occurred and actually for head teachers to say, “Yes, I can now see from the evidence in front of me that our centre is enabling all students to have rich practical experiences.” One of the teachers at our discussion table talked about how, on hearing the way in which Ofqual was representing how teachers were behaving, they felt quite offended and hurt at the suggestions of levels of malpractice and that teachers could not be trusted as assessors.
Q24 Stephen Mosley: I was at the table at that time and heard that point made, but I also heard one of your other colleagues mention the fact that—it was you, Mr Britz Colwill—there can be a tendency on the A‑levels, which are graded in the league tables, compared with the AS‑level equivalent you were talking about, which is not—
David Britz Colwill: No, it is slightly different.
Q25 Stephen Mosley: Explain.
David Britz Colwill: The outcomes of A-levels, as we know them as subjects—biology, chemistry and physics—for colleges and schools are high stakes. Of course they are; we are judged by them. We were having a discussion about the EPQ, which is a slightly different qualification and could potentially be in any subject. The conversation that was being had at the time was that with high stakes, such as biology, chemistry and physics, where your department is under scrutiny—and your credibility as a person, I suppose, as a teacher in the department—you are obviously going to prepare your students very well. You are going to ensure that they can achieve to the best of their abilities; of course you are going to do that. An EPQ project, which is founded in independence, is their own independent inquiry, and it is a project management qualification as well. It is about the process, them doing it themselves, so it would defeat the object of the qualification for me to prepare them to the same level. One of the criteria is utter independence, so I am less inclined to help and more likely to say to them, “Go away and find out for yourself.” That is part of the process and I would then award them more marks for doing that. Part of being an academic subject teacher is the deal that I give you some information about how the process works. Part of it is going to come from me, so obviously I am going to prepare them in a very different way from a qualification in independent study. That was the point that we were making, and they were saying that perhaps the grades might be more genuine in that respect because it is all theirs. I cannot say, and I do not know what the evidence would say.
Professor Main: One of the concerns we have had for some time about the Ofqual proposals has been that the idea of using an extended investigation as part of the graded assessment, which in some A‑level specifications is one of the most attractive parts, is more or less precluded by the proposals, because you simply cannot have an extended investigation consistent with what is being proposed, with the 12 sets of competencies and so on.
Chair: Thank you very much. Panel 4, please.
Examination of Witnesses
Witnesses: Glenys Stacey, Chief Regulator, Ofqual, Philip Britton, Headmasters and Headmistresses Conference, Sir John Holman, Council for Science and Technology, and Sarah Main, Campaign for Science and Engineering, gave evidence.
Q26 Chair: Finally, what are the strengths and weaknesses of the CST approach, Sir John?
Sir John Holman: The group asked me to introduce this with a summary, and the good news is that a lot of the issues that arose in our discussion have already been discussed in the preceding three groups, so hopefully we will not keep you from Prime Minister’s questions.
The overall point, from the point of view of a Russell group university, from which I personally come, is that with the Ofqual proposal the certificate will not give us information about the practical skills that are so important to what goes on in universities. That is the core problem because, as currently formulated, the Ofqual proposal is for a pass and fail, although you, Chair, described a possible model involving a distinction, which would be much more useful for a Russell group university.
Q27 Chair: That was the York proposal, yes.
Sir John Holman: Yes. Very briefly, the Council for Science and Technology set out two possible models that might be considered as alternatives. One was an extended investigation, and we have had a discussion of its strengths and weaknesses. From the point of view of universities, the great thing about an extended investigation is that for two or three weeks an individual student engages in something that really interests them and over which they have control. Universities find that students who have done these investigations not only have something exciting that they are motivated to talk about at interview, but they have different kinds and a wider range of practical skills when they actually arrive. This is the big strength of the individual investigation. There are plenty of models in existence.
The objection from the point of view of Ofqual and those who are involved in the practicalities of education is manageability. They are a demanding thing to manage and they require a lot of equipment, and we had a good discussion about that. They are clearly not for the world as it is now, but I think we should believe in the possibility of a better world in which this very high form of practical assessment and practical activity could exist some time in the future.
The CST also proposed an alternative and more practicable set of arrangements—a carousel of practical experiments, whereby you have around the laboratory stations of experiments. Pupils, candidates, go round those experiments and carry them out; they might take 10 minutes or so. They are invigilated by a teacher from another school to ensure fair play and they are marked. There are two models: one could be externally marked and one could be teacher marked and moderated, but it involves moderation by teachers from other schools because they are the sternest critics of teachers; they do not like to see other teachers cheating. We think that is an important component of it. When we discussed it, we heard from Ofqual that this proposal was welcomed by them as they developed their proposal, and indeed Ofqual considers—Glenys will correct me if I am wrong—that their proposal is a development of this idea, extended over time to make it more manageable by schools.
Finally, that does raise, in my mind at least, the question that, if Ofqual feels that what it is putting forward is practical, reliable and valid, why don’t we use it to get more information that will be useful to universities, which would be to give it grades beyond just a pass and a fail—a distinction, a merit, a pass and a fail?
Q28 Chair: Is there any rebuttal from Ofqual?
Glenys Stacey: As to taking it out of the grade, we have done it for a number of reasons. In trying to grade it, we come back to the points that Dr Michelle Meadows spoke of just a short while ago: we would need to be satisfied that we could have reliable discrimination for those practical skills by teachers. We do not have the evidence of that at the moment.
Q29 Chair: So you are back to the question about teachers cheating.
Glenys Stacey: I am coming back to a number of things, one of which is the point that Peter made earlier about the professionalisation of assessment; that is the first thing. The second thing is that, if it is within the grade of the qualification, there are all of the pressures that teachers are under and that are well recognised around the table here. The third point is about whether or not when you are assessing by direct assessment those practical skills—much of which are about basic dexterity—you can safely finely grade those skills. We are not looking at the list of individual skills at the moment, but there are some questions about how finely you could in any event grade some of those skills reliably. What I would come back to, though, is that the carousel idea was very welcome—John is right—and we have done our level best to make it into something that is practical, manageable and actually develops those skills for students studying science.
Q30 Chair: Including the external invigilation.
Glenys Stacey: We are talking with exam boards at the moment about how these assessments could be moderated as well as evidenced, and we are very interested in collaboration, if that could be done.
Q31 Chair: It has happened in the past.
Glenys Stacey: That’s right. We are very interested in collaboration.
Q32 Chair: My exams were; I am that old.
Glenys Stacey: I am not far behind you, but let’s see how far we can get with it.
The other point I would like to make, and I know John recognises this, is that within the qualification itself—still graded and subject to all the pressures of the qualification—15% of the marks are allocated to questions that relate to the student’s understanding of designing experiments, understanding experiments and asking the right questions. It is going to be difficult for a student to succeed sufficiently well on those questions unless they have real experience of experimentation.
Q33 Chair: That is where the debate is.
Sarah Main: Could I just add something?
Chair: Very quickly.
Sarah Main: My independent hearing of the discussion was that, in the CST 2 proposals for the extended investigation and the practical carousel, it seemed that the strengths were an increase in validity, in the sense that they were testing a wider range of skills that universities say they want; an increase in discrimination between students, because the evidence from people who already do extended practicals is that they have much wider grade ranges; a decrease in malpractice and an increase in teacher professionalisation. In other words, they have feedback from people who already do these that, if you have very wide grade ranges, it indicates that you are not getting bunching at the top end, that teachers are discriminating widely across grades and that you are providing skills required by universities and employers. In the proposal, both those models are in the A‑level grade and, therefore, as the lady from UCAS mentioned earlier, they become practicable for universities to use in admissions, and the message to society is that they are highly valued.
From what I heard, the weaknesses were mostly around logistics, and it struck me across the whole conversation that what Ofqual had proposed was the best that could be achieved now. There were a number of logistical problems that they considered but they seemed to me to be not insurmountable, sometimes because other people already do them—Northern Ireland already has a carousel process—and sometimes, with local school moderation, it has been done in the past, as you say. A number of logistical issues were raised, such as the extended project being hard for large centres to do, and not using direct assessment, but I fear that—to put it more positively, I hope that we would not tend to the lowest common denominator, and that we would set our ambitions high and not simply set them at the lowest provision that any large centre can provide.
Q34 David Tredinnick: Building on what you are saying, do you think there is a fundamental problem in the increased emphasis on practicals that some schools simply do not have the right equipment to deliver proper experimentation? There is an in‑built disadvantage for the least well-resourced school.
Philip Britton: That is a question around one of the points I would make, which is aspiration. In terms of this proposal, which is an aspiration to have investigative work in schools, which requires professionalisation of teachers, and looking at resources and at proper ways of making assessment valid, I think the counsel of despair that “The best we can do is whatever the best we can do now is,” is not wise. One of the outcomes of the current phase of discussions ought to be an aspiration to work towards a better system in the future while attempting to make the system we have now as workable and productive for practical science as we can.
Q35 David Tredinnick: Sir John, how much contact did you—the Council for Science and Technology—have with Ofqual when you were developing these proposals?
Sir John Holman: We had several discussions; Glenys may correct me. The Council for Science and Technology had quite extensive correspondence with Ofqual, and it must have been—
Glenys Stacey: Several meetings, yes.
Sir John Holman: At Glenys’s level—a number of interactions.
Q36 David Tredinnick: And the chief scientific adviser?
Glenys Stacey: Yes, I met the chief scientific adviser, with John as well actually.
Q37 David Tredinnick: Having listened to all the sessions, do you share my concerns with Ofqual that there is a danger that some coursework is not actually part of the formal assessment on the grounds that it increases the likelihood of cheating and that this is something that needs to be addressed?
Glenys Stacey: The validity of the qualification is influenced by a number of things: obviously, the content, the nature of the assessment and the quality of the assessor, but also the pressures that the qualification might be put under. Those pressures might be, for example, if the student’s, the teacher’s and the school’s performance is judged by the grade, and then the financial pressures as well—the equipment and so on. You have quite a complex interplay when you are thinking about the validity of the qualification, not just about its design but about the resources that are applied to it and the pressures that are imposed upon it as well. Yes, of course we are concerned to produce the qualification that is best fit for purpose and can best deliver the right educational outcomes, actually taking into account those pressures. If we do not do that, we produce something that is potentially not fit for purpose.
Q38 David Tredinnick: The last question, building on Sarah Newton’s point, is do you think we really need a town and country solution, because outside assessments in very sparsely populated counties would be a major challenge?
Glenys Stacey: I believe that the Ofqual solution, as it is called, is a town and country solution. It is deliverable whether the school has 20 students or 200, and whether it is in Norfolk or Nottingham. It is deliverable.
Q39 Chair: In terms of Sir John’s proposition, do you think that is still workable? We understand Ireland is appraising it.
Glenys Stacey: My own view is that the carousel proposition would be relatively high risk, not only from the town and country perspective and travel, but because—
Q40 Chair: Yes, but you have been told your proposition has high risk from the point of view of universities and employers.
Glenys Stacey: Yes, but you are asking me about Sir John’s proposal. The carousel is concerning for a number of reasons, not least the issue about the skills of teacher assessment, which we would need to be sure about, but also the practicalities of designing that on any one day and in any one school: do they have the right amount of equipment in that school for the right number of students, and what about students that are not able to attend that day or can only get halfway through it? There are lots of very practical issues about that. It would be for us a relatively high risk solution. But we think that what we have proposed and decided instead, which is taking those skills and seeing them develop over a two‑year period and seeing the evidence that they have been done and assessed, is a lower risk way of getting to the same position. On the extended project, the issues for us are a little different, but I am happy to talk about those if you wish.
Sir John Holman: On your specific question about town and country and the difficulties for rural communities, we must remember that science is only one of a number of subjects that is assessed—uncontroversially, in many other subjects. If we take music and art, these are subjects that involve a great deal of teacher assessment and indeed inter‑school moderation. If it can be solved for those, it can be solved for science.
Could I pick up one other point which was made earlier about the 15%—the notion that within the written paper 15% of the marks will go on questions that are designed to assess practical skills? When I talk to teachers about this, they smile. They smile because they are past masters at preparing young people to pass exams. They are smiling because they are thinking about how they would teach their children to pass those questions without necessarily having done the experiment. I was very encouraged earlier to hear Mr Opposs tell us that they have some specimens of the kind of questions that will do that sort of thing. It would be very valuable, I am sure, to the Committee and for others of us who are concerned about this if we could see them and if we could see the evidence of what works well.
Q41 Chair: We can then examine Ofqual in the traditional manner.
Philip Britton: That is absolutely crucial. As I have heard it this morning, the whole future of practical work in school hinges on being able to answer those 15% of questions only if you have done meaningful practical work in schools, so the burden placed on those questions actually doing what Ofqual say they will do is enormous; if they do not, they will not be drivers for heads and leaders of schools to commit funding to practical science, and therefore practical science will be in difficulty. That is the crucial area for scrutiny.
Chair: That is the challenge for all of us. Thank you very much for your attendance. Thank you to all of the witnesses and the participants in the round table. It has been an intriguing morning. This is a different approach to doing Select Committee work, so we are conducting an experiment today, and hopefully people outside watching this will evaluate it as well. Thank you very much.
Oral evidence: School practical science, HC 329 18