Science Education Policy Alliance – Written evidence (EDU0099)

 

  1. Following a meeting with Lord Watson at the pan-science Parliamentary Affairs Committee, the Association for Science Education, Institute of Physics, Royal Society, Royal Society of Biology and Royal Society of Chemistry are writing to the Education for 11-16 year olds Committee to provide further evidence on the issue of grading severity in the sciences and inter-subject comparability. At the committee Lord Watson raised the issue of the “forgotten third” of students, those that do not attain Grade 4 or above at GCSE, and was asked if the committee had been considering the value of the whole range of grades at GCSE. Lord Watson invited SEPA to submit further evidence to the committee and we welcome this opportunity to provide some background on grading severity and inter-subject comparability ahead of the committee’s oral evidence session with Ofqual.
  2. The Science Education Policy Alliance (SEPA) brings together the Association for Science Education, Institute of Physics, Royal Society, Royal Society of Biology and Royal Society of Chemistry to coordinate and collaborate on science education policy, with Professor Dame Athene Donald as Chair. This partnership originated with SCORE (Science Community Representing Education) in 2008, and following dissolution of that group in 2015 our organisations have continued to work together to provide a strong collective voice for the sciences. We would welcome further opportunities to present evidence to the committee and are happy to engage with any questions the committee may have on the joint evidence submitted here, or our individual submissions to the committee as part of their inquiry.

Relative difficulty and grading severity

  1. The relative ‘difficulty’ of the sciences (especially physics) and mathematics is often talked about but rarely understood. If we ask, ‘is sprinting more difficult than swimming?’ the question obviously has no meaning – they require different capabilities and people have different aptitudes. The comparison only becomes valid when you set standards for each discipline and compare the number of people who achieve those standards. We can ask, ‘is it more challenging to run 100m in 15 seconds or swim 100m in 60 seconds?’ and draw conclusions from the world records for each.
  2. In the same way, it is meaningless to ask whether physics is more difficult than biology. However, we can meaningfully ask whether it is a greater challenge to get an A grade at A-level in biology or mathematics. The answer is that they are about the same. But that is not true for, say, English.
  3. The issue is not the inherent difficulty of a subject, but how the exams are graded. Put simply, physics, along with French, biology, mathematics and chemistry, is graded more severely than subjects like English, business studies and art. A level grades in these groups of subjects are therefore not comparable.
  4. While we can identify the problem, there are barriers to solving it. The obvious response would be to lower the grade boundaries in more severely graded subjects, but this would further exacerbate the lack of discrimination between top grades. And, of course, the alternative – to raise grade boundaries in other subjects – is a hard sell to practitioners in those subjects.
  5. As things stand, Ofqual is required to make each year’s level of difficulty comparable with the previous year, within (but not across) subjects. Any small adjustment from one year to the next would require special dispensation.

Grading severity in the sciences and inter-subject comparability – SEPA organisations and Ofqual

  1. In 2008, a Durham Curriculum, Education and Management Centre (CEM) report commissioned by SCORE found that the sciences, mathematics and languages are graded more severely than other subjects at A-level: up to a grade more severely in some cases. Though the Education for 11-16 year olds Committee’s focus is not on A level, the progression from GCSE to A level is certainly important to consider as part of the 11-16 school experience and the choices students make as a result.
  2. In 2016 SEPA organisations wrote to Ofqual’s Chair asking Ofqual to consider its future policy on inter-subject comparability. Ofqual held several roundtable discussions with our organisations, hosted an inter-subject comparability conference and in 2017 committed to undertaking research to further investigate the matter via a comparative progression analysis from GCSE to A level.
  3. In the letter, we presented Ofqual with evidence in the graph below, which shows the final grades (at A level) of students who achieved a grade B in the same subject at GCSE. There are two distinct distributions. In the sciences a student achieving a grade B in, say, biology at GCSE is most likely to achieve a D in A level biology. In the humanities and English, they are most likely (and more likely) to achieve a C at A level. Indeed, 70% of students achieving a B in GCSE geography progressed to a C or above at A level geography. The equivalent figure for each of the sciences was more like 47%.

  1. The following table shows distributions of prior GCSE grades across A level subjects and is presented in Ofqual’s 2017 report Progression from GCSE to A level. The chart shows the grades that students achieved at GCSE in a subject who then took that subject at A level. For example, 50% of A level biology students had achieved a grade A* at GCSE (and it is the modal grade). The same is true for the other sciences and mathematics, which have a similar distribution. However, the distributions are different for the humanities (modal grade: A) and English (modal grade: B). As well as showing how different the prior attainment of the intake is and how this should affect outcomes, it also shows that there is some form of selection going on – ether overtly or self-selection by students – based on the perceived differences in requirements to get high grades at the end of the course. That is to say, this chart demonstrates that it is highly likely that the differences in grading severity affect students’ choices at 16 – either directly or by implication as they affect the reputation of the sciences for being “difficult”.

  1. Our organisations wrote to Ofqual welcoming the findings in the Progression from GCSE to A level, and urged Ofqual to move on from discussing the validity of the analysis methods to considering how to address a genuine problem in grading standards.
  2. In the most recent consultation (2018), Ofqual asked whether to seek permission to adjust grading standards in the sciences and languages. Our organisations pushed strongly for adjustments to make grading more comparable between subjects, but Ofqual concluded that the differences in grading severity have no effect on student choice, so chose not to address the issue. In 2018 Ofqual announced that while adjustments would be made to modern foreign languages, and while they recognised the issue at A level in the sciences, no changes would be made except to ensure the issue of grading severity did not worsen. Further investigation was not carried out at GCSE for the sciences despite the sciences and languages falling into similar patterns of skewed distributions of GCSE grades across A level subject choice. Ofqual have not accepted that this issue affects student choice – and therefore chose not to address the problem.
  3. Our organisations submitted a joint letter to the Chair of Ofqual raising concerns that no adjustments were to be made to grading standards for A levels in biology, chemistry and physics even though incomparability in grades was acknowledged. And reiterating the damaging consequences this lack of comparability can have on the progression from GCSE.
  4. Our organisations received a reply in August 2019, which stated that Ofqual did not find that all of their criteria were met in order to form a compelling case for an adjustment had been fulfilled, including any potential impact of grading severity on a subject’s popularity i.e. application numbers and supply of teachers. With falling numbers applying to study chemistry at university, and the collapse of teacher training applications to teach the sciences the landscape has certainly changed since 2019. NFER’s Teacher Labour Market in England annual report 2023 shows all sciences failed to meet the 2022/23 ITT recruitment targets.
  5. Ofqual intended to review the impact of its decision after the 2019 A levels. However, cancellation of exams in 2020 has made this difficult to meaningfully pursue. We do not know whether the issue of inter-subject comparability and grading severity in the sciences has improved or worsened following disruptions to exams and awarding processes since 2020.
  6. In 2021 our organisations commissioned further research into entry requirements for post-16 qualifications – this research found that at least on paper, schools do not advertise higher entry requirements for studying the sciences post-16. However, a student voice survey conducted as part of that research and the reality of the perception of sciences suggests that knowledge of the severe grading does affect student choice either directly (using prediction systems) or indirectly (as it plays into an image of the sciences and mathematics as being ‘difficult’).

Quantifying the problem of grading severity

  1. In 2008, the Curriculum, Evaluation and Management Centre at Durham published Relative difficulty of examinations in different subjects, which used a range of methods to determine whether grades in different subjects are comparable. Each method showed that they are not. The most obvious method is a regression analysis to GCSE grades (in blue).
  2. The effect of this differential severity is frustrating and inequitable. And it is likely that it is putting students off choosing the sciences and mathematics – at a time when it is government policy to increase the number of students taking these subjects. This is especially true if schools use a system for predicting a student’s likely grades at A level (which is based on previous years’ results – which are skewed through incomparable grading severity).
  3. In effect, for a given set of GCSE grades, predicted outcomes at A level will be lower for any of the sciences than for English, business studies and so on. Students whose grade requirements are not tied to specific subjects are incentivised to select A level courses that offer a more likely outcome of a higher grade – irrespective of the skills they may want or need for the future.
  4. Ofqual’s analysis found chemistry to be the third most severely graded A level subject in the two years which were scrutinized (2013 and 2017). Students with prior attainment of a grade B in GCSE Chemistry were the most severely graded at A level (except for physics).
  5. The following table presented in CEM’s Relative difficult of examinations in different subjects compares five different methods for comparing difficulties of 12 subjects, ranging from “easier” to “harder” left to right:

  1. It has also made the sciences self-selecting: candidates in the sciences tend to have high average GCSE scores, which results in more of them getting the top grades at A-level. In 2022, 40% of candidates got A/A* in physics.
  2. Published today (28 June 2023) Education Datalab’s Revisiting subject difficulty at Key Stage 4 further explores the relationship between attainment and grading severity at GCSE by comparing the difference in points score achieved in GCSEs with English language and mathematics for the 2022 cohort.
  3. In data from the 2019 A level cohort, physics, chemistry and biology, Education Datalab’s 2022 analysis notes were graded more severely than mathematics – used as comparison as it is the most popular A level. In the case of students studying physics, mathematics and chemistry at A level the lowest grade of the three was generally in chemistry.
  4. This reinforces the notion that the sciences are only worthwhile for those with high prior attainment. Nothing could be further from the truth: like any discipline, each of the sciences can be practised at many different levels and all grades at A-level have value.

 

7 July 2023

5