WOO0026

Supplementary written evidence submitted by Ofqual

Setting Grade Boundaries

1. Exam boards are responsible for marking and setting grade boundaries (awarding) and Ofqual oversees the process as the qualifications regulator. Ofqual’s priority during summer awarding is to align standards between exam boards in a subject and over time. So it should be no easier, or difficult, to get a particular grade with one board than another and, all things being equal, a student who got a particular grade in a subject last year should get the same grade this year. 

Marking

2. In most subjects students take their exams in May and June each year. Examiners are trained to mark to a required standard using a mark scheme, which provides guidance about how marks should be awarded. Before they can start marking, they have to show that they can mark a sample of ‘live’ answers to the required standard. 
3. During the marking window, which is usually 3-4 weeks for each paper, each examiner's work is quality checked by their respective exam board to ensure their marking is consistent, and to the required standard. The types of check vary depending on whether scripts are marked on paper or electronically (on-screen), as well as whether they are marked by question or as a whole paper.

Setting grade boundaries (awarding)

4. The process of setting grade boundaries is prescribed in Section 6 of the regulators’ Code of Practice[1].  They key purpose of the process is to ensure that standards are maintained from one year to the next.
5. Once marking is complete or almost complete, grade boundaries are determined in a process typically referred to as ‘awarding’ or ‘grading’. This involves determining a grade boundary with the use of both quantitative and qualitative evidence and convening (either face to face or remotely), an awarding committee for each specification.

The Task of the Awarding Committee

6. For each specification or syllabus an awarding committee is convened. The awarding committee usually consists of:

(a) a Chair of Examiners who has overall responsibility for maintaining standards across different syllabuses in a subject area;

(b) the Chief Examiner who is responsible to the Chair for the syllabus as a whole and;

(c)  Principal Examiners who are responsible for the marking and/or question setting for the individual papers (units) that make up the syllabus.

7. The committee’s task is to recommend grade boundaries for each of the units within the qualification so that there is comparability of standards both at unit level and qualification level from one year to the next.
8. This is achieved by combining:

(i) Qualitative information in the form of expert judgement on the quality of performance seen in the candidates’ work (‘scripts’), compared with performance of students in previous years.

(ii) Quantitative information on current and also previous sessions. This includes information about the mean mark achieved on each paper, to give awarders a sense of any difference in the difficulty of one year’s paper compared to previous years. It also includes the prior attainment profile of the cohort of students who have taken the unit/specification.  For AS/A level the profile of prior attainment of the cohort is constructed from mean GCSE score.  For GCSE, the profile of prior attainment is based on Key Stage 2 results. These prior attainment profiles are used to determine the predictions for each syllabus (see below).

9. It is the job of the Chair of Examiners to weigh all the available evidence – quantitative and qualitative – and recommend a single mark for each of the grade boundaries.  These recommended grade boundaries are then subject to further inspection within the exam board before they are signed off by the Responsible Officer (usually the Chief Executive).

The principle of comparable outcomes

10. Exam boards have always used statistics to guide their decisions about grade standards. These decisions are based on consideration of different sources of evidence – including current and past student work, data about the prior attainment of this year’s students compared with previous years, and recommendations from the senior examiners.

11. In 2001 and 2002 discussions between exam boards and regulators about new ‘Curriculum 2000’ A levels were focused on how best to avoid disadvantaging the first cohort to sit these new A levels. Exam boards could have prioritised what Cresswell (2003)[2] referred to as ‘comparable performance’ – expecting students to demonstrate the same level of knowledge, skills and understanding as in previous years in the previous qualifications. Or they could have prioritised ‘comparable outcomes’ – roughly the same proportions of students achieving each grade as in previous years, providing the cohort of students is similar to previous years.

12. It was agreed that exam boards should prioritise comparable outcomes, to avoid disadvantaging the students who were the first to sit these new qualifications. This approach was also used for new A levels since 2010 and for new GCSEs since 2011 and 2012.

Predictions based on prior attainment

13. In recent years Exam boards’ use of statistics in setting grade boundaries has become more sophisticated, based on the prior attainment of the cohort. Exam boards use prior attainment at Key Stage 2 to predict GCSE outcomes, and prior attainment at GCSE to predict AS and A level outcomes.

14. One of the advantages of using predictions based on prior attainment is that it gives us a way to compare grade standards across exam boards. The predictions are based on national results from previous years, and so provide a common basis for all boards to use. They are then split into predictions for each board, based on the prior attainment profile of each, so that each board’s prediction reflects the prior attainment profile of its entry.

Our role

15. Each year we agree the bases of the predictions to be used to guide awarding decisions. This is published in the data exchange procedure, which is a regulatory document and published on our website each year.[3]

16. The data exchange procedure sets out the reporting tolerances which the exam boards must apply when sending data to us. If the results differ from the prediction by more than the tolerance, then they must provide additional evidence to support their decisions. The reporting tolerances vary according to the size of the entry.

17. We review the data from every GCSE, AS and A level award, and we pay particular attention to awards where one or more grade boundaries are out of tolerance. We can, and do, challenge exam boards where we do not believe the supporting evidence is sufficiently strong, and we publish a summary of the out of tolerance awards at the end of the summer.[4] If necessary, we can direct an exam board to set different grade boundaries, although to date we have not needed to do this. As a result of our approach, significant year-on-year changes in performance at the national level are not likely. However, if the exam boards have evidence of genuine improvement, the comparable outcomes approach allows that to be reflected in results. 

Summary of grade boundaries for GCSE in 2015

18. The key grade boundaries for GCSE are grades A, C, and F.  There are three types of GCSE unit.

19. In an untiered unit, all grades are available. We would therefore, expect the grade boundary mark for a C to be towards the middle of the available marks.

20. In a higher tier unit, only grades D and above are available. C is therefore the lowest grade boundary set by the awarding committee and therefore we would expect it to be set at a lower proportion of the available marks, compared with an untiered unit.

21. In a foundation tier unit, only grades G to C are available. C is therefore the highest grade boundary set by the awarding committee and so we would expect it to be set at a higher proportion of the marks compared with an untiered unit.

22. The table below shows that, across all units, these expectations are borne out in practice. On average, the proportion of marks needed to achieve a C is highest on a foundation tier unit and lowest on a higher tier unit.

Table 1: Mean boundary location expressed as a percentage of the maximum mark for GCSE units awarded in 2015.

Summary of AS/A level boundaries in 2015

Table 2: Mean boundary location expressed as a percentage of the maximum mark for AS/A level units awarded in 2015.

Awarding reformed qualifications

New AS and A Levels

23. We have already agreed that we will carry forward standards at A* (A level only), A and E so that, in general, a student who would have achieved a particular grade on the old syllabus will receive the same grade on the new. Exam boards will use statistical predictions based on the cohort’s prior attainment at GCSE to achieve this, and to make sure that grade standards between boards are in line.

New GCSEs

24. New GCSEs in English language, English literature and mathematics in 2017 will use a new grading scale: 9 to 1 instead of the current A* to G.

25. We have agreed that exam boards will use statistical predictions for the cohort, based on prior attainment at Key Stage 2, to provide ‘anchor points’ between new and old grades, as follows:

• Broadly the same proportion of students will achieve a grade 4 and above as currently achieve a grade C and above.

• Broadly the same proportion of students will achieve a grade 7 and above as currently achieve a grade A and above.

• The bottom of grade 1 will be aligned with the bottom of grade F.

November 2015