RETHINKING ASSESSMENT – WRITTEN EVIDENCE (YUN0081)

Youth Unemployment Committee inquiry

This document is an edited version of an internal summary document initially provided to the Advisory Group of the Rethinking Assessment organisation. It outlines our key principles, developments to date and priorities for the coming years. It is best read as a working document, and is extensively supplemented by materials on our website, www.rethinkingassessment.com. We trust it gives a flavour of our arguments for change, and our thinking about how to bring it about. There are real alternatives to the status quo, and it is our view that a new approach is necessary to equip young people for fair access to fulfilling employment.

The Context

Exams have a stranglehold on our entire education system. They dominate what is taught, how it is taught, and lived experience of every child, parent and teacher in the country.

Many young people find the way our exam system works increasingly stressful and not a true reflection of what they can do or are good at. The arms race for grades is brutal, and the notion of ‘raising standards’ redundant; the GCSE system necessitates that the bottom third fail.

Headteachers feel that high stakes exams distort priorities and stops them from providing a well rounded education for their pupils. Our assessment is not giving universities, colleges or employers the kind of information they want, or evidencing the kinds of dispositions and capabilities which help young people to succeed at school and in life.

About Us

Rethinking Assessment is a broad coalition of school leaders from the state and independent sectors, researchers, policy-makers and employers working with a sense of urgency to find the best ways of evidencing the full range of young peoples’ strengths.

You can see the members of our Advisory Group here.

Mission: To create a rigorous and equitable assessment system that recognises the strengths of every child by building compelling arguments, powerful coalitions, and practical solutions

Our aims:

1. To make the case for change through case studies, analysis, evidence, and thoughtful blogs
2. To provide workable solutions, practical ideas and approaches (from classroom level to system level) that we can pilot and offer as real alternatives
3. To build a coalition for change that mobilises and harnesses teachers, parents, employers, pupils and other stakeholders
    

Our work to date has focussed on five main strategic objectives:

1. Scanning the world for promising practices

2. Coordinating working groups to
+ explore ways of better recognising academic strengths
+ recognise broader skills & dispositions
+ consider the role of technology
+ develop a holistic learner profile and understand system change

3. Identifying a portfolio of action research pilots for 2021-2023, each with a research partner, to test out our thinking

4. Building the coalition and making the case for change

5. Forging strategic partnerships and raising resources
 

Our end goal is to reform what is measured, evidenced and valued in education, in pursuit of a more expansive and holistic education for all young people. We are working towards the creation of a new flexible learner profile that can be used across the world, which recognises the full breadth of a young person's strengths, and provides a richer set of information (beyond just numbers and letters) for further and higher education providers and employers.

Our theory of change is that assessment reform leads to curriculum reform - broaden assessment and the curriculum becomes broader and more flexible. We envisage the learner profile being a catalyst for wider impact, in providing an incentive for schools and colleges to focus on a broader and more balanced offer. Over time, the profile may form the basis of a new qualification that has the potential to bridge the long-standing divide between academic and vocational learning, bringing different forms of learning together under one unified qualification.

Why does assessment need to change?

This is a critical area where change is needed because the current assessment system:

• provides no evidence of the strengths of every child (particular harm to those with specific learning needs or who are neurodiverse) 

• relegates and undervalues vocational education, and does not value essential skills or work-related learning 

• does not provide employers with information about wider employability skills or dispositions

• narrows the curriculum, making exams the key driver 

• creates huge pressure and stress on teenagers reducing motivation, and at a critical time for brain development

• labels a third of young people a failure because exam results are comparable

In making the case for change, we have produced:

1. Over 40 blogs from a range of people and organisations, interrogating a range of perspectives asking the big questions. These are best viewed here

2. Case for Change materials, identifying promising practices

Including our key publication so far - Rethinking Assessment: The Case for Change

Professor Bill Lucas outlines the case for assessment reform in association with the Australian Centre for Strategic Education.

3. Webinars and events with politicians and policy makers, held in partnership with the Edge Foundation and other organisations

4. News articles and podcasts, including:

Guardian: Let’s not return to flawed exams. We have better ways to assess our children

Schools Week: The battle for the future of assessment

FE Week: Assessment Reform

FT: Educators around the world seek to take axe to exam based learning

Telegraph: Its time to scrap GCSEs

Rethinking Education podcast

Rethinking Assessment

Working Groups

In January 2021 we established two main working groups, comprised of members of our Advisory Group and additional specialists, who have spent 6 months undertaking a design thinking inquiry.

Working Group structure and teams looking at specific thematic focus areas:

With support from a project team at the UCL School of Management who have provided in depth research and analysis for consideration

Explaining the relationship between, knowledge, skills and dispositions

The following text is drawn from a paper written by Bill Lucas for the Skills & Dispositions Working Group

On several occasions so far we (and other considering these matters) have struggled with definitions. What is a disposition? What kinds of knowledge are we really talking about? Where do skills fit?

In our work at RA it might make sense to think of two entities, Knowledge and Dispositions, with skills as the ‘connective tissue’ between them.

Knowledge and dispositions are not polar opposites, just different ways of categorising what we can learn. The ‘currency’ of both is skills. As we practise a skill in different contexts we become more competent, confident and capable until it becomes a disposition, something we are disposed to do.

• A disposition is a skill or set of skills which has been so well practised in a range of settings that it has become a habit and a learner is routinely disposed to do it. Creative thinking is a disposition made up of clusters of skills associated with, for example, imagination, curiosity, making connections, perseverance, challenging assumptions, idea generation, risk-taking and several others. Listening is a disposition with a smaller number of skills involving paying attention, maintaining eye-contact, empathy and thinking before responding.

• Dispositions vary in size just as knowledge categories do. Science is bigger than biology. Biology is bigger than mitosis. Creative Thinking is bigger than Imagination which in turn is bigger than connection-making.

• Progression in the development of dispositions can be described in terms of strength, breadth and depth. Strength is the level of independence demonstrated by pupils in terms of their need for teacher prompts or scaffolding, or their need for favourable conditions. Breadth describes the tendency of learners to exercise a disposition in new contexts or in a new domain. Depth is the level of sophistication or understanding a learner shows in applying a disposition and the extent to which the application is appropriate to the occasion.
 

Skills are what matter in life and ultimately what gets assessed.

Planning an essay. Delivering a speech. Critiquing an argument. Having a good idea when you need one. Tying your shoelace. Trying different approaches when faced with a tricky problem. Using your common sense when your satnav takes you to a cul-de-sac not yet updated in its software. Reading the mood of those with whom you are working. Facilitating a workshop where you are a content expert. Facilitating a workshop when you have only a basic knowledge of the context but can transfer facilitation skills learned in other contexts to the task at hand.

And, yes, recalling decontextualised information in a pencil and paper examination is also a skill, but not one that adults need to use much in a digital age...

Skills, as the list above shows, are an element of knowledge in action and of what we are doing when we are disposed to behave in certain ways i.e. exemplifying a disposition.

The deeper your knowledge and the more you practise your skills in a variety of contexts, the more capable you become. Dispositions are clusters of skills which have been practised so well that they have become habitual; you are routinely disposed to deploy them. And skills are the mechanism by which knowledge is applied and dispositions are lived out.

In describing the kind of education we might like young people to receive today to prepare them for a changing world, global bodies such as the World Economic Forum, the OECD, UNICEF (and many researchers and policy-makers) tend to use other near synonyms for dispositions - capabilities, competences/competencies, habits/habits of mind and non-cognitive skills.

Defining the problems with our assessment system

The working groups started with analysis and definition of the problems underpinning our current approach to assessment. The following problem statements were created through deliberation and crowdsourced contributions from the wider Rethinking Assessment community.

Firstly, the recognition of Academic Strengths

What we assess: The problem is that the conception of knowledge in our education and assessment systems is too narrow in the following ways:

1. There is an emphasis on quantity of factual knowledge over quality, and a lack of deep thinking, problem solving and understanding as a result

2. There is a narrowing of options early eg limited breadth post 16
3. There is too little interdisciplinary learning
4. There is a lack of applied knowledge - knowing how as well as knowing what
5. There is a lack of diversity and range of voices, reflective of breadth of diversity of society. Doesn't cover new knowledge eg AI/environment
6. Knowledge is defined in terms of the head (embodied cognition)

How we assess: The problem with how we assess knowledge is that it neither recognises the depth and breadth of knowledge young people need, nor assesses students in ways that allow them to demonstrate the range of their knowledge. The problem is because:

1. Written exams are one dimensional

2. GCSE assessment, although criterion-referenced, is effectively norm referenced which builds in unnecessary competition and forces a third to fail
3. Exams are timebound meaning that students can't necessarily demonstrate all that they know and/or the depth of their thoughts in the hour or so they are given to do so
4. Fixed age single point assessment assumes uniform pace of acquisition of knowledge and understanding. One approach fits all - the system artificially selects lengths of courses and points of assessment.
5. There is no agency for students.
6. Exams are less good at manifesting the skill with which different students use, combine, and/or amplify knowledge

Secondly, the recognition of Skills & Dispositions

What we assess: The problem is that our education system does not recognise or evidence the key dispositions which, alongside academic strengths, help young people to succeed at school and in life.

1. Some key dispositions for learning are not developed systematically in all schools because they are not mandatory and not assessed

2. There is a lack of national agreement on which skills/dispositions are important and valuable despite broad consensus internationally and among employers, FE, HE, schools
3. Most exams are written/text based and these do not enable learners to demonstrate their wider skills and dispositions
4. The current Ofsted framework privileges curricula ‘coherently planned and sequenced towards cumulatively sufficient knowledge and skills for future learning and employment’ with character/dispositions seen only as an aspect of personal development. As a consequence dispositions have low currency when it comes to accountability.
5. Schools have largely failed to learn from (and adapt to) strength-based assessment approaches increasingly used in other settings including workplaces, HE and FE.

How we assess: The problem is that key dispositions currently lack currency within our education system, and there is a lack of knowledge/understanding about credible methods of assessment.

1. Where in the world are broader skills and dispositions being taught (in schools/colleges) and assessed in a way that is credible, reliable, and has currency (with Universities, Employers, Parents etc)

2. What can we learn from these examples - what does the assessment look like (teacher assessment, self assessment, peer assessment), how is currency and validity assured (eg what entity provides validation/certification), how are teachers trained in teaching and assessing broader skills/dispositions

The consequences of ‘What’ & ‘How’ we assess currently includes:

•  Poor mental health/stress and wellbeing amongst learners

•  Poor learner motivation and low self esteem

•  Teacher attrition due stress, poor mental health and wellbeing

•  Forgotten third of young people who leave with nothing

•  Narrow curriculum, and early narrowing

•  Lack of options/tracking/streaming - due to fixed point in time assessment

•  Employers saying young people don’t have the right skills - lack of breadth, lack of relevance,
the system doesn’t develop/deliver the skills/dispositions they need

•  Universities complain that system poorly prepares students for independent thinking,
collaboration, complex thought etc

•  Learners who don’t excel academically (at age 16) are demotivated, failed, disincentivised from learning, and developing what is not recognised further

System level considerations connected to assessment reform includes:

•  League tables (exam results based). Subject hierarchies (Ebacc)

•  Exam boards and specifications (commercial aspect)

•  Poor assessment literacy amongst teachers

•  Student choice in when to take assessments - stage/age

•  Credibility and currency of existing qualifications

•  Manageability and scalability of multi modal forms of assessment
 

In Summary

What we Assess The content of what we are currently assessing is too limited and narrow and focused on knowledge acquisition. We know that a much wider portfolio of skills, experiences and competencies are what young people need, and both employers and wider society wants.

How we Assess Exams are a limited way of assessing learning, with an emphasis on factual recall over deep thinking. We rely on written timebound exams which don't give the full picture and are biased towards linguistic skills and those good at working under timed conditions.

We need a more balanced, holistic and “multi modal” approach to assessment which supports ongoing learning and enables young people to demonstrate what they know and can do.

Horizon Scanning

See paper by Bill Lucas and research document on assessment innovations by UCL

Analysing & Synthesising - an overview of our inquiries

Generating Ideas

Our working groups formulated a series of questions around which to establish pilot projects to test hypotheses and develop answers.

WHAT WE ASSESS How might we create new assessments to evidence the range of capabilities valued by employers, colleges and universities?

•  Create a new literacy assessment that is recognised widely

•  Create a new numeracy assessment

•  Create a range of interdisciplinary courses and assessments

•  Create internships, real world placements and PBL that are assessed/validated rigorously by employers

•  Assess creativity in English, Maths/science, Music and in its own right

•  Assess collaboration in English, Maths/science and PE and discretely?

•  Assess critical thinking in English, maths/science and PE/music and discretely

•  Assess oracy in English, maths/science and PE/music and discretely
 

HOW WE ASSESS How might we conduct a series of rigorous pilots in schools demonstrating validity, reliability and practicability? .

•  Create credible oracy components to current exams e.g. Maths A level, History GCSE?

•  Create a successful oracy assessment in its own right based on the 4 strands of oracy

•  Devise successful open book exams

•  Devise successful exams with extended time or no time limit

•  Devise successful, moderated, ‘unbiased’ teacher assessments

•  Assess the EPQ or HPQ through a portfolio of work

•  Devise successful ‘observational assessments’ (similar to medicine and dental OSCEs)

•  Devise and pilot a comprehensive learning record

•  Devise a credible digital profile transcript that brings together the different components of the comprehensive learning record?

•  Devise assessments so that students take them when ready?
 

1. Trialling different modes of assessment which allow for deeper thinking/problem solving/demonstration of breadth and depth of knowledge and understanding

Proposed Modes we are most interested in:

•  Oral assessment - Vivas

•  Written assessment - Open book exams. On demand. Non timed

•  Multimedia/digital - eg games based assessment, AI driven

•  Project based/research
 

Options that have been considered by the multimodal team

2. Creating breadth in both content and forms of assessment through Interdisciplinary Learning and Applied Learning. Includes assessment/evidencing of knowledge and dispositions

Types: Non accredited projects/courses. Accredited projects/courses
 

Ways of doing this:

•  Developing interdisciplinary projects/courses within existing curriculum and through extracurricular programs

•  Adapting existing qualifications - such as HPQ/EPQ/AQA STEM - which support broader aims - multi modal assessment, real world learning contexts, interdisciplinary, problem based

•  Developing new accredited courses with examining bodies, Universities, professional associations etc
 

3. Defining and evidencing key dispositions for life and work. To build learner recognition, currency/value, teacher assessment literacy, and meet employer needs

Options:

•  Single disposition (4Cs individually)

•  Multi disposition (combined)

•  Working with one or more employers to look at developing strength based models
 

In schools (within subjects, interdisciplinary/cross curricular, extracurricular):

•  Creativity

•  Critical thinking

•  Collaboration

•  Oracy 

For recruitment (demand led): 

•  Employer led eg looking at games for developing key dispositions

•  Skills for employment with awarding bodies - diagnostic, value add of skills combinations

Our working groups have arrived at the following strategic priorities for 2021/22, in order to demonstrate the feasibility of alternative approaches to assessment:

•  Develop Interdisciplinary Projects for delivery within schools from January 2021

•  Set up pilots in alternative assessment through IPQ, HPQ and EPQ

•  Set up pilots in evidencing oracy using the Voice 21 framework - both within existing subjects and in its own right

•  Prototype assessments in creativity, collaboration and critical thinking

•  Develop a digital prototype of new learner profile reflecting the full range of a young person’s achievements - something akin to a modern version of the Record of Achievement

Developing a new ‘Learner Profile’

Our end goal is to develop a learner profile that recognises both academic and vocational learning, and the skills, knowledge and dispositions needed to thrive in a rapidly changing world.

The profile would incorporate valid, reliable feedback on the wide variety of skills and achievements of a young person, including academic achievements, subject-based and interdisciplinary, their key skills and dispositions, and evidence of their projects, products and passions beyond the traditionally academic.

The profile will start to be curated by young people in school but be usable throughout a persons life. It will capture what an individual can do - their strengths, achievements, qualifications, best work, testimonies - across a number of domains. It will be a more rigorous and equitable way of recognising the strengths of every young person – using multiple forms of assessment.

Ultimately we believe this is a fairer representation of people’s varied strengths, and will lead to schools and colleges putting more emphasis on those skills most needed to create employable, fulfilled young people. We aim to create a prototype of such a digital record in 2021/22.

With three levels of assessment:

•  Track eg activity logs, course completion, self-review, self-report, group critique

•  Evidence eg artefacts, photos, testimonials, exhibitions, performance tasks, investigations

•  Record eg rubric based judgements teachers/experts using a skills progression framework

Rethinking Assessment is closely connected with global trends... this is already happening elsewhere

•  Scotland - recommendation around a profile

•  Australia NSW and South Australian Government’s

Mastery Transcript