OPA0020

Written evidence submitted by York Consulting 

 

 

Introduction

 

  1. York Consulting (YCL) is the lead partner in the consortium[1] commissioned by the Department for Education (DfE) to undertake an evaluation of the impact of Opportunity Areas. In this submission, we present information on the work carried out in the first year on this study (2018-19).

 

 

Evaluation Aims

 

  1. The key aims of the evaluation are:

 

    1. Assess whether the increased focus, funding and accountability an Opportunity Area receives is associated with an improvement, or a likely future improvement, in educational and related outcomes, relative to similar areas which are not Opportunity Areas.

 

    1. Build the evidence base on social mobility interventions, examining what has worked within Opportunity Areas and identifying interventions which have the potential to be implemented in different areas or regions.

 

    1. Understand what works in creating and delivering a sustainable place-based approach to tackling educational issues at a local level. This includes both understanding whether a place-based approach has benefits in effecting systemic improvement and building the evidence base on what works in particular types of locations, e.g. coastal or geographically isolated locations.

 

  1.                           The key elements of the study are as follows:

 

    1. Assessment of overall impact: Construction of statistical models to capture the added value of the OA programme on educational and related outcomes. This will be achieved through a quasi-experimental design involving comparator areas and will seek to conduct analysis at three levels of aggregation: area, school and pupil using the National Pupil Database (NPD). This work is being undertaken together with the University of Leeds.

 

    1. Cost-Benefit Analysis (CBA): Design of a predictive CBA model to estimate the net benefit of OA funding. This will be consistent with the life-cycle model of social mobility and will explore benefits at three levels: fiscal, economic and social.  A macro analysis will be conducted using value-added data from our statistical model in Theme One. This will be complemented by a micro analysis of relative cost-effectiveness based on a customised CBA for each of the Intervention Level Evaluation projects. This work is being undertaken together with the New Economic Foundation (NEF) Consulting. This stage of work has also included an exploration of the extent of tilting of national programmes to Opportunity Areas.

 

    1. Place Based Evaluation: Involving close monitoring of developments in each of the 12 OAs through: initial attendance at Partnership Board meetings; development of progress indicator matrices and the development of logic models at the priority level for all 12 areas. This work is being undertaken together with the National Foundation for Educational Research (NFER). The focus is on what works in delivering a place-based approach and what types of interventions offer the greatest opportunities for expansion and sustainability. We have built relationships with staff in local area teams and conducted rich, qualitative research in each OA. The research fieldwork for this stage has involved two strands of investigation: a) planning and delivery; and b) cross-cutting themes:

 

      1. Planning and delivery. The first wave of the place-based impact evaluation explored the implementation of the place-based approach to OA activities and interventions. We investigated both the strategic and operational dimensions of progress in achieving the priorities and targets identified in Delivery Plans across all 12 OAs, and explored current plans for sustainability. During November 2018 YCL and NFER research teams carried out ten semi-structured telephone interviews, lasting approximately 45 minutes each, with partnership or subgroup board members and attendees from within each of the 12 OAs. Three additional interviews were carried out at the national level with representatives of: the DfE delivery central team for the programme, the Careers and Enterprise Company (CEC) and the National Citizen Service (NCS). The interviews were audio-recorded with interviewees’ permission. Overall, we conducted 123 interviews (some paired).

 

      1. Cross-cutting themes research. The primary objective of this research was to ascertain what works (and why) across different cross-cutting themes used to deliver a place-based approach: what outputs and outcomes do they achieve and what types of projects offer the greatest opportunities for scale-up and sustainability? We focused on the following thematic areas: behaviour and inclusion; mental health and wellbeing; parental engagement; teacher recruitment and retention; school-to-school support. YCL and NFER undertook the baseline research for this strand consisting of 15 semi-structured telephone interviews conducted in March 2019 (one interview per project) plus review of documentation supplied by projects.

 

    1. Intervention Level Evaluation: Review of the range of local interventions (4-5) in greater detail across the OAs to provide additional evaluation to contribute to building a stronger evidence base about their effectiveness, value-added impact, and scalability. These are in the process of being agreed.

 

 

Reporting

 

  1. This evidence will be reported on after the final wave of research covering:

 

    1. Statistical modelling

 

    1. CBA

 

    1. National programmes tilting research

 

    1. Place based evaluation

 

    1. Cross cutting themes (forthcoming).

 

 

May 2019

[1] The consortium comprises York Consulting Ltd (lead partner), NFER, NEF Consulting, and University of Leeds