OPA0019

Written evidence submitted by the National Foundation for Educational Research 

Introduction

1. The National Foundation for Educational Research (NFER) is a partner in the consortium[1] commissioned by the Department for Education (DfE) to undertake an evaluation of the impact of Opportunity Areas. In this submission, we present information on the work carried out in the first year on the place-based evaluation: progress and delivery research and the cross-cutting themes research.

2. The purpose of the progress and delivery research is to identify what works in planning and delivering a place-based approach providing insights into both strategic and operational dimensions of progress in achieving priorities and targets set out in delivery plans across the 12 Opportunity Areas. In contrast, the purpose of the cross-cutting themes research is to gather in-depth evidence and undertake a comparative analysis of the progress achieved by different types of provision focused on the same theme delivered in several Opportunity Areas.

3. The place-based evaluation is a continuation of the evaluation of the implementation of Opportunity Areas[2] undertaken by NFER for the DfE in 2017-2018. This was a process evaluation of OAs’ set up and implementation activity in the first year of the OA programme.

Place-based evaluation

Objectives and research questions

4. The primary objective of the place-based evaluation strand is to: understand what works in creating and delivering a sustainable place-based approach to tackling educational issues at a local level. This includes both understanding whether a place-based approach has benefits in effecting systematic improvement and building the evidence base on what works in particular types of locations e.g. coastal or geographically isolated locations.

5. The research questions which the place-based evaluation strand is addressing are as follows:

1. What are the conditions for a successful delivery of a place-based approach and what factors act as barriers to delivery?

2. How far can a place-based approach create the conditions for improvements in educational performance and social mobility at a local level?

3. Are there location-specific factors and approaches that have influenced delivery and outcomes in Opportunity Areas? Do national and local delivery and governance structures work as intended and enable successful delivery?

4. To what extent has the OA programme created sustainable change in organisational cultures and structures which can ensure activities are sustainable beyond the life of the programme funding?  How far has this had an impact on the experiences of learners and their families?

6. Here it should be noted that context (‘place’) underpins the rationale for the OA programme and is central to identifying what works and why: what are the distinctive features of each area that account for why activity is working or not working?

Methodology

7. The first wave of the place-based impact evaluation explored the implementation of the place-based approach to OA activities and interventions. We investigated both the strategic and operational dimensions of progress in achieving the priorities and targets identified in Delivery Plans across all 12 OAs, and explored current plans for sustainability.

8. During November 2018 NFER and York Consulting (YCL) research teams carried out ten semi-structured telephone interviews, lasting approximately 45 minutes each, with partnership or subgroup board members and attendees from within each of the 12 OAs. Three additional interviews were carried out at the national level with representatives of: the DfE delivery central team for the programme, the Careers and Enterprise Company (CEC) and the National Citizen Service (NCS). The interviews were audio-recorded with interviewees’ permission. Overall, we conducted 123 interviews (some paired).

9. Interviewees included:

1. All DfE OA heads of delivery, most area leads and all programme managers

2. An OA partnership board chair in each of the 12 OAs

3. Representatives from the early years, primary and secondary schools, colleges/post-16 providers, universities, the voluntary and community sector, local authorities (LAs), local enterprise partnerships (LEPs), employers and the skills sector, along with other independent individuals involved in the programme.

10. Prior to data collection, the researcher team worked with DfE heads of delivery and area leads to identify interviewees outside of the DfE team. The people interviewed were those most closely involved in the OA programme, either as partnership board and/or as subgroup members, and were selected to cover a range of sectors.

11. The topics covered in the interviews were:

1. Progress to date including priority areas making most and least progress and factors facilitating or barriers impeding progress.

2. Exploring OAs’ journey to impact including perceptions of progress on this journey to date, their expected outcomes by 2020 and whether these are realistic and achievable.

3. The role of OAs in improving capacity to date and by 2020 and their involvement in national programmes such as the Teaching and Leadership Innovation Fund, Strategic School Improvement Fund, Careers and Enterprise Company, National Citizen Service and Research Schools.

4. OAs’ approach and provision through local infrastructure and projects for sustaining any positive changes and impact in the future.

5. The role of local stakeholders including any changes to their engagement with OAs and their influence on commissioning decisions.

12. NFER carried out a systematic and detailed analysis of the data at OA level as well as by theme. We presented the early data from the first wave of research at an interactive workshop with DfE staff on 5th February 2019 which helped us to further interrogate and interpret the findings. Following this, the research team will feedback on the interim Wave 1 findings and combine these with future waves of research to produce a report including findings, conclusions and recommendations for the DfE.

Cross-cutting themes research

Objectives and research questions

13. The primary objective of this research is to ascertain what works (and why) across different cross-cutting themes used to deliver a place-based approach: what outputs and outcomes do they achieve and what types of projects offer the greatest opportunities for scale-up and sustainability?

14. The questions guiding the research are:

1. How did this project come about; why was it commissioned?

2. What are the targets or interim outputs that the project is aiming to achieve?

3. Did any of the projects offer/provision already exist

4. What has gone well in terms of project delivery so far?

5. What, if any, challenges has the project faced to date?

6. What, if any, risks have been identified for the remainder of the project?

7. How are you monitoring and evaluating the project?

8. What have you learnt so far about your delivery model and what works in supporting this theme in the target group, e.g. young people, parents, you are working with?

9. What has the role of local stakeholders been in planning, recruitment/take-up and delivery of the project?

10. Is stakeholder engagement at the level needed for your project to achieve its objectives?

Methodology

15. The themes of investigation were identified through an assessment of Opportunity Area delivery plans and progress indicator priority actions conducted by YCL and DfE.

16. The five cross-cutting themes are:

1. Behaviour and inclusion

2. Mental health and wellbeing

3. Parental engagement

4. Teacher recruitment and retention

5. School-to-school support.

17. NFER and YCL undertook the baseline research for this strand:

1. 15 semi-structured telephone interviews conducted in March 2019 (one interview per project)

2. projects could nominate who best to represent them in the interview – in some cases we conducted group interviews

3. documentation supplied by projects was reviewed (e.g. to identify project aims, outcomes and budget).

18. We carried out a systematic and detailed analysis of the first wave of data and will present these interim findings to DfE shortly.

19. The final outputs from the place-based evaluation (progress and delivery research and cross-cutting themes research) will be reported on after the final wave of research.

May 2019