International Development Committee
Oral evidence: DFID’s work on education: Leaving no one behind, HC 639
Monday 30 January 2017
Ordered by the House of Commons to be published on 30 January 2017.
Members present: Stephen Twigg (Chair); Fiona Bruce; Pauline Latham; Wendy Morton; Paul Scully.
Questions 111‑184
Witnesses
I: Dr Alison Evans, Chief Commissioner, Tina Fahm, Lead Commissioner, Korina Cox, Team Leader, Independent Commission for Aid Impact (ICAI).
II: Rt Hon. Lord Bates, Minister of State, Anna Wechsberg, Director, Policy and Global Partnerships, and Anna French, Head of Education and Policy Team, Department for International Development (DFID).
Examination of witnesses
Witnesses: Dr Alison Evans, Tina Fahm and Korina Cox.
Q111 Chair: Welcome, everyone, to this session on accessing, staying in, and succeeding in basic education—UK aid support to marginalised girls. This is slightly unusual, in that generally ICAI reports are dealt with in the ICAI Sub‑Committee, but because the Select Committee is in the midst of an inquiry into DFID’s work on education, it seemed sensible to take evidence both from ICAI and from the Department before the full Committee.
Welcome to our first panel. We are going to take evidence from ICAI for around 45 minutes and then from DFID for about an hour. We have seven questions that we are seeking to address with the ICAI panel. I will kick off. This is a critical review; it raises some serious concerns about DFID’s approach, including both its coherence and its effectiveness. Could one of you take this first question: is DFID failing to deliver for the most disadvantaged girls in the world? Tina, do you want to kick off and introduce yourself?
Tina Fahm: Thank you, Chair, and thank you for inviting us here today as part of your education inquiry. I am Tina Fahm, Lead Commissioner for the review, and I have prepared a short statement to introduce our review, which addresses the question that you raise.
The Committee will be aware that this is a major issue in terms of marginalised girls in education. We know that globally 60 million primary and lower-secondary-age girls are out of school, and we know that educating girls offers significant benefits in terms of health, education and socially for the girls, their families and communities.
Turning to DFID, for many years, DFID has committed to prioritising the education of girls, particularly the most marginalised, to achieve these benefits and as part of its wider programme to address gender inequality. However, our review concluded that UK aid is falling short of its ambition to educate the poorest and most disadvantaged girls. While DFID’s commitment to and significant investment in girls’ education is to be welcomed, there are significant risks that it is falling short of its ambition to help marginalised girls and deliver on the commitment to leave no one behind. Very briefly, we identified three major issues with DFID’s work. The first is the lack of a coherent and co‑ordinated approach to achieving its goals, as a result of which we found that its efforts are inconsistent in terms of depth and focus, and poorly joined up.
Secondly, we saw recurrent factors across the portfolio that resulted in a loss of focus on girls, particularly marginalised girls, during implementation. We found that a large number of programmes performed poorly against their original objectives. At times, programmes were displaced by competing priorities and, in some cases, targets for supporting girls were abandoned altogether. Thirdly, we found that DFID’s approach to value for money across its education portfolio did not adequately reflect the challenges of tackling marginalisation. Without this, we concluded that there is a risk for the drive of economy and efficiency rather than equity, which could create disincentives for DFID and its partners in focusing on marginalised girls.
In conclusion, Chair, we awarded our review an amber‑red score overall, signalling to DFID a requirement to improve its performance in its education programming for marginalised girls. What we expect to see is clear, strategic direction and a commitment to tackling the pattern of underperformance that our review uncovered.
Q112 Chair: Thank you very much. A number of the issues you raised in that summary will be explored by other Committee members as we go through. Let me stay on a process point to start with. We understand that DFID issued 150 factual corrections at the fact‑checking stage, a third of which you accepted. Two things strike me about that. First, there were 100 factual errors that you did not accept; could you give us an example of them? The second is that there were 50 that you did accept, so could you give the Committee an example of where you accepted that there was a factual error, because that does sound like a lot of factual errors in the original draft?
Tina Fahm: Thank you, Chair. I will briefly address the fact‑check process and then hand over to Korina Cox, Team Leader for the review. The fact-check process is a laid-down protocol as part of ICAI reviews. This review was conducted over nine months; we engaged with many different stakeholders across the globe, and in doing so, before going to print and presenting our findings, we offered DFID an opportunity to check through our findings, some of which were made some time ago. We aim to triangulate the evidence that we collect, and what we have noted is that during the lifetime of the review, following periodic meetings with DFID—all credit to them—they have started implementing some of our findings.
Q113 Chair: So some things that I am presenting as a factual error are more where events have moved on, and therefore something that was true when it was—
Tina Fahm: Yes, in some cases.
Q114 Chair: Okay. When you say “triangulate”, what does that mean? As in, you double-check it with something?
Tina Fahm: Yes. We go beyond that; we take evidence from different sources—from DFID itself, from stakeholders, from NGO partners—in order to have confidence in the evidence that we are reporting.
Q115 Chair: Alison, is that an unusual number, 150?
Dr Evans: It is not a particularly unusual number, but I would say there are differences of view often between ourselves and DFID about what is a fact and what is an interpretation of evidence. Quite a lot of the facts that we do not accept are because we stand by our interpretation of evidence, whereas DFID may have raised a particular query about how we interpret that evidence. In a few cases, particularly, as Tina has indicated, where the world has moved on during the life of the review, we look to accept the factual change on the grounds that a piece of guidance that we said did not exist may now exist. That does happen quite a lot during the lifetime of our reviews, so we are very happy then to make it clear that such guidance now exists, although we have not been able to judge, perhaps, the way in which that guidance has been implemented.
DFID reserves the right to present a lot of push back, very often, on the reviews, but the percentage of what you might call facts that we have fundamentally got wrong—ie a number, a date, something to do with the way in which targets have been specified—is relatively small.
Korina Cox: To give an example of something that is more of a factual nature rather than an interpretation, we had given a number around how many vouchers had been distributed for an education programme that was providing vouchers to girls in Pakistan. The number was disputed by DFID as being in the wrong order—in the tens of difference—but we were talking about the number of vouchers distributed; DFID was talking about the number of vouchers redeemed. That would be an example of where it may have appeared to be a factual error but in fact was not.
Another example is that there were some questions around the number that we provided for out-of-school girls, again in Pakistan, in the region that we were looking at. We were able to show that our data was, in fact, taken from UNESCO. There were a number of different sources, and that was the one that we decided was the most logical to use.
Q116 Chair: In that example, DFID disagreed with the UNESCO data and put to you an alternative figure, and then you looked at it and still upheld your original view that the UNESCO data was the most accurate.
Korina Cox: That is right. There were other situations where, for example, we had described a particular methodology that DFID had used to define its results targets and we had got the wording of that, which is important, slightly wrong. Obviously, we accepted that we needed to make sure that the methodology was described precisely how DFID preferred. Quite a few of those kinds of improvements to the report were made as well.
Q117 Fiona Bruce: Good afternoon, everyone. Could you tell us why you decided to conduct a performance review in this case, as opposed to an impact or learning review?
Tina Fahm: Yes. We thought that a performance review was appropriate because DFID has been operating in educational programming for some time and it enabled us to look at accountability, effectiveness and value for money. In our report, we give a timeline that goes back to 2005 in terms of expenditure on education, but in fact, DFID has been working in this space since 2000, so we had considerable data on which to review and look at accountability and value for money as well.
Q118 Fiona Bruce: Your predecessor commission in the last Parliament also looked at girls’ education. Why did you not draw more fully on what they had done—or if you did, could you tell us how you did draw on it?
Tina Fahm: The former ICAI report. We looked at DFID’s programmes within the period of 2011 to 2015, so we were quite focused on the review programme in question. We set that out in the report as well.
Q119 Fiona Bruce: Yet you say that your rationale for the decision to make this review was that education has been a major area of assistance for DFID for many years. Its commitment to promoting gender equality goes back to at least 2000, so it is an appropriate subject for a performance review.
Tina Fahm: It is. If we give some regard to the level of expenditure in this area in the period under review—£2.6 billion and £4.3 billion, in terms of the total budget for those programmes—we are talking about a considerable sum of money. In addition, education and marginalised girls fall under two global goals, and with DFID’s commitment to leave no one behind, we thought that this area was certainly ripe for a review, and that is why we undertook it. Finally, in terms of our own work plan, the “leave no one behind” theme is a cross‑cutting one, so it is important that in discharging our work we are mindful of how DFID is ensuring its accountability in this domain.
Q120 Fiona Bruce: Can I ask Dr Evans about drawing on previous ICAI reports and how far you use them?
Dr Evans: It is a very good point. We are always looking to build. We are always looking to draw on lessons that have already been put out into the public domain by ICAI and to see if we can take those forward. The particular conjuncture around this review has been to take what has been quite an interesting deep-dive look, under phase 1, at one aspect of education and to say, “Let us look at a thematic area for the Department, which is around leave no one behind, in particular trying to reach the hardest‑to‑reach girls, and take into account a new set of delivery channels that it is now operating”, including the Girls’ Education Challenge, which is, of course, relatively new and was not a major focus of the previous review. We wanted to take in, if you like, some new delivery channels and look at how marginalisation of girls is being treated across the entire education portfolio.
This is not a question of either/or. This is very much taking a slightly different set of optics on the question of reaching the hardest to reach in education. We are not displacing a previous review; we are trying to build on it and grow out from it.
Q121 Fiona Bruce: You are incorporating the priorities of an impact review or a learning review into this as well.
Dr Evans: In this case, as Tina said, some of the programming is quite historical, but there is a lot that is very much changing in the here and now. The new modalities around the Girls’ Education Challenge mean that this is much more suited, at this stage, to a performance review, because we do not have a wealth of independent impact evaluation evidence, particularly on GEC, for example. That is planned, but we do not have it yet to be able to go and verify some of those big results claims, which is what we try to do in our impact reviews.
Q122 Fiona Bruce: It is an interesting subject, and it is unfortunate that we do not have more time, but it brings back again the question of whether these different types of reviews really do work. That is a question I leave in the air, having asked my previous questions.
Let me move on to context. You chose just six countries for the country assessments: Lebanon, Pakistan, South Sudan, Tanzania, Kenya and Malawi. First, can you explain why Kenya and Tanzania, which are both in east Africa, were the only ones that were chosen for country visits and then, perhaps, why these case studies were chosen, and whether they really give a fully reflective overview of the context in which DFID is working in this area? Was the context broadly enough assessed?
Tina Fahm: In terms of our selection of country visits, we were presented by DFID with a list of 25 countries in which they were operating programmes, and we focused on 13. Our criterion was that the country needed to have at least one girl‑focused programme operating. We sought to get under the skin of DFID’s programming and to examine marginalisation and the experience of girls from different contexts. Of the six countries, four were in Africa, one in the Middle East and one in Asia. Four were low-income countries, and two middle-income—one just, and one a higher middle income. I will invite Korina to add to this reply, but we spent considerable time ensuring that we had a diversity of contexts and situations. If you think of marginalisation, there are many different facets to marginalisation; sometimes it is geography, which is the Pakistan example, the refugee situation with Lebanon, and in the African examples there were different levels of gender parity at different stages of the education system.
Q123 Fiona Bruce: I know the Chair needs to move on to other questions, so very quickly, why did you choose those two countries for the only visits you made—both in east Africa—if you are saying different contexts?
Korina Cox: We took a very structured approach to selecting originally, as Tina said, the six countries that came out of our selection matrix to be case studies in broad terms. However, when we were choosing the countries that would be justified for a visit, it was very important, because of the questions that we had around looking at the coherence of girls’ education programming, that we selected countries that maximised the opportunity for us to look at different modes of expenditure. Those two countries were the only ones that gave us the opportunity to look at country‑led programmes, the Girls’ Education Challenge, particularly in Kenya, which has a large number of GEC projects, as well as some smaller, centrally managed programmes that are relevant to girls’ education. Obviously, we saw that they are both in the same geographical region, but they provided the best opportunity for us to address the questions that we had.
Q124 Pauline Latham: Your report states that you looked across DFID’s portfolio as a whole. Aside from the six focus countries that we have discussed, there seems to have been no systematic analysis of where and to what extent DFID is allocating resources to marginalised girls’ education. Was this data collected and, if so, why was more information not included in the final report?
Tina Fahm: In terms of marginalised girls, we refer to the overall expenditure—the budget during the period under review, £2.6 billion, and £4.3 billion for the lifetime programmes—but what we found was that the data was not disaggregated into spend on marginalised girls, essentially.
Korina Cox: What we were able to do is a very high‑level analysis of expenditure across the different ways that we classified DFID programmes; there is girl‑focused and highly girl‑focused, including the GEC, as well as a very large number of general programmes that clearly do work with girls but do not have specific objectives around girls. We looked at the extent to which the pattern of that expenditure mapped on to wider UNESCO data on the level of need, in terms of the number of girls either being out of school or experiencing a lower degree of performance than boys.
There is a very short paragraph in the report that comments that from that analysis, we felt that there was, overall, a good fit between where DFID is spending resources on dedicated girls’ programmes and the level of need. There was not enough opportunity within the context of this review to delve into that in more detail, and there are caveats around the original countries that DFID prioritises for working in and so on, so there is not an exact match, but we were confident enough from that analysis to at least make the statement that there is, generally, a good fit between girl‑focused programming and the level of need.
Q125 Pauline Latham: The report does refer to the Global Partnership for Education and national education plans, but there seems to be no substantive analysis of DFID’s engagement with multilateral organisations, or even national Governments. To what extent did you assess these relationships, and why is there not more information in the report?
Korina Cox: The GPE, as a multilateral fund, was not the main focus of our report. The report and the ICAI review set out to examine the performance of bilateral funding.
Q126 Pauline Latham: Only bilateral funding?
Korina Cox: That is right, although we acknowledge that, in order to address the question of coherence, it was important to take into account the work that is done through the GPE. The way that we have done that is, in the countries we visited, we looked at the extent to which the education programmes that are implemented by DFID in‑country are co‑ordinated and, in strategic terms, coherent in the context of broader ambitions around marginalised girls. That is where we find some weaknesses in the co‑ordination between the country work and the GPE, probably largely stemming from the situation at the moment; the GPE is only just beginning to take on board more of a gender focus in its own work.
Q127 Paul Scully: The report concludes that DFID lacks a coherent approach and its efforts are inconsistent in its depth of focus and poorly joined up. Could you tell us how you came to that conclusion specifically, and what the major problems were that you identified that you have not already spoken about?
Tina Fahm: Yes. Korina has referred to programmes that we determined to be girl‑focused or highly girl‑focused, and programmes where there was no particular focus on girls at all. If we step back for a moment, we are looking at marginalised girls in education, so this is certainly about education, but also about women and girls, and DFID has a very strong track record and has made very strong commitments in this area. We know about the intersectionality around disadvantage and development, and some of these girls are facing significant ranges and different forms of disadvantage. DFID has the expertise in terms of women and girls, and in the past in its statements has said that the girls should be at the centre in programming. In looking at the different programmes, we found that there were not the linkages. In the highly girl-focused programmes, there was better performance there, but in the general education programmes, too often we found that the interventions focused on girls were falling off the edge. In terms of our programme, the Chair spoke about a fact check, but these were the sorts of issues that we were bringing to the fore during the course of the review.
Q128 Paul Scully: You talked about marginalised girls’ education, and you said in the report how DFID should have country‑specific strategies. In DFID’s response, it said it did not sign up to creating specific strategies, but instead says it will embed its new marginalisation framework across its education portfolio specific to each country. I am wondering where you are with that response.
Tina Fahm: It is interesting that the Girls’ Education Challenge, although there is a lot to be said for it, was a centrally managed programme, and we concluded that whilst it brought many benefits as a flagship programme, one of the weaknesses was that there were not the linkages within countries in which DFID was operating; it was very much a linear approach. Part of the business case for the Girls’ Education Challenge was the innovation and the learning. In thinking on the strategic approach, there has to be a commitment to ensuring that the learning is embedded and sustained.
Q129 Paul Scully: Talking about embedding it across its education portfolio specific to each country, does that sound like it is moving towards what you want or to an overall framework—to still having a wider framework that is not girl‑specific?
Tina Fahm: It is moving towards that direction, very much so. It is something that DFID has accepted, and I believe that the Girls’ Education Challenge is now gearing up for a further stage, and some of our recommendations are being taken forward in that regard.
Q130 Chair: Can I ask Dr Evans to comment more broadly on the issue that Paul has just highlighted? This is an issue that comes up a lot in terms of DFID’s central strategies and in‑country strategies and a lack of coherence. It does feel that we are, as a result, not getting the value for money that the taxpayer would expect.
Dr Evans: It has come up a lot recently, and we have noticed this. In a number of cases, what we are observing is some obviously very significant expenditure around some really important issues driven by some very ambitious outcome targets, and then a sense that there is something of a strategy vacuum about how you deliver that in ways that are compatible with context and the challenges one sees in different countries. It seems to us not to be the case that in order to tailor‑make spend around specific country challenges one needs to ditch strategy at a departmental level, which would, if you like, create a platform around which you are trying to ensure value for money in that spend. We are observing, in a whole number of areas, that there seems to be a tendency to say, “We have identified the outcomes we want, so we really do not need the strategy to tell us how to do that. We will just do that on a programme-by-programme, country-by-country basis.” Our sense, in this review and elsewhere, is that you need both. You need that strategic driver, and you need the commitment to work in context.
Q131 Wendy Morton: I just want to follow up on that. Can you give us a specific example of where the strategy vacuum is evident, maybe in a specific area or in one of the countries that you have been focusing on? Korina probably has a good handle on that.
Korina Cox: Kenya is an example where there are a number of Girls’ Education Challenge programmes, but also a number of country office‑led projects that have a very strong focus on marginalised girls in particular. We saw a research project, for example, that was working in the slum area of Nairobi and really generating strong learning about the sorts of things that attached girls to attending schools, and addressed some of the cultural norms that can stop them doing that.
Within Kenya, we saw lots of individual projects that were making real advances in putting the girl at the centre of education intervention, but in a way that was also bringing in other social and sometimes health interventions as well. However, what was got was a sense that, stepping back from that, there was no strategy that was allowing the learning from individual projects to influence the main-country, Government‑focused programme. There were lots of mechanisms for doing that—learning workshops, written materials—but, as yet, there was no particular strategy that would really decide what type of intervention was best suited to which location, and the different barriers that were being experienced in different parts of the country.
Q132 Fiona Bruce: We have talked about this before, and you have pointed out the lack of a clear strategic approach and clear strategic co‑ordination with regard to areas of DFID’s work. Within DFID, and I am talking about the officers, who should be responsible for ensuring that this is set? Which officer role is it?
Dr Evans: I would be very interested to hear DFID’s view on where it sees strategy now residing within the Department.
Fiona Bruce: We will be asking that.
Dr Evans: There is clearly a strategic function that has to exist at the level of the sector competency, so at the SCS lead around education. What is the role there for setting some kind of strategic course? However, it seems to me that needs to be informed even further up the chain, so it would sit with Director-General, who oversees policy division and so forth. This seems to be a bit of a feature now with the Department.
Fiona Bruce: Yes, it is.
Dr Evans: In our conversations, we get quite a lot of push back, which is, “We do not need that now, because we are all so results‑focused. It is now a question of what we are trying to deliver; we work out the ‘how’ in our country operational plans and our programming”. We are coming back to it time and time again. How then do you know? How do you know that the sum is greater than the parts? How do you know to draw on lessons across the piece and share what works well? We are a little confused as to why this is persisting.
Q133 Chair: In this report, you have made no specific recommendation about learning.
Tina Fahm: No. Of our three recommendations, the first is strategy, the second is around effectiveness and the third is value for money.
Q134 Chair: On learning lessons across the different programmes about what works and why, coming back to Fiona’s point about these different categories of review, it feels as though while you have given some good reasons for a performance review, we have lost the ability to learn—you, us, and most importantly, the Department.
Tina Fahm: I just want to say, briefly, before I bring in Korina, that certainly from the Girls’ Education Challenge and in our findings, DFID has taken opportunities to learn from the projects as they have been performing them, but certainly learning was not a feature of this particular review. It was not a learning review; it was a performance review. However, we certainly saw evidence of learning from innovation and research into what works.
Korina Cox: Briefly, within recommendation 2, which talks about the issue of programmes losing focus and our recommendation that effort should be made to focus on whether they are explicit or implicit objectives around supporting girls, often we saw that projects would migrate away from their original ambition and it took several conversations to understand the reasons for that. There is definitely a case for that to be better documented, so that it does provide that source of learning, both for the future of that programme and as a repository for future learning.
Dr Evans: Can I just add, very quickly, that it comes back to the point that learning needs a home? It needs to be able to refer back to something bigger than individual cases of what works in different contexts. The absence of a clear strategic vision is partly that it has no home, so the learning is happening, the investment is happening, but it is about where the buck stops, if you like, in terms of how that learning then shapes future action. That is the point.
Fiona Bruce: Yes, that is right.
Q135 Stephen Doughty: The report gives a number of examples of where DFID has underperformed against targets, but I would not say that it necessarily systematically backs up the statement you gave that the performance of DFID’s intervention on girls was generally poor. Can you just explain why you use such a strong statement when there is no systematic evidence provided in the report to back that up?
Tina Fahm: I would argue that we present very strong evidence on page 25 of our report, where we present cases in terms of performance where DFID is performing well and where it is not. I would invite the Committee to reflect on what this means for the girls. It is not just changing a business case slightly or not upgrading a log-frame so that the implementation does not happen. Potentially, this has very significant implications for the girls in question. On page 25, we present very well, and fairly, in terms of performance, where it is achieving its own objectives and where it is not, and this was a persistent theme in our review.
Q136 Stephen Doughty: Very helpfully, you have given me the page again. Clearly, there are examples and I acknowledge that, but on the claim that it is generally poor, there are some quite clear examples here of where it was exceeding performance in a number of areas on the sites, so that is quite a sweeping statement. That is the sort of thing that could be taken out of context, for example, by the media or others to characterise the whole programme as useless or worthless. Clearly that is not the case, because it is performing well in some areas. I accept the criticisms that are made, but do you think that is a helpful phrase to be using?
Tina Fahm: I stand by it. If you look at our scoring in terms of relevance, we recognise DFID’s commitment, DFID’s investment, the ambition and the vision that it addressed this portfolio with initially. However, in terms of the follow-through, we found that there was case after case where it was not succeeding, and this was significant.
Q137 Stephen Doughty: Okay. Can I bring you back to the Girls’ Education Challenge, which has been mentioned a number of times? Can you give us a frank assessment overall of whether you think it should continue, or similar schemes should be brought forward? Obviously, again, you are pointing to a number of successes and failures in it and have done in the evidence so far today, but what is your overall view of whether it has been a success?
Tina Fahm: My view of the Girls’ Education Challenge is to applaud DFID. It is the largest programme of its kind. It made the commitment, there was the vision, and there was the ambition. It was the first time girls’ education had been given such prominence, and it signals to others in the field that this is an important area of programming. I have already mentioned the fact that it was a centrally managed programme, and we have had some debate about that and its ability to influence other countries locally. However, our finding regarding the Girls’ Education Challenge was that it worked essentially through public schools directly without any other state actors intervening, so there is the wider issue of sustainability and, ultimately, for the UK taxpayer, value for money. In making that sort of investment, we need to ensure that vision on sustainability is retained throughout.
Q138 Stephen Doughty: So it is a good thing in and of itself, but there are questions about where it could go and what its wider catalytic effect was.
Tina Fahm: Yes, absolutely. This refers back to the question asked by the Chair about learning. We have evidence through our fact check and through our discourse with DFID that DFID has learned from phase 1 and, for this next phase, I am aware that some of the points we have raised are being taken forward.
Q139 Stephen Doughty: DFID says it feels that the recommendation on effectiveness was too vague. What do you want to see change? How can DFID do things better? Can you give me some practical steps about what it should be doing?
Tina Fahm: We received DFID’s management response last week—
Chair: We are going to ask them about that.
Tina Fahm: I am pleased to say that, of our three recommendations, the first, regarding strategy, was accepted. The other two—effectiveness and value for money—were partially so. I would say that for 90% of the recommendations, they are very minor points that DFID has challenged us on. I would argue that it has taken a slightly different view, but essentially the recommendations have been accepted.
Q140 Stephen Doughty: Specifically, in a nutshell, for the benefit of the Committee, what is it that you want DFID to do on the question of effectiveness?
Tina Fahm: It is follow-through: keeping an eye on the ball and not allowing the objectives around marginalised girls to fall off the board when convenient. It is harder to do; this is part of the tough stuff that DFID has really got to engage in.
Q141 Fiona Bruce: Your report is highly critical of DFID’s value-for-money approach to marginalised girls’ education. You talk about a lack of a clear strategic approach. Once again, I am referring to this in the value-for-money assessment that you have given. For example, it has not adapted its value-for-money framework to reflect its commitment to leave no one behind. I wonder if you could just clarify what you felt were the biggest flaws in DFID’s approach and what it needs to change to achieve better value for money. Obviously, an amber‑red assessment is really quite critical in an area that they have been working on for some years, including with a framework for value for money.
Tina Fahm: Yes, very much so. I must state that my background is in audit, so the three Es are emblazoned on my chest: economy, efficiency, and effectiveness in public services is very well known. Given the portfolio and given the focus of our review—looking at marginalisation of vulnerable individuals—we were really looking for, as well as the economy and efficiency, an equity lens, and this was missing. Absolutely, I agree with your point that, given the amount of time that DFID has been working on in this area, this is something it should have thought of long before now. It has accepted the recommendation, and I think it has adapted its tool, so we welcome that, but certainly it was a finding in our review.
Q142 Fiona Bruce: Would you agree that it is, again, another indication of work being piecemeal, rather than being co‑ordinated in the most efficient way to maximise value for money for the UK taxpayer, with inadequate learning right across the piece?
Tina Fahm: Absolutely.
Q143 Fiona Bruce: Sorry, Dr Evans, did you want to add something?
Dr Evans: No, I was wondering if Korina could give an example specifically on that.
Korina Cox: Yes. An example of, I suppose, the need to improve the cross‑portfolio learning is that through the GEC, again, there has been some really good evidence of good practice around collecting benchmark data as to how much reasonable costs might be for all sorts of types of interventions that support girls in particular. However, we found that the only way that that was being used more extensively across these much larger education programmes that also had objectives of supporting girls was through serendipity, rather than through any structured approach. Where education officers were particularly engaged on that, or where they had value for money expertise embedded in a country team, you would see examples of that being used. However, we did not see it really come into play so much in the business cases for the much larger expenditure that was for girls through the in‑country programmes. That is a really good example of where there are very good examples of good practice that are just not yet migrating across to the country programmes.
Q144 Fiona Bruce: As you say, DFID partially accepts your comments. What would you like to see it do, and what would you like to see it do more quickly? Is there a sufficient sense of urgency about this?
Tina Fahm: Absolutely, very much so. It is really to redesign its value‑for‑money tool, so that it is an appropriate measure of equity dimensions, given that the “leave no one behind” consideration is part of the global goals and it is something that DFID will have to consider in many other contexts.
Q145 Fiona Bruce: Would you say, Dr Evans, something that needs to be got hold of again at a very senior level in DFID?
Tina Fahm: Yes.
Dr Evans: We will be looking at its progress in this regard in the value‑for‑money review that we are now setting out on.
Chair: Thank you very much indeed. That is a very appropriate note to draw the panel to a close. Thank you for coming along. Feel free to stay for the second evidence session.
Examination of witnesses
Witnesses: Rt Hon. Lord Bates, Anna Wechsberg and Anna French.
Q146 Chair: Welcome, Minister. I think this is your first appearance before us in your new role. I explained earlier this is slightly unusual. ICAI’s reports are generally dealt with in our ICAI Sub‑Committee, but because we are in the middle of an inquiry into DFID’s work on education, it seemed appropriate to take this in the full Committee. However, I imagine we will be asking you to come before us at a later stage of our education inquiry to talk about the other aspects of the inquiry and its findings. I am going to ask Fiona, who chairs our ICAI Sub‑Committee, to lead the questioning.
Q147 Fiona Bruce: Thank you for coming, Minister, and thank you for coming, DFID representatives. I am going to address my first question, if I may, to Anna French, and it relates to responses to ICAI reports. DFID has agreed to respond to ICAI reports within three weeks, but the response to this review, which was only published on Thursday afternoon, just two working days before this meeting, was almost six weeks after publication. It is not an isolated situation, and we have been waiting over 10 weeks for the response to ICAI’s exit and transition report. In fact, the initial response seemed to be more of a press release and was very poor. I have to say that this one is slightly better, but I would like to have an explanation as to why it is taking so long, particularly when you have them earlier for fact‑checking. Who is responsible for responding, because this response that we had just a couple of days ago does not even have a name on it? Who is responsible for this? I am not asking the Minister; I am asking the DFID staff representatives, because at the end of the day the DFID Minister will rely on you to produce this for him. You may wish to reply, Minister, but I do not feel you should have to be put on the spot.
Lord Bates: I would respond in the sense of saying, of course, that was something that I signed off last week, because we were particularly mindful of previous criticism and we wanted to make sure that the report went out in time. Effectively, that was the response from the Department, and it was I who signed off on it.
Q148 Fiona Bruce: You would have signed it off, but you are reliant on your officials to provide you with this information and put it in front of you. How does this work, and who is responsible in DFID?
Anna Wechsberg: Thank you for the question. Exactly as you say, DFID officials do have sight of the report at fact-checking stage, and while there is some change between the fact-check stage and the final report, those changes, hopefully, are not significant. We are then able quite quickly to draft a response to the report itself. At that point, it will need to be considered by Ministers. It will also need to be looked at by DFID’s central team, who look after the ICAI relationship. I think part of what has happened is that this has got caught up in a discussion with the Committee, ourselves and ICAI on the length and format of responses, and that has been the cause of some of the delays that you have seen. With this one, as the Minister said, we have tried to get it to you in time.
Q149 Fiona Bruce: I am sorry, but as Chair of the Sub‑Committee, I am going to press this point, because I am not satisfied. Who is it in DFID who puts that document in front of the Minister? Who is responsible for ensuring that this Committee and ICAI are treated with the dignity they deserve, that we are given appropriate time to consider DFID’s response, and that a substantive response is given? As I say, we are still waiting for one on the exit and transition review.
Anna Wechsberg: I cannot speak on the process for the exit and transition review. Each of these goes from the particular team through the relevant director—in this case me, for the education one—and then through the Permanent Secretary to Ministers for sign-off.
Q150 Fiona Bruce: Thank you. I think I do have an answer there: it is the Permanent Secretary.
Anna Wechsberg: Yes.
Q151 Fiona Bruce: Can we have an assurance—and perhaps we need to write to the Permanent Secretary—that in future we will get responses within the agreed timeframe?
Anna Wechsberg: I will take that away.
Q152 Pauline Latham: Obviously, you were here listening to the previous session. Marginalised girls’ education is very important for DFID to tackle and it has been for some time. It is a priority. Why does DFID not yet have a coherent, joined-up approach across the Department for the different strands of in‑country and centrally managed programming?
Lord Bates: The approach that we are taking is, first, recognising that as progress towards the MDGs and the sustainable development goals gathers pace on education, a lot of the, if you like, easy-to-reach children, in terms of getting them into education, have been reached. There is, therefore, a growing group, estimated at around 61 million, of children who are not in education and they are marginalised in the barriers that they face in getting into education.
We also, of course, know that education is the best empowerer of economic development that it is possible to have, and that is exceptionally the case, as the Education Commission found, when it comes to girls. Therefore, if your target is economic development and elimination of extreme poverty, then it would follow that you would place a very high emphasis on getting rid of the barriers to education, particularly for young girls. That was the thinking behind the Girls’ Education Challenge. It was something that would be tough to do, ambitious, some might say courageous to embark upon, but nonetheless not taking the easy option but going for something, which was a very difficult cohort of children to reach, and looking for innovation.
In that respect, and this is something that the ICAI report came out with, it recognises that, as a result of that, DFID and the UK Government have taken a leadership or a prominent position in terms of advancing girls’ education around the world and generated a huge amount of learning, which is then shared with other partners to increase outcome. There was a very clear rationale for doing it, and I would call that a strategy, because you have a clear rationale for doing it and you have a clear target. However, we are all aware that when we are talking about degrees of marginalisation, cultural barriers, working predominantly in conflict‑affected zones, in rural areas, with people with disabilities—these are all very difficult to do, but nonetheless, in our view, worth doing.
Q153 Pauline Latham: That is fine, but during the review, DFID Kenya highlighted to ICAI the difficulty country officers faced in co‑ordinating across multiple programmes with different objectives and time periods. How are you going to address this to ensure a much more joined-up approach?
Lord Bates: This is very much recommendation 1, which came out of the ICAI report, and we very much agree with that. We need to recognise that we have to do better in that particular area. We have looked at the UK country business plans that are now being prepared and we are going to ensure better co‑ordination between the Girls’ Education Challenge activities in Kenya, for example, and the bilateral programmes that we have in that country as well. We are going to look at strengthening the in‑country co‑ordination, whether that is physical personnel within country, as it is in some cases, or better supervision through our education advisers.
Q154 Pauline Latham: That is something that I wanted to go on to. The in‑country education advisers are usually best placed to pull the different strands of DFID’s education work together, but how many education advisers does DFID have across the portfolio now, and is there one in each of the priority countries?
Lord Bates: There are currently 41, and that is an increase from about 35, so it is not a huge increase. We are talking about looking at the Girls’ Education Challenge being in operation in some 18 countries. We have bilateral programmes in 22 countries. With 41, we feel that we do have at least a presence of one to two per country, if you wanted to divide it that way, and they do provide a great deal of input for us. We had the education world conference here last week, and a lot of our education advisers were back for that and it was a great opportunity to exchange with them how we do improve our co‑ordination in the light of this report. We are absolutely mindful of the fact that we need to do better, because one of the objectives, of course, is to generate learning outcomes, and the learning outcomes, together with the programmes that are operating with the Government agencies within the country, must be a prime source of that. We want to increase learning, so we have to increase co‑ordination. We are mindful of that and we will do that.
Q155 Pauline Latham: Were you mindful of that before the report came out?
Lord Bates: Officials are nodding.
Q156 Pauline Latham: In which case, why have you not done anything about it?
Anna Wechsberg: It is an issue not just in education but more broadly, where we have a mixture of centrally managed programmes and country‑specific programmes. It is something the Committee has looked at before and it genuinely is a challenge to make those join up better. We have taken a number of steps across the Department. A key one for us is that our management information now allows anybody across the Department to see which programmes, including centrally managed ones, are operating in each country. That means that Pete Vowles, for example, who has been a real champion of this join-up in Kenya, is able to have a list of everything that is operating there and understand and ask questions. Then, as the Minister said and I think you mentioned yourself, the education advisers are also a key join-up.
On education, it is important to acknowledge that the institutional structures—I do not just mean DFID’s but around the sector—are complicated. You have the Global Partnership for Education, there are obviously country Governments doing their own things, and you have private provision in some places. One of the ways in which we need to tackle that is to use our education cadre to join up, so we not only have education advisers in our country offices but we also have taken deliberate decisions to second some of our advisory experts into GPE and into Education Cannot Wait, so that we can try to make those links better. However, it is absolutely a work in progress.
Anna French: The Global Partnership for Education also has its education sector plan. It is supporting Governments to develop their education sector plans, so co‑ordinating behind that plan is what we have decided to put most of our investment behind. We have the Girls’ Education Challenge, which is doing some of that innovation and testing, and we are just beginning to get that mid-line evidence coming out to really inform the direction of our bilateral programmes. We are also using that to inform what the Global Partnership is doing through those education sector plans and what Government are, therefore, doing as well, so that coherence is coming through the Local Education Group and in co‑ordination also with other donors.
Q157 Wendy Morton: I just wanted to probe a little bit more on the advisers. You said that you recognised that there was a challenge. You have increased the number of advisers from 35 to 41. I have made a note here that you are going to use the educational cadre for joining up work and thinking and that some advisers have been seconded to GPE and others. Is that part of the 41, or is it other staff who are being seconded? If you are taking part of the 41, are you, therefore, increasing capacity or is that additional capacity being spread more widely? Do you get where I am coming from?
Lord Bates: I do get where you are coming from. The 41 is the absolute number and, therefore, when we have seconded to other organisations then that is it, clearly. In a sense, the balance is it is also part of a mission, because part of the mission is to try to share the learning of what works and what does not work from the programmes. We are now building up that body evidence, which we share at events, like hosting the Girls’ Education Forum on 7 July last year, and bringing people together and sharing. The Secretary of State shares that through the high‑level group on the Education Cannot Wait group and sharing information through that way. We have to get this information out. It is pointless it sitting there on databases if people cannot access it. There is a huge amount there. I was just reading through one baseline report produced by Coffey on the Girls’ Education Challenge, which was published in January last year. It runs to some 600 pages and is publicly available online. That type of information contains a lot of really good, granular information about what works and what does not work, and we need to share that as widely as possible, both in‑country and at these cross‑cutting bodies like the Global Partnership for Education.
Q158 Paul Scully: The ICAI report looks at the Girls’ Education Challenge and the pros and cons of it being a stand‑alone fund. Why did DFID decide to make it a separate, centrally managed fund rather than incorporate the work and the additional funding into the country education programmes?
Lord Bates: When it originally happened, there was a document, which is our position paper on education from July 2013. However, prior to that, there had been a feeling that in the choices that were being made—and I am sure the Select Committee has looked into this as well—too often the options that people were going for were looking for headline numbers that need to be achieved in terms of getting children into school. In going for the easy to reach, they were taking the safe options and there was a lot of criticism.
Although I was not at the Department, I had been interested in particularly this aspect of international development, and I am conscious that there was a lot of criticism and people were taking safe options to deliver outcomes, to deliver numbers that look good, and they were missing out on some of the harder to reach, but ones who would, if you could get them into education, yield multiples. The Education Commission talked about that 70p invested in the most marginalised perhaps yielding as much as £10 in economic development. There were huge returns, and therefore the feeling was that there was a place for a programme that went out to try to do these innovative, risky, marginalised things that are particularly focused on girls, because, of course, again, that was part of our strategy. We wanted women and girls to be at the heart of everything that we do, so therefore it was natural to realise that, because girls are amongst the most marginalised groups, you would go in their direction.
I am sure that that was the thinking behind it, but I am also very conscious that we have people who were around at the beginning—I was not—who could perhaps give you a better answer.
Anna Wechsberg: I was not in this place at the beginning but, as it happens, I was a little involved with the Girls’ Education Challenge when it was established, and Anna can also add to that. I think that is exactly right.
The other point to add, which comes through in the ICAI report, is that this is not an area that is easy, and it is not an area where we know the two or three things that work brilliantly. What we wanted to do was to create an opportunity to gather lots of ideas, to try different approaches in different contexts and see what worked; by having that organised in a single central programme, we could make that learning work across the piece and gather that information more efficiently and effectively. In fact, that is one of the points that ICAI have made in their report about the value of having a central programme. It was a choice, and those were the reasons behind the choice at the time.
Q159 Paul Scully: One of the cons among the pros and cons that they talked about, to quote the ICAI report, is that, as a centrally managed fund, it “may have made it more difficult to develop a coherent approach to marginalised girls’ education in each of its partner countries.” To what extent do you think the contractors that are delivering the challenge projects work with national Governments and DFID country offices?
Anna French: To take an example, Leonard Cheshire are working in Kenya and they are supporting the Government to develop their inclusive education strategy. They have taken some of the learning they have on how to support the most marginalised disabled girls, and they are now supporting the Government. We also have a lot of the evidence coming in from the mid-line findings, which is some of the most important evidence that we have globally in terms of what it actually takes to support those most marginalised girls. We are beginning to share that around with partners, and we are bringing in further investors because they see that what we are doing is actually working. We are partnering up with others such as USAID and some of the private sector, and that is also now contributing to the sustainability. We have this multi-pronged approach that is really beginning to support the Governments in their approach to girls’ education.
Q160 Paul Scully: When programmes are not embedded within the national systems, how are you working to ensure the sustainability of the projects?
Lord Bates: On the sustainability, this probably speaks to one of the areas that you raise and that we recognise, which is a gap in the current programme. Some of these are innovative, in terms of the programmes designed to increase learning and to share experiences that can then be shared across Government programmes more widely. Therefore, the question is whether they are meant to be long-term stand-alone programmes, or whether they are actually meant to prove a particular theory of how you overcome a barrier to education and to access to education; when proven, the case is made. That is part of the reason why, when we set off, there were 41 programmes and now the continuation of funding has gone to 35. We will continue to bear down on that and question these. If it is innovative, naturally some of them are not going to be working.
We accept that we need to do better in joining the dots within countries. That point has been well made by ICAI in their report, and it has been accepted by us, and we now need to make sure that we prove how that is happening going forward. We have suggested that the two ways in which we would like to show that are, first, through strengthening and co‑ordinating the role of education advisers and, secondly, through strengthening the strategic role of the UK aid country business plan as a means of actually achieving that.
Q161 Wendy Morton: The report accuses DFID of a lack of mechanisms to ensure that learning on what works is used to inform future programming. This is a very serious criticism, particularly in relation to the innovative pilot programming in the Girls’ Education Challenge. What is your explanation for this?
Lord Bates: That falls into the category of one of the points that we partially accept; we do not fully accept that. That is quite a strong level of criticism, and we are not quite with them there. We believe that we have generated a lot of information on what works and what does not work. We accept that a lot of the barriers that we face are, at a basic level, about geographic access, when you look at projects that are in remote areas where there is not a school building that is easily accessible to children.
I met with the Education Minister from the Khyber Pakhtunkhwa province in Pakistan last week, and he was talking about the challenges that he has in actually making sure that there are enough chairs in the schools and making sure that there are boundary walls around the schools, particularly for girls, which is very important culturally to make sure that they are safe. There are also challenges in making sure that there is running water and electricity there.
When you come to areas of disability, we know how important access is, and we know how important it is to provide simple things like spectacles, hearing aids and correct seating, and the difference that that can make to a child. We know, particularly in relation to girls, that mentoring works. We know that having more female teachers works. We know that having toilet facilities and sanitary facilities and making sure that they work is very important. We know that having extra‑curricular activities works.
These are a fraction of the number of things that we are generating. Would we have known them if we had not had these innovative programmes through the Girls’ Education Challenge? We may have done, and we may have stumbled across some of them through serendipity, but this has been a systematic approach to try something new, and it is yielding, I would argue, a lot more information that would be very important in actually designing the education programmes of the future.
Q162 Wendy Morton: It is really helpful that you are giving us examples of where learning is happening, but I would like to take you back to the issue of new mechanisms. In your response to the report, you state that DFID will develop a number of new mechanisms to encourage the sharing of knowledge, so it is not just the learning but the sharing of it. I wondered if you could help the Committee by giving us an idea of when these will be established. How do you anticipate them working in practice when DFID staff and delivery partners are so often already overstretched in their normal day-to-day work?
Lord Bates: That is a good question. The first one is through that sharing, via the education advisers. There are other mechanisms that we have such as Building Evidence into Education, which is a cross‑cutting group with UNHCR, UNICEF and other bodies such as the World Bank sharing information on what works.
More and more data is being published online. I mentioned the new mid‑line data that is beginning to come through, which will be published and shared. We have a particularly exciting project called RISE—Research on Improving Systems of Education—which is about improving systems in education and which is being undertaken by the Blavatnik School of Government at Oxford. That is a long-term plan, collecting evidence in a systematic way and then sharing it.
I agree that it will be absolutely critical to focus on how you are going to get the information out there and how you are going to share it. They are some of the mechanisms that we are already looking at.
Q163 Wendy Morton: ICAI highlighted that there is no overall theory of change or detailed analysis on how to get marginalised girls into education, and there is no distinction between girls as a whole and those who face particular disadvantage. I wondered if you could explain how you plan to develop the theory behind DFID’s programming in this particular area.
Lord Bates: I can see that. One of the areas that I think would help—and I invite colleagues to come in on this—is that we have the education quality assurance unit now, which will particularly look at identifying how things can be improved going forward in the context of the value for money. Are there other mechanisms that you can think of?
Anna Wechsberg: Anna, do you want to comment on the theory of change point? That is quite important.
Anna French: We have developed a new theory of change. As the Minister has pointed out, we are beginning to get all the different lessons in terms of what the barriers are to girls’ education. That has been coming through the mid-line, and we have been using that to inform the business case of the second phase of the GEC. We have developed a much more nuanced theory of change in terms of what it will take to get those really marginalised girls and the not-quite-so-marginalised girls into schools. Different factors will inform that, and that is what has taken forward the development of the new programme.
Anna Wechsberg: The final thing to add on that is that one of the things that comes through the country-by-country examples in the ICAI report is how country-specific a lot of this is. There will be some value but it will not capture everything if we try to do a theory of change at a sectoral level in a generic way. It will get us so far but it is not going to help in those very specific contextual situations, so we just need to be a bit careful.
Q164 Wendy Morton: I have one final question. On that specific point, you are collecting the data, we have talked about sectoral change and you have clearly acknowledged that this is not a case of one-size-fits-all. Building on from that, will there also be a country‑specific element to it that addresses some of those different cultural and even geographical issues within one specific country?
Anna Wechsberg: That is what we intend to do. In accepting the first recommendation, that is exactly the spirit in which we want to take that forward.
Q165 Chair: How long do you think that will take?
Anna Wechsberg: I am reluctant to commit here to a firm timeline, but I would have thought that it would be during the course of this year.
Q166 Chair: ICAI highlight that you “cannot achieve transformative impact” unless you “also influence national education policies and systems”. They say that your lack of influence on that has hampered effectiveness and relevance in this area. Do you accept that and, if you do, why do you think DFID has had limited success in influencing national education policies?
Lord Bates: One of the difficulties of looking at this is that there are many, many examples that you could use across the 36 programmes that are currently running where you would say that they absolutely are working with Government. Anna gave one example of Leonard Cheshire in Kenya, in disability, which is working very much with Government, and therefore the learning is very much there. The same is true with many of the programmes in Pakistan, Malawi or the RISE programme in Tanzania. There are examples where you could make that point but, similarly, when you have a large number of different programmes seeking to test out different barriers to education for marginalised girls, then of course you will find that some of them will be working within the Government’s sphere and having a good influence and some of them will not.
Q167 Chair: That is a very helpful answer. Do you accept that there are countries where you have not been as effective as you might have been and, if so, what is the plan to change and improve that?
Lord Bates: That reverts back to recommendation 1. Yes, the learning outcomes are one thing to, if you like, undertake the literature review and the research exercise, but ultimately the objective is having lasting recommendations and theories of change that will result in better outcomes. That is only going to be achieved by working with Governments, and that means working with our partners within countries. It is not just DFID that is working in these areas; other countries have programmes within countries. Having a better country co‑ordinated plan is a very important part of that. The natural nature of the thing of having so many diverse programmes—some of which are more suited to working with Government, and some of which are more stand-alone, such as the ones on technology, where you are trying to test a theory as to how it can all be used to overcome barriers—means that there may be one that does not influence as much, whereas others will. We need to make sure that the lessons learned are integrated into the national strategies.
Q168 Chair: Can I move on to the broader judgment that ICAI has made about DFID’s effectiveness in this area? The review says that DFID’s effectiveness in getting marginalised girls into school is, I quote, “at best mixed and at times poor”. How do you respond to that?
Lord Bates: I accept it is mixed. I would accept that. We make no defence for that. I keep coming back—and do forgive me if I make this point again—to the fact that we are talking about the most marginalised people in South Sudan, Somalia and Khyber Pakhtunkhwa. People are facing incredible barriers. These are not the easy opportunities that we are looking for. We really are challenging the boundaries, targeting girls aged 11, 12 and 13, because we know that that is a huge barrier in terms of getting them to make the transition from primary school education into secondary education. However, if you can get them to make that transition, it delays marriage and it increases their opportunities and life chances immensely.
I would accept that it is mixed, but it is mixed because DFID is being bold and ambitious in tackling a problem that everyone in the education world knows exists, but other countries, to be quite frank, have not had the guts to actually put their money where their mouth is and actually go out and challenge it. That is what we have done, and the fact that we have to put up with a bit of flak for the ones that do not quite work out as we planned is a price worth paying for challenging this particularly needy group of people who are in danger of being left behind.
Q169 Chair: You mentioned other countries, and this is often something that comes up in all sorts of contexts, such as education and humanitarian response: that the UK often does rise to challenges that some other wealthy countries do not rise to. Is there any sense in this specific area that that might be changing, and that women’s education and girls’ education might be moving up the agenda of some other donors?
Lord Bates: I like to think so. There is a particular graphic that I found helpful that I am happy to share with the Committee afterwards on what others are doing in this area. What it points to is that, if you want a very high success rate, you should offer scholarships. Give them a scholarship to a great university. We are doing that as well. In-country as well, you get them into a safe zone and you improve their learning. Where do they go after that? All you are measuring is: have you actually given education and have you invested in them?
Q170 Chair: If you do not mind me interrupting, Sir, we had evidence from Gordon Brown on Thursday, who made exactly the same point. He made exactly the point that some other countries have seen their education aid increase by providing more scholarships to their own universities.
Lord Bates: Exactly. When you actually look at this particular example of the ones who are particularly investing in secondary education—that crucial second phase of education that makes all the difference, particularly for girls—then we are streets ahead of everyone else. There is no one close. Some do very well. The USA does very well in primary, but the UK does well in primary, does very well in secondary and then has a small amount of post-secondary or scholarship funding. I will share that with you, because I think it is something that, to be quite frank, ought to make us proud of what our aid budget is actually doing.
Chair: Thank you. I gave you that opportunity to give a positive advertisement for what the UK does.
Q171 Wendy Morton: On that point, if we are talking about education and sustainable development goals, and even though we are behind agenda, is there a role for DFID to play, as DFID so often does, in getting other donors to step up to the plate? Is there a role for DFID to play in encouraging others to go beyond what they are currently doing and reach to some of these more hard-to-reach areas and these marginalised girls?
Lord Bates: Absolutely. You have put your finger on it. That is absolutely what we should be about. This is not just a DFID issue; it is a global issue. If it is going to be solved, we all have to be part of it. We have all signed up to the sustainable development goals, which include those ambitions that we are all very familiar with, so we have to share this information. It is encouraging, as has been mentioned, that, in places like Afghanistan where we are showing that our approach is working, USAID have actually come in there and partnered with us. We have an objective of actually getting more matched funding in, and that is a good imperative because we have to prove that it works, and then we actually have to get the partners coming in with us; we are committed to doing that.
Q172 Chair: I have allowed us to do it—I think I even prompted it—but we have slightly gone outside of the remit of the meeting in talking about some of the broader issues. Can I take us back to some of the points that ICAI has made? ICAI found that, throughout the lifetime of education programmes, particularly the country programmes, “the focus on girls often gets lost”. Do you accept that and, if you do, how do you plan to address it?
Lord Bates: This was another one that fell under the category of partial acceptance. I am very happy for either of the Annas to come in on the detail of this point. It seemed that one of the issues is that it is not a contradiction to say that, in order to have a strategy for girls, you actually need to have a strategy for boys. That is no contradiction.
Chair: I absolutely agree.
Lord Bates: Part of the entrenched conservative attitudes towards education and girls’ roles and violence against women and girls and their safety and security comes through having policies and programmes that actually focus in on boys’ education as well. You then say, “I absolutely buy that”. Therefore let us try to actually have a strategy for boys in these circumstances. You then have a programme that is addressing boys but it is possible that someone can come along and say, “Hang on; I thought this was supposed to be about girls.” There is therefore a bit of an argument that you are “losing focus”, which I think was a particular phrase in there.
However, from the ones that I have looked at, I do not really get that. It is part of the strategy just as, in the same way, we know that a huge difference to access to education for girls comes through having the support of the community and the support of family where the community is actually saying, “We realise that there is a value in educating girls”, and that is very important to them in terms of outcomes. Therefore, just as we might have a programme that focuses on improving community actions and family attitudes towards girls’ education, we might have a programme that focuses on boys in actually challenging their views of girls’ education. That is not a loss of focus.
Anna Wechsberg: Just very briefly, it is one of those areas where the commissioners were talking earlier about the nature of the fact-check where the ICAI report classified some programmes as girl-focused. We never intended them to be girl-focused. They are about supporting national education strategies, and you take what you get in terms of the children who are in schools. Perhaps that has led to a slight colouring of the report that we would not entirely accept. It is fair to say that there have been cases where, in implementation, things have not gone brilliantly. For example, the Malawi project has fallen behind schedule. That has genuinely fallen behind schedule and we are trying to do something about it. The project with Avanti that they picked up as well is something that has genuinely not worked out as we had intended. However, we are on to it and we are trying to fix the problem.
It would be fair to accuse us of setting ourselves ambitious and possibly optimistic objectives in some of these but, where things have gone wrong, we have tried to fix them.
Q173 Paul Scully: I will come back to Avanti in a minute, if I may, but ICAI talked about the fact that the Girls’ Education Challenge failed to meet three outcome milestones: attendance, learning and sustainability. Notwithstanding what you have said, why do you think this is and how are you going to move that forward into the next phase?
Anna Wechsberg: Is it alright if we ask Anna to comment on that? The annual review that we carried out in December, which was after the ICAI report had concluded, has better data that actually shows a more positive story.
Anna French: It does. At the time when ICAI were doing their review, we only had partial information on the mid-line findings in terms of learning. However, the final mid-line data has now come in, and it has shown that we are over‑achieving on the learning target that we had set ourselves. That just shows that we disagreed because of the timing, and the ICAI team fairly pointed out that some things came through after their report was concluded.
Q174 Paul Scully: Could I look at the Avanti project? Avanti is a private sector provider of satellite data communications. How did a company with little experience and knowledge of education end up running a £13.8 million project?
Lord Bates: This fits very much into the heading of the technology aspect of education. We believe that technology has a potentially strong role to play in delivering education in some of the more remote parts of the world and we wanted to test that out. Clearly, education technology is something, apart from anything else, that the UK is a world leader in, and it is something that we want to see incorporated more and more into the way in which we actually approach education around the world. Are there any specifics on Avanti?
Anna French: I do not have anything particular to add to that apart from the fact that we have partnered them up with Math‑Whizz, which is a very good education technology deliverer as well. What we have tried to do with some of the different GEC projects is ensure that they are working with the experts, and we have given them very specific support if we see them falling behind, in terms of the education and learning outcomes, to ensure that we are getting the best value for money for the taxpayer.
Q175 Paul Scully: ICAI talked about the fact that one of the failings, as they saw it, in the project was the lack of education expertise amongst staff. You obviously do not seem to agree with that, because of the link up with Math-Whizz.
Lord Bates: Again, this comes back to the fact that, when you innovate, you are sometimes innovating with new partners that have not been in this field before in order to achieve your outcomes. The thinking behind Avanti was very much in that sphere on education technology. I am very happy, Chairman, to write a little bit further on Avanti.
Chair: Please; that would be helpful.
Lord Bates: If there are any other specific ones that you would like to drill down on a bit more as to the rationale for the selection of the partner, then I will ensure that you receive that in a timely manner.
Anna Wechsberg: Just to be clear, our own annual review of the programme also confirmed that finding and that is why we brought in Math‑Whizz. That was a response to exactly what you identified, which is a lack of expertise.
Q176 Paul Scully: ICAI also talked about the lack of expertise, as a general concern, in relation to DFID’s work on marginalised girls’ education. What weight do you actually put on strong experience in education? How much do you take that into account, notwithstanding what you were saying about technology advances, when you are considering bids? Where do you weight education experience and the other innovation that you were talking about?
Lord Bates: For each business plan, there will be a weighting in the evaluation, but do we have it specifically here in terms of education experience?
Anna French: We look at the results and impact. We would have to investigate further, as I do not have the answer here. We can write to the Committee in terms of what expertise they have. We work with some of the experts in the field, such as Camfed and Save the Children, so those tend to be where they get the most funding. As the Minister said, we have been testing with some other smaller, newer partners in this field with whom we have not worked before, and, if they are not working, we have stopped those programmes.
Anna Wechsberg: The Girls’ Education Challenge was deliberately structured to have a step‑change window that worked with more traditional partners like Unicef and Save the Children, an innovation window to do what it says on the tin, and these partnerships with private sector companies, including Coca-Cola and Avanti, where we were deliberately working with people with whom we would not normally work in the sphere. It was quite consciously structured in that way, but the majority of the money went into the step‑change window with more traditional partners to reach as many marginalised girls as we could.
Q177 Fiona Bruce: ICAI’s report was very critical of DFID’s value-for-money framework, in fact categorising it as amber-red, which is defined as “unsatisfactory achievement in most areas”. In DFID’s response, you partially accept this. Which areas of ICAI’s criticism do you not accept? Perhaps Ms French could assist us with this. Which areas do you not accept?
Lord Bates: We recognise that there are a number that we are changing. Some of the criticism is that we have lost focus on girls, which we did not accept for the reasons that we talked about before. They questioned some of the educational credentials of our partners, which, again, we did not accept for the reasons mentioned earlier. We did not accept the ones that related to in-country strategy and sharing, because we felt that they had been superseded by changes that we were already beginning to make in the way that we went around it. They are some of the ones off the top of my head. There is one that we will probably come on to, which is the whole area of value for money and the importance of equity. We partially accept that.
The three Es of the National Audit Office requirement on value for money put the emphasis on equity. We totally get that. There was a report that showed that 46% of education spend goes to the 10% most educated. If your mission is to eradicate extreme poverty and, through a particular programme, to reach the hardest to reach in the hardest-to-reach countries, then clearly equity has to be a major part of that. We believe that it will be as we move forward.
Anna French: This really explicit focus on equity is something that we were really glad to see in the ICAI report. We currently have that in our value-for-money guidance. It says that, “DFID’s approach to VfM does not mean we should take the easy options, ignoring difficult to reach populations or problems which are difficult to tackle.” We do not have an explicit equity focus in our current VfM guidelines for education, but I am working with my team to ensure that that is now incorporated into the updated guidelines that we are developing and that are due in a month or two.
Q178 Fiona Bruce: You are right. ACAI says that, “DFID does not routinely require this”—that is giving due weight to equity—“for education programmes”. What concerns me is that you have acknowledged, therefore, that, while there appear to be some considerations in guidance, it has not been implemented in practice. This has been a flagship programme over many years, lauded by the previous Secretary of State. You have been running education programmes now, certainly if you look at the report on page 15, from 2011 onwards at a cost of at least £1 billion for girl-focused country programmes. I am struggling to understand why you have not addressed this much earlier than 2017, when the NAO report guidance was in 2010.
Anna French: We have for the Girls’ Education Challenge. They took that equity aspect as part of the DFID weighting of what we judge to be a good project or not. They have looked at that. We are ensuring that that learning from the Girls’ Education Challenge is rolled out across all of our education programmes.
Q179 Fiona Bruce: It is interesting; you talk again in your response about the Girls’ Education Challenge exemplifying good practice as to how to approach VfM analysis. However, the report says on page 34 that the Girls’ Education Challenge may in due course provide DFID with some of the information it needs to begin applying value-for-money analysis to its “leave no‑one behind” commitment. It seems to me, from what you and ICAI are saying, that there is a disparity, and that there needs to be much more attention to value for money, probably driven from the top, as we heard earlier, to ensure that VfM really is a clear strategic focus. You are nodding, Ms French; do you agree?
Anna French: We have, from the Girls’ Education Challenge, got this very clear detail in terms of what works, what does not and the cost per girl. We have looked at the cost-per-girl metric and that does inform what we are doing in terms of development. It is something that we can look at as we are going forward.
Q180 Fiona Bruce: You offered, Minister, to write to the Committee. I wonder whether you could, after two months, when you say that you are looking at this again, perhaps write further and give us a further response to this VfM aspect of the report. Your response that you have given within the last few days does not seem to satisfactorily tally with the comments that ICAI are making in their report.
Lord Bates: It is a very good point to make and we will certainly write to you on that in follow-up. On the general point of value for money, if you wanted to get a good ride in terms of scrutiny, then the easiest thing in the world to do would be of course just to pass it all to the Global Partnership for Education and say, “There is a big cheque” and operate in all the countries around the world and achieve the results. However, the consistent theme all the way through is that we are trying to do something difficult and very challenging that has never been done before. We are breaking new ground and we are innovating. Constructing an algorithm that correctly gives us an impression of value for money has perhaps been quite difficult, and perhaps part of the reason for that is that we have been focusing on the NAO model of effectiveness, efficiency, etc, but missing out on that crucial element of equity. The recommendation of equity, which we will adopt going forward, will help us—if we can find a way of quantifying what is, in many ways, a qualitative judgment—in factoring in a more recognisable value for money of the whole programme.
Q181 Chair: In response to Fiona’s question, might we hear something on that in two months’ time?
Lord Bates: Yes.
Q182 Chair: I have three questions that I want to ask you and we have six minutes left, so I will fire away. The first is on payment by results. On the basis of the evidence, do you intend to continue with that approach in phase 2?
Lord Bates: This was part of the education position paper in June 2013, which set out this idea of trying to introduce payment by results. We have a table that I am happy to share with you about the cost per child in education that shows a wide disparity. We recognise that performance by results in a lot of, for example, fragile states just simply does not work as a metric. Where it is there, it is often a relatively small sum.
Chair: That was not the answer that I was expecting, so thank you. I look forward to receiving that table.
Lord Bates: Basically, performance by results overall is a very important objective. In this particular focus of marginalised girls, we just have some questions as to how it could be applied.
Q183 Chair: ICAI tell us in their review that “smaller civil society partners are unable to take on the financial risks associated with payment by results”. This is an issue that this Committee has raised in a range of contexts, not just in payment by results. Is that part of why you are expressing the scepticism that I was surprised to hear?
Lord Bates: It is part of the reason. With small trials it is often statistically more difficult to generate the data that is able to then inform what the payment by results should be. That is part of the answer but I do not know if Anna wants to add to that.
Anna Wechsberg: Time is short and you have covered it, Minister.
Q184 Chair: Lovely. I will just finish with a very straightforward issue, which is that of private schools, and particularly the low-cost private schools. ICAI again states, “DFID lacks evidence on the effectiveness and value for money of working through private schools”, and highlights the concerns of many stakeholders. You spoke, Minister, about the importance of equity. Does DFID have an evidence base to show that working through private schools provides value for money, and will you therefore continue to work through low-cost private schools in phase 2 of the Girls’ Education Challenge?
Lord Bates: Our position on this is that we are not ideological. In this country we actually have a mix of private and public, and therefore we do not think that we ought to prescribe that there should be only one means of delivering in other countries. Additionally, as you will realise, the only people delivering in many countries are actually low-fee state schools. Lagos is an often-quoted example, with 16,000 or 18,000 small private schools and only 1,600 state schools. Therefore, people do not have a choice. Where we have found situations like that, we have been prepared to pay scholarships and fees because fees can often be a barrier to access. We recognise that because of these hard-to-reach areas, state provision is often not universal. We long for the day when it is universal and people therefore have the ability to have access to that, but while it is not, we work with whoever is there delivering a decent education to children.
Chair: That is certainly an issue that we will return to as part of our broader inquiry, but at this point I would like to say thank you to the three of you for your evidence here. There are clearly a number of areas of concern that we have raised in the evidence session today. Minister, you have kindly agreed to write to us on a number of those issues and our offices will liaise on them. Can I just at this point thank you for giving us evidence here today and the Committee members for their time?
Lord Bates: Thank you, Chair. We very much welcome the partnership in working with yourselves and also ICAI in informing how we should move forward on this. I would be particularly interested to hear and read feedback from your upcoming visits into West Africa, which would be very helpful to us.
Chair: Thank you very much indeed.