International Development Sub‑Committee on the Work of the Independent Commission for Aid Impact
Oral evidence: ICAI’s Review on Violence Against Women and Girls, HC 101
Wednesday 13 July 2016
Ordered by the House of Commons to be published on Wednesday 13 July 2016
Members present: Fiona Bruce (Chair); Dr Lisa Cameron; Mr Nigel Evans; Jeremy Lefroy; Stephen Twigg.
Questions 1-58
Witnesses
I: Dr Alison Evans, Chief Commissioner, Tina Fahm, Lead Commissioner, and Peter Grant, Team Leader, Independent Commission for Aid Impact.
II: Nick Dyer, Director General for Policy and Global Programmes, Gerard Howe, Head of the Inclusive Societies Department, and Professor Charlotte Watts, Chief Scientific Adviser, Department for International Development.
Witnesses: Dr Alison Evans, Tina Fahm and Peter Grant.
Q1 Chair: Good morning. It is good to see you again. Thank you for coming here today. This is our second meeting and we are here to discuss the ICAI report on DFID’s efforts to eliminate violence against women and girls. This is a learning review. Alison, could you tell us why you chose a learning review?
Dr Evans: Certainly. Good morning, everyone. As you know, the learning review is a new instrument that we have put in place in the second phase of ICAI’s work. It is very much designed to tackle a number of issues when DFID or other Government departments, which we will hopefully be looking at in the future, are working in an area where the evidence base is very incomplete and relatively untested—where there is a need to put practice ahead of an understanding of what works. In this area, although DFID has been active for a number of years, it is still a rapidly emerging field of great complexity in which, frankly across the whole sector, the evidence base is incredibly thin and evolving.
In the context of a learning review, we want to test the extent to which DFID is positioning itself to deliver on its promise in this area, by looking very closely at the way it is investing in and utilising evidence to inform its work, but also how it is beginning to programme its commitments to deliver potential impact. The fact that it is a learning review is our recognition that we are still very much working in the realms of potential impact, rather than actual, because we are frankly still not far enough down the programming cycle to be able to assess results on the ground consistently and comprehensively.
Q2 Chair: Thank you. It is about the transition from learning into credible programming, to use your words. If I can quote from the report and your recommendations at the end, recommendation 2 on learning, you say, “DFID is not maximising learning from its own portfolio, particularly about how to mainstream VAWG.” Elsewhere in your report, on page 19, you say, “DFID is not as focused on learning from its own VAWG portfolio” as its other activities. “There is limited evidence that DFID is systematically capturing lessons from its own VAWG programmes and sharing them across countries.” “There is no overarching evaluation strategy for the VAWG portfolio.” “A stronger role for the central VAWG team in the planning of evaluations and the dissemination of results would enhance learning.” Why did you give it a green overall score?
Dr Evans: I am going to let Tina answer that, as Lead Commissioner on this particular review, and maybe Peter can come in too.
Tina Fahm: Thank you very much. Yes, you are absolutely right. As Alison has said, this is the start of a relatively new area of programming and a lot has happened in a relatively short space of time. We are looking at from 2012 to 2015. The number of programmes increased from 64 to 127 and, in the same period, spend increased from £20 million to £184 million, so a huge increase in a relatively short space of time.
In terms of the scoring, I would like to invite you to look at the report on page 40, if I may, where we explain our scoring in a little bit more detail. You will see from that that in two areas DFID scored green: learning and influence. In the area of relevance, however, we gave a green/amber, so overall we concluded that green was appropriate in this case.
The green score is very much based on the evidence we found, and that is that DFID has made a really impressive start in a very challenging area. Our report clearly shows what DFID is doing well, which is exceptional in many cases—world‑class—and where there are areas for further development. In conclusion, currently these programmes are small in scale. To achieve the impact that Alison mentioned, DFID needs to scale up. It has made an excellent start in a very challenging area, but now it is for DFID to take it to the next level.
Q3 Chair: Thank you very much. I appreciate what you are saying, although four years is still four years. It is not as though it started last year. Your explanation of what green means is: “Strong achievement across the board. Stands out as an area of good practice where UK aid is making a significant positive contribution.” You have just said that there are areas where DFID could do more. Your green/amber reference is: “Satisfactory achievement in most areas, partial achievement in others. An area where UK aid is making a positive contribution, but could do more.”
Tina Fahm: I acknowledge that differentiation but, from our perspective, our findings are based on the evidence relating to our review questions. As ICAI, we need to ensure impartiality and fairness so that, where we find excellent practice, we report it as such, and where there are areas for improvement, we do likewise. Within the context of our findings on VAWG, in many cases DFID was leading the field globally. Therefore, making these changes was really beyond green/amber to excellence in this field of dealing with violence against women and girls, within the context of social norm change.
Q4 Chair: We do not doubt that DFID is leading on this and there are many positive aspects of DFID’s work, but your report is intended to be evidence‑based. Just as a final comment, and then I am going to pass to my colleague Stephen Twigg for the next question, you do say in your report, “DFID has not put in place systematic and comprehensive learning procedures.” That is quite a strong statement, isn’t it?
Tina Fahm: Absolutely, Chair. What I would like to do is perhaps introduce Peter Grant, Team Leader for the review, to give some more insight in that.
Peter Grant: To expand on Tina’s point about where the evidence points, it is quite interesting to break down what elements we might expect to see in DFID expanding a policy commitment into those programmes. Some of those where we feel DFID has performed extremely well would be in terms of leadership and priority regarding policy pronouncements; building up central capacity, where we saw evidence that they put in place a dedicated team to address this and also set up a helpdesk to support the countries that are bringing it in; and funding research programmes. As Alison said very clearly, this is an area where knowledge is not well developed.
DFID has been very proactive in terms of commissioning leading researchers to explore what those gaps are and then to fill them in the major What Works programme that we highlight, but also specific research programmes on female genital mutilation and cutting, and on child marriage and a longitudinal study of girls over their lifetimes. We then expect them to be investing in some global programmes. They have supported major UN programmes on child marriage and on female genital mutilation and cutting, giving a major impetus to that. We tested that out with those involved in it. We saw that they produced high‑quality country programmes, albeit at a very small scale, and that was one issue. We also saw on the influencing side, which I am sure we will come on to, the Girl Summit and the Global Goals work. Across that range, there was some extremely strong performance.
We also highlighted the stage the work is at. That is key to your learning point. You are saying that there are not yet that many evaluations coming from the process. One of the things we noted is that, since DFID has decentralised its evaluation strategy and work, it is much harder for the centre to give this direction around a key theme. That is an issue that could be explored here, and that is why we recommend a stronger central role there.
We see this challenge as being perhaps more in the future. A key dimension of this green is saying that it is very good so far but there are a lot of challenges ahead, and that certainly includes marshalling those evaluations that are going to be coming on stream over the next two to five years particularly, and utilising the information coming from those.
Q5 Stephen Twigg: Can I pursue further the point about impact? I understand what has been said in terms of your approach not looking systematically at impact at this stage, because we are not far enough down the programme and it is too small in scale. I understand that you did look at DFID’s own reporting of the impact it was having. Can you tell us if you tested the evidence that DFID used and, if so, how you went about doing that?
Tina Fahm: As a learning review, we did not look directly at impact but at the propensity for programmes to have impact. The evidence that we gathered was based on a very robust methodology, if I can just introduce that.
You will see that we started with a literature review. We then looked in depth at DFID’s strategy, in terms of VAWG commitments. We undertook a review of DFID’s VAWG programmes and looked at 23, as part of a series of desk reviews. We followed through and undertook country visits to Ethiopia and India to assess VAWG programming in practice, and we also considered DFID’s international influencing work, which is absolutely vital to scaling up this work.
I must say that, in terms of the methodology and our approach, it was peer‑reviewed, so we are confident that the approach we adopted was robust. It was really testing the evidence across the piece.
Q6 Stephen Twigg: When you mention a literature review, is that looking specifically at DFID’s own literature or is it also looking at independent research into the area?
Tina Fahm: It is much wider. It is looking at all independent research in the area. During the course of the review, we consulted widely with stakeholders and undertook around 70 interviews, the bulk of which were with DFID. There were 24 interviews with DFID, seven with other Government departments, five with donor agencies also working in this field and 15 with leading academic researchers, in terms of gathering the evidence.
Q7 Stephen Twigg: It sounds to me that, although earlier you said that the programme is quite young and small‑scale, you did rather a lot of research in terms of looking at impact. Can you give us a sense of what you think the impact of the work is, even though it is at a relatively early stage?
Tina Fahm: Yes. I will start and then perhaps bring in Peter to talk a little bit more about our findings with regard to the review. What I set out is very much ICAI’s approach, so that would be similar regardless of whatever review we are conducting. It is a thorough approach to the gathering of evidence on which we base our findings and our reporting. We are, however, still very much at an early stage of this particular portfolio, given where we were in 2012.
In terms of the ability to review the impact with a results focus, these are early days. We are looking at a problem with which it is difficult to engage: violence against women and girls. In judging where DFID is now and looking at the work it has done, but recognising that there is a lot of work still to do, we have been very impressed with what we have seen in terms of impact.
Peter Grant: The evidence is not yet there to make judgments on impact, but it is something we could expect to see coming up over the next two to five years, as more of these evaluations come through.
There are a couple of things to flag in this area. One was that, in the report, we distinguish between this idea of VAWG‑focused programmes, which are looking specifically to address the issue, and VAWG‑component programmes, which are looking at much wider programmes where there are VAWG elements. We had some data on beneficiary numbers within the VAWG‑focused programmes. One of the interesting things there was to see how small they are relatively. We are talking about beneficiaries in the tens of thousands. However good those programmes are, unless they are either scaled up or brought into these much bigger programmes, the impact is going to be at that kind of level. For us that immediately raised the question of going to scale, which becomes the key theme of the report.
The other thing to flag is that the assessment of impact is a complex question in this area, not least because of the different types of violence against women and girls that have been identified and how different those are, between intimate partner violence, sexual violence, FGM/C and so on. There are very different approaches. The body of evidence that will need to be built up to be able to reach decisive conclusions in any of those areas will be significant in itself.
Q8 Stephen Twigg: Thank you. The approach paper states that strengthening and expanding services would not be the focus for this piece of work, because it was partially covered in a previous ICAI review. The final review says that you did not assess support services for survivors of violence against women and girls in detail. Can you tell us a little bit more about what you included, what you excluded and what grounds there were for excluding some things and including others?
Tina Fahm: Our review, as outlined in the approach paper, essentially focused on social norms. We were looking at norm change and DFID’s efforts in this regard. I should state that the House of Lords published a report in April, I believe, following a review of violence within the context of conflict. We were very clear that this review would not include violence against women and girls within a humanitarian or conflict scenario. We were clear from the outset; that was part of our research.
We were also mindful that a review undertaken by this Committee in 2013, I believe, suggested to DFID that it needed to do more work around social norm change, which is the reason why we wanted that to be central to the review that we undertook.
Dr Evans: That adds a particular dimension to why we regard this as a learning review and appropriate for a learning review. A lot of the programming we were looking at has come into play in the last couple of years, not back in 2012, because it is looking at social norm change. That is a much more recent feature of the DFID portfolio than the stock of programmes that was available for the IDC to look at in 2013. We were keeping up with what is a moving frontier inside the department.
Q9 Jeremy Lefroy: I apologise for being late. Your report has several serious criticisms of DFID and I would like to take a couple of them, in particular the lack of a clear strategy for scaling up. What would you see as a strategy for scaling up? What would you understand that to be?
Tina Fahm: Fundamental to scale‑up, which is absolutely vital to this piece, given the scale of the problem—this is a global problem intractable in nature—is the very good work that DFID is currently doing. There is consideration of increasing the size of those programmes. Currently, what they are doing is small‑scale. There are excellent results emerging from those programmes, but to have real transformational impact it is absolutely vital that DFID is able to deliver these programmes at scale. We did not see any credible approach to inform us as to how DFID plans to do this.
Q10 Jeremy Lefroy: Would you think it is DFID’s responsibility to deliver them at scale? Would you not see this as DFID doing some excellent pioneering work, when it is then up to the Governments of the countries involved or perhaps other organisations, perhaps multilaterals, to take up? DFID is showing the way but then saying, “Look, this is not just something for us; this is something for everybody.”
Tina Fahm: I would like to answer that and then hand over. I agree with the point that you have made to a degree. In explaining our methodology and approach, I joined the visit to India and there I saw first‑hand how local government was engaging with the programme. There was that real sense of looking forward and sustainability.
With that said, in some areas it has been about shining a light on very difficult challenges faced by women and girls. An example would be the Girl Summit in 2014, which DFID very much led and which has inspired follow‑up in both Ethiopia and India. There are examples of where we found good work that has then been taken forward and followed up.
In some cases, particularly with a view to the question of value for money, yes, there has been a fantastic start, but in terms of really manifesting and maximising value for money, the programmes that we saw need to be increased to scale.
Peter Grant: Part of any DFID strategy to see scale‑up would have to include a strategy for addressing the multilaterals and raising the priority of this issue in that context. What we are saying is that DFID has created a huge opportunity here. In its work with other governments, in its work with multilaterals, it can take that forward.
There is also an agenda for its own programming, coming back to this question of VAWG‑component programmes that we talked about earlier. For me, the challenge would be to say where DFID injects the VAWG perspective into its water programmes and into its health programmes. How systematic is it about that? It has produced good guidance on that, but to take it forward would require a much more intentional approach. It would also include things such as staff training. We saw evidence, for example, that a lot of the expertise resides with social development advisers, who are clearly able to advise on this. It also needs to be water engineers and health advisers who have that understanding to be able to see the ways in which violence against women elements can be mainstreamed into these much bigger programmes, which would then reach millions of people, rather than tens of thousands.
Q11 Jeremy Lefroy: I very much understand that. If I may, Chair, it relates to something that both you and I saw last week in Kananga. There was what we consider to be a pretty good DFID programme with the police in the Democratic Republic of Congo, which sought to break down the enmity between the population and the police, and would therefore I think have a direct impact on violence against women and girls. But that programme had, for understandable reasons, been stopped because of issues and problems with the police in a completely different part of DRC.
It occurred to both of us, I think, that the consequences of stopping such a programme in an area where great progress has been made in the relations between the police and the community—and obviously women and violence against women are very much part of that—had possibly not been considered. What you are getting at, if I may—and maybe I am wrong—is that when decisions like these are made, the aspect of violence against women and girls needs absolutely to be considered in terms of the potential impact of a decision like that on that work.
Tina Fahm: In our report, we address the importance of the intersectionality across the aid platform to ensure that DFID’s strategic approach looks at how the programmes impact on violence against women and girls, be it through WASH programmes, VAWG‑focused programmes or wider.
I must say that, certainly in the programmes that we visited in-country, there was real effort being made to work with government and with local community organisations to ensure the sustainability of the efforts and the sustainability of the investment that DFID has made.
Q12 Jeremy Lefroy: Just finally—I think this is something we could talk about for a very long time—you gave DFID a green rating overall, yet these are very substantial points, perhaps criticisms, that you have made. Why did you arrive at that overall assessment, given what you see as some clear shortfalls?
Tina Fahm: We explained how we arrived at the rating earlier. In defence of the rating, if I may, we have judged DFID’s progress based on the evidence over a relatively short period of time. We are at the start of the portfolio. There are really impressive findings emerging, but we have been consistent in saying that to make a real impact and to transform the lives of women and girls who are everyday victims of violence, DFID really has to take these programmes to scale. What we have found is good and defensible, in light of the score we have given.
Q13 Dr Cameron: I have a couple of questions on methodology, before my main question. Have you undertaken a meta‑analysis of your own programmes? Was that conducted after the literature review? I was not quite sure; did the literature review form the basis of your programmes?
Tina Fahm: Yes, the literature review was first. There are four main components: a literature review, then a strategic review of DFID’s VAWG policy to understand where they are going and what they want to achieve. We then undertook 23 desk reviews of DFID’s main VAWG programmes. This amounted to approximately a third of the overall programme. We followed that through with visits to countries and then we looked at DFID’s wider international influencing piece.
Q14 Dr Cameron: In terms of the meta‑analysis that you conducted, are you measuring the same things in each of the programmes or are the programmes looking at different things?
Tina Fahm: No, we are measuring the same thing. Essentially, that approach—what we measure and what we focus on—was set out in our approach paper, which we touch on briefly in the report. We have review questions that are absolutely fundamental across the review.
Q15 Dr Cameron: You said that you conducted interviews. Is that correct?
Tina Fahm: Yes, a number of interviews.
Q16 Dr Cameron: Did you conduct interviews with service users?
Tina Fahm: Absolutely. Given this portfolio, there is huge sensitivity with regard to service users, as you would imagine. It was really accessing that information and engagement through intermediaries. I must say again that, certainly from the visit I undertook to India, there were service users as part of some of the programmes we went to visit who were able to speak for themselves as to the impact they believed engagement with the programme had had for them.
Q17 Dr Cameron: How many service users did you interview?
Tina Fahm: We interviewed a number of service users. As I said, of the 70 interviews, 24 were with DFID, 15 were with donor organisations that were also working in the same space and 15 were with academics. The service user interviews would not have been direct; they would have been through those intermediary organisations. Throughout the review, we have had a very open and robust stakeholder engagement plan with representatives of NGOs and organisations and, post publication of the review, we have continued with the engagement to discuss our findings and recommendations.
Q18 Dr Cameron: Given that this is a hidden problem in many areas, I am wondering how you are able to assess the scale of the issue in the countries that you are working in. In terms of the data collection, how will you assess the progress?
Tina Fahm: The scale is huge. Our report points to some of the headline figures. One in three women globally is affected by intimate partner violence.
Q19 Dr Cameron: Is that an underestimate?
Tina Fahm: I do not think so. I think that those figures that we have estimated are accurate estimates of the size of the problem. This is a huge problem, which is why our message on scale‑up is so key to how this agenda goes forward.
Q20 Dr Cameron: It is obvious that, in the UK, the figures we have through research are an underestimate, because many people do not report this. The scale‑up is obviously important, but I am wondering how you get reliability of the data.
Dr Evans: That is a key point. Tina is right to say that, based on the best possible available data, those figures presented in the report are solid, but this is an area of massive under‑reporting. It would be wrong of us to suggest that there is not a hidden challenge as well. DFID is but one player. Of course, not only is it vital that DFID engages proactively with others to wrap their hands around this problem but, as you say, it is also a major problem for how we get representative data.
In many ways, this review was asking to what extent DFID is an investor in some of that data because that, for us, is a key part of its role: not only to programme but also to be an investor in data for the wider public good. Clearly it is making real strides in that area and is a very proactive contributor to discussions about getting better data in this field, which seems to us to be a really important part of its function. However, it would be completely wrong not to acknowledge that we are dealing with huge data gaps and massive under‑reporting.
Q21 Dr Cameron: There are a number of areas where you are unable to reach clear conclusions, including on DFID’s contribution to international processes and whether DFID is covering the full range of violence against women and girls. Why were you not able to reach clear conclusions on those areas?
Tina Fahm: As we have stated, this is very much an emerging area of programming for DFID. We are aware of the role DFID has played on the global stage, in terms of the What Works agenda, and indeed the efforts DFID has made in getting women and girls recognised within the Global Goals. Clearly DFID is making and has made an impact there but, again, we felt that there was more that DFID could do in the influencing role and in supporting the influencing role of other multilateral agencies.
Q22 Dr Cameron: Can you give us some tentative impressions as to how you are doing in moving forward in these areas?
Tina Fahm: Do you mean how DFID is moving forward?
Dr Cameron: Yes.
Tina Fahm: That is an area that we have referred to in our recommendations, so it is really for DFID to be able to explain. We have had a response to our review, but we would like greater clarity, as I think this Committee is aware, of exactly what DFID is going to do within that wider influencing arena.
Q23 Dr Cameron: Given the major limitations to your methodology appear to relate to limitations in DFID’s monitoring and evidence‑gathering, does this show fundamental flaws in how DFID is learning from its portfolio?
Peter Grant: We did find some gaps. In terms of the programmes, there were a number of desk reviews we looked at where they did not specify which type of VAWG they were addressing and therefore we were not able to classify it on that basis. In terms of the international influencing, you can look at that in two very different ways. One is to say what the objectives were and what was achieved. When you look at it through that lens, we can see DFID set out certain goals for both the Girl Summit and the Global Goals, in terms of a standalone gender goal and a VAWG target. These were definitely achieved and these were significant achievements.
But if you ask, underpinning that, if there was a strategy to achieve that, the answer that we saw is not explicit. Also, is there a way of capturing the real‑time influencing work that is done that would allow you to make those causal connections between the objectives and the outcome to build the strength and robustness of the conclusion that DFID made a significant contribution to those goals? That also is not in place. DFID has commissioned some work to advise on the methodology required for measuring that kind of work, so we wait to see the response to that, in terms of what is taken on board.
In terms of some of the evidentiary gaps, we are also highlighting the issue of collecting better value-for-money data and the impact data that we have talked about, which will come on stream. Lots of these are very much challenges to be addressed in the future, but it is clearly urgent that they are addressed in order to maintain the momentum that has been achieved.
Q24 Chair: Thank you very much. I would like to turn, if I may, to the issue of value for money in a bit more detail. If I recall correctly, when we first met at your appointment, Alison, we talked about the fact that there were three types of review—learning, impact and performance. None of them had the title “value for money”, but you responded that the thread of value for money and addressing that issue would run through all of your reports. I note that, in this report, you appear to have been quite constrained by DFID’s lack of proper monitoring and evidence on value for money. In fact, can I just quote from your report? On page 14, you say, “Evidence on the value for money of DFID VAWG programmes is currently limited.” In your recommendations on page 35, as an issue of concern, you say, “DFID does not systematically collect expenditure and value for money data for its VAWG work.” Is that satisfactory?
Dr Evans: The short answer is that we have to get our expectations in the right place on this one. When we are interrogating the question of value for money, we are looking for really quite high standards of evidence around the treatment of economy, efficiency, effectiveness and, going forward, equity. We have to understand that getting all the evidentiary basis to meet that standard is quite a big lift. This is not something that, quite frankly, one should do or could do on the back of an envelope, and basically say, “We know where we are going on this particular question.” The evidentiary demands of doing a really good value-for-money analysis, be it programmatic or across the portfolio, are quite high. We spent quite a lot of time looking at how DFID is investing in that evidentiary baseline, in order to do that analysis.
In this case, it is very much evolving. They are commissioning work precisely in this field but, for example, there is a real paucity of good comparable unit cost data in the field of violence against women and girls programming. Pulling that together consistently is something in which we now know DFID is investing. We are looking at it hard, but we do not want to give the wrong signal by saying either that DFID is investing and therefore it has done an entirely good job, or to give the signal that we expect this to all be complete, at this early stage in the development of the portfolio, because it is quite a big lift.
As a general point, and maybe Peter or Tina want to add some more, we are beginning to see through our work a bit of a disconnect in DFID. This is something we will probably come back to you on more and more. On a business-case-by-business-case basis, the thinking around value for money is mature in the department. There is a lot of discussion. The metrics that one would want to see applied exist in the putting‑together of business cases.
The problem that we then have is that those are not necessarily comparable across business cases and therefore across programmes, nor is it possible to add them up across the portfolio as a whole. We are somewhat constrained in how much sight we have of a general value-for-money case, because the data simply is not in a format that we can really scrutinise and test.
I am hoping that DFID will fill that gap and do so for a whole number of areas that we are scrutinising. At the portfolio level, we need to understand better whether this commitment of resources makes sense versus a commitment to another sector where VAWG goals might be achieved. We really need to have that but, at the moment, there is a bit of a disconnect. At the business-case level, we are confident that value for money is being applied. What we cannot do is aggregate that up across the portfolio as a whole.
Q25 Chair: Is that a “no” then?
Dr Evans: Sorry, that was a long answer. Are we concerned? We are concerned that we want to put pressure on DFID to keep filling that gap. We are not concerned that there is no attention to value for money. There is attention to value for money. It is very much part of the day-to-day business of how DFID works.
Q26 Chair: Can I ask you to comment on DFID’s management response to this issue and then more widely? Do you think it is sufficiently detailed to address the very specific concerns that you have raised in your report?
Dr Evans: No, I do not think it is. Yet again, it is part of our letter to DFID to that effect. We have set out very clearly where we feel it falls short. We want to see improvement in that regard. We have had assurances that this process will be looked at inside DFID, so we are waiting to hear a response back.
Q27 Jeremy Lefroy: I just want to ask a very final question, Chair. I apologise if this has been referred to before, but did you come across any evidence that DFID was keen to support small‑scale but extremely effective programmes tackling violence against women and girls? I will just give one example that I know about, having been involved, of a refuge for girls in the Mara Region in Tanzania. It was constructed through the local Anglican diocese there, but with support from the Britain‑Tanzania Society, of which I am a member. It was very cost‑effective. Demand for it was hugely more than they even anticipated and it has been extremely beneficial in protecting particularly girls against FGM, as a place to which they could come. I just wondered if you saw work from DFID either supporting that second‑hand or indeed first‑hand—obviously not that, but other such places that are really concrete examples of the effectiveness of protecting particularly girls but also women against violence.
Tina Fahm: Yes, absolutely. We saw many small‑scale programmes that, as we have said, if scaled up could really make a transformational impact. One such programme, which stood out for me, was a programme that DFID initiated in India, called SafetiPin, and we refer to it in our report. What was impressive in that case was that this was really innovative partnering with a private sector provider to develop an app that will enable women to assess and survey their own environment, with regard to safety. What was particularly impressive was that, within the programme, this was a step that DFID did not have to take but did, and it has been hugely successful. That will remain with me, in terms of the impact that is making in empowering women with regard to dealing with and making an assessment of violence.
Peter Grant: Complementary to that, the challenge for DFID is how you can fund small initiatives in a way that does not completely expend your resources, in terms of the analysis and so on to do so, so finding umbrella mechanisms by which they can provide that funding. When we were in India, we also saw the Poorest Areas Civil Society strengthening programme, which is supporting a very large number of grass-root NGOs, through a consortium, where we did see exactly those kinds of projects flourishing. That has been done to scale. That is impacting millions of people, so we felt that was a very strong example.
Chair: Thank you very much, panel.
Witnesses: Nick Dyer, Gerard Howe and Professor Charlotte Watts.
Q28 Chair: Good morning, panel. I believe that you were all here for the previous session, so you will have heard that there were quite a lot of concerns expressed by ICAI in their report, notwithstanding the fact that they gave you a green overall score. I would appreciate it if you would comment on a number of those concerns now—perhaps the key concerns. Particularly let us know which you think are the most important. Just to refresh us: DFID does not yet have clear plans for scaling up; it is not maximising learning; external research is limited outside the department, so the potential value of the work might not be fully realised; and it does not systematically collect expenditure and value-for-money data for its full work. I am sure that these will be explored in more detail as we go on, but would you like to comment on where you see that there is room for serious improvement?
Nick Dyer: Thank you, Chair. Let me start by saying that we really welcome the report, particularly the appreciation of the challenge of violence against women and girls and the leadership that the UK has been showing in this space. I would add to that emergencies, and you picked this up in your conversation previously. I know that is not covered in the report, but we have done quite a lot of work in that area as well, which we may touch on during the course of this conversation.
For me, the overriding challenge in this space, which is picked up in the report, is that this is about real deep‑rooted social norms and behaviour change. The challenge is getting local leadership in getting those behaviours to change and those norms to change. The report raises some useful and helpful channels around our learning. We need to pick up on and respond to the questions around value for money, which we will do in the course of this.
For me, the question goes to the heart of the issue that Mr Lefroy raised around how we get the international system to scale up into this space. I think that DFID’s value over the next few years is as much in its knowledge work, and in driving ambition and leadership from others, as it is from our own programming. Getting our programming right is important, but DFID just cannot lead this itself; we have to get others to step up and lead alongside us, because we will not be able to do it alone. I am sure we will pick up on some of those issues as we go through.
Q29 Chair: A specific area of concern that ICAI outlined is insufficient beneficiary engagement. Could you comment on how that could be developed?
Gerard Howe: Thank you, I will take that forward. As ICAI was saying earlier on, we recognise that there are sensitivities around beneficiary engagement regarding violence against women and girls. It is really important to bear that in mind. Overall, the direction of travel on beneficiary engagement from feedback in DFID is now very positive, so we can see it is reflected in our standard operating procedures and the Smart Rules, both as a value and a principle, setting out that our staff should take the views and experiences to inform design and delivery, and to use that feedback. Again as ICAI was setting out, they saw some good practice in the Poorest Areas Civil Society programme in India, which is very welcome.
Across the portfolio, there are some really interesting interventions at the moment. The Sonke Gender Justice programme in South Africa is working on the interconnection of masculinity and violence. They have established community advisory groups of men and of women to feed back on the success and the design of that programme. The STRIVE programme in India is working with female sex workers. Again, you have female sex workers forming community advisory boards for the pursuit and monitoring of that programme. In the COMPASS research programme, you have adolescent girls in Ethiopia guiding that research, selecting sites for research and feeding back. That web of learning around beneficiary feedback is there. What Works is thinking about the ethical ways of taking that forward.
Q30 Chair: Can I just ask about the statistics in ICAI’s report? “Consultation with beneficiaries was mentioned in only five of the 23 business cases that we reviewed, and only two of our desk reviews showed evidence of beneficiaries’ involvement in the governance of the programmes.” Are you saying that would be different now? Are you saying that this was out‑of‑date information?
Gerard Howe: I am saying that there are some examples of good practice, but we acknowledge that we do need to do more learning on this, absolutely. One of the specific things that we would now like to take forward is to use that resource that we have in the VAWG helpdesk, which we have referred to a number of times already. In terms of learning, we will look across our portfolio at what lessons are emerging on beneficiary feedback so we can feed it back better into business-case development and programme design.
Nick Dyer: I do think, Chairman, it is fair to say that, in response to previous reports, there is recognition of the importance of beneficiaries and building that into our operating procedures. We need to do it better.
Q31 Stephen Twigg: Can we explore a bit more this challenge of scaling up, which ICAI identified as the key challenge? Why is it so difficult?
Gerard Howe: First, and Nick set this out at start, we and ICAI recognise that challenge. The global evidence base is just emerging on violence against women and girls, and it is only relatively recently that international attention has been paid to this issue. We are in the foothills of moving forward on scale. We all accept that overall challenge that a global, national and local response at scale to violence against women and girls is utterly critical.
Referring back to Mr Lefroy’s question in terms of the right contribution and responsibility of DFID in that space, this is the critical question for us. What we see ourselves, as Alison described, is DFID positioning itself. We would describe our position at the moment in that question of scale as revolving around three main elements, on which we also seek to work together.
The first element is around knowledge. In the ground‑breaking world‑leading investment in What Works, we are doing a range of things, including thinking about bringing together action at national level, for example. In Ghana, the What Works advisory committee is coming alongside the National Advisory Committee on Girls and Women, thinking about that national response. National responses are obviously critical. In that same area, we have also seen our work on FGM and CEFM. We have worked extremely hard to promote African leadership and African ownership of that agenda, as we have seen in the African Girls’ Summit in Zambia recently.
We are also building knowledge, so that we are ready for the future. Under What Works, we have reviewed the implementation of guidelines on violence against women and girls in the response to Typhoon Haiyan, including consulting with women’s rights organisations. That was very well received. We are now ready for the next emergency to take that forward at scale. Our knowledge is an enormous part of that. Alongside What Works, we have a humanitarian COMPASS programme, which is a complement component to What Works, and the Global Girls research programme, looking at the whole range of adolescence.
Alongside research, we have our programming work. We have talked a little bit about that. There have been 80 new programmes since 2014 and work in 29 countries. We have talked about the scale of PACS. The work on social norms is important. There has been a 40% increase on work on social norms in our programming work over the last three years. Again, that is DFID responding.
Finally, we have our work on global influencing and across HMG, which we have referred to a few times. For example, we can point to influence over the World Bank’s gender strategy to better take on issues of violence against women and girls. We have talked about success in Global Goals terms and work with the EU.
It becomes really interesting where we bring those three together, in terms of scale. The Rwanda Indashyikirwa programme is a very good example of that, which is the Rwanda country programme working on gender‑based violence. What Works added an evaluation specialist to improve the quality of that programme to look at what that would look like going forward, and then our country team is working with the World Bank, which is now interested in taking that to scale. Similarly, we are bringing evidence from What Works into our international discussions with multilateral partners. We believe that is part of the answer to scale.
Q32 Stephen Twigg: You have given a number of really striking examples there. ICAI concluded that you do not have a clear understanding of how to mainstream initiatives in this area without affecting quality. Do you not accept that conclusion? You have just given us some examples of where things have been mainstreamed.
Gerard Howe: We would point to some examples in What Works, where we are building the platforms for mainstreaming. For example, we are working on schools and life skills programmes, Ujamaa in Kenya and education and development in Uganda. Alongside large sectoral programmes, we are introducing components that will then have a broader effect on that multi‑sectoral approach. Again, we are still at the very early stages of providing the evidence and the ways of doing this.
Nick Dyer: Can I add to that? Mainstreaming is, in part, about getting people to do things that they will normally not do. If you look outside of DFID, particularly in new areas like this—and we see this with our international events—you have to persuade people, first, that there is a problem and, secondly, that this is a problem they feel and they need do something about. That is partly about the pressure on other leaders to feel that they have to do something about this. It is about making them feel that you can do something about it, in terms of the evidence and putting in place examples of how it works.
That really takes time. To do that well is quite hard. This is why the evidence work we are doing in this new area is so important. Bringing and continuing to bring pressure to bear on other leaders and community groups, and giving examples of what works, is a really important part of this agenda.
Professor Watts: If I can just add, in terms of the evidence base on prevention currently, most of our evidence comes from quite small interventions. They are very exciting, because they show huge impacts on violence—50% reductions—but that evidence has not moved to how we mainstream. What are the components you might integrate into education, labour and women’s empowerment interventions? That is where the studies that What Works is doing are trying to take evidence of that next step, where we get a much better handle on how we do integration in a meaningful way that leads to positive impacts on women’s experiences of violence.
Gerard Howe: If I could add one final point, in terms of a specific commitment and learning from the reviews, we now want to look at and develop a theory of change on scale, particularly for violence against women and girls. Responding very much to the review, the violence against women and girls team will start to think about what the components are in terms of knitting together those three elements and what scale would look like. We will work to develop a theory of change on scale and have that ready before the end of this year.
Q33 Stephen Twigg: As you scale up, how are you going to ensure value for money?
Professor Watts: The question on value for money is clearly very important. I am glad that it has been highlighted and it is something that DFID is taking very seriously. There are a range of actions on value for money that DFID is supporting. There is the overall OECD DAC indicator, which measures and monitors DFID’s spend on violence. Within What Works, they have developed guidelines on how you rigorously document the costs of different types of violence‑prevention programmes. As part of that, they will be costing seven different intervention models, so we will get more evidence about not only the unit costs of intervention but also the cost‑effectiveness of the different interventions being tested. Those guidelines are being promoted more widely, so we are also trying to influence the broader violence sector to start getting better data on costs.
DFID is also looking at how we better understand the benefits of violence prevention. We know the direct benefits in terms of women’s experience of violence, but what are the downstream benefits in terms of health, wellbeing and children? When we are thinking about value for money, we get more nuanced in terms of quantifying the returns of investment that we might get.
In terms of mainstreaming, what is interesting is that we are talking there about incremental costs. What might be the incremental cost of, for example, integrating a life skills programme with an existing education platform? There we potentially have important opportunities to have highly cost‑effective programmes in the future.
Nick Dyer: The Chief Commissioner made a really interesting and helpful point that, at the programme level, there is a sense that DFID is getting this right. We have explicitly made a lot of investment in value for money over the last five years. I am encouraged that ICAI thinks we are taking this seriously at the programme level.
There is a good challenge on portfolio‑level information and whether we have quite got it right in terms of collecting aggregate information and benchmarking information. We discussed the WASH report a couple of weeks ago; we are doing that on WASH, pulling together data so that we can make comparative benchmarked examples in our projects. That is a good challenge for us.
Q34 Dr Cameron: It is really helpful to hear you speaking about the importance of beneficiary involvement in this, because it strikes me that, if we are going to empower women and girls, they have to be at the heart of what we are doing. Their views on what works and how to take things forward should be integral to that, so I am pleased to hear that that is the road you are taking.
ICAI identifies an interesting dilemma: that scaling up is most effective through mainstreaming, meaning that programmes on this will be managed not by specialists on violence against women and girls but experts on other works, such as education and health. How will you integrate and monitor this across your sectoral programmes without compromising quality?
Gerard Howe: That is right; part of the challenge of scaling up is making sure there is an organisation‑wide response. So far, we as DFID have taken the tack that focusing on three particular areas—FGM, childhood and early forced marriage, and evolving emergencies—has given us an entry, including the non‑specialists. We know we need to recognise that shift from thinking about types of violence to the underlying drivers of violence, which again is why the work on social norms is so important. We are pleased about that. We also need to learn lessons from our work on girls and women, which is also within my department. It is leadership, it is evidence, it is communications that really matter across the piece, as much as specific technical skills.
We have over 130 members of our community of practice on violence against women and girls across DFID, which has been a really important forum for sharing learning. These are not all technical specialists or social development advisers; they are a broad range of staff. We want to review the effectiveness of that under our country-support programme, so again another concrete commitment that we are making is to look at that community of practice and how we could make that even stronger by the end of this year.
We have produced a range of how-to notes. There was discussion on health earlier on, but we have how-to notes on health, education, economic development, justice and social norms for staff. These are accessible documents that tell people how to go about this stuff. At the moment, at last count, we had 9,000 downloads of those how-to notes.
We have just recently commissioned another piece of guidance for our staff on the right delivery channels for working on violence against women and girls. We will be producing that. That will say, if you are sitting in a country office as a programme manager thinking about civil society organisations or working with the UN, what the benefits are and what we have learned.
It is important to make that learning and knowledge accessible. We know that ICAI found evidence of increasing staff engagement in our teams in India. We know that Malawi, by taking this integrated approach, has built up the competence and experience of all the staff there, and we want to continue to monitor that. Our continuing professional development competences for advisers are a really important part. Again, another concrete commitment is that we will take learning to at least three of those in terms of the how-to notes and work with groups of advisers to spread that knowledge. Finally, we are going to make additional resource available to the VAWG helpdesk to promote learning and uptake among a group of staff to ensure that we spread that competence and engagement as widely as possible.
Q35 Dr Cameron: In terms of monitoring and the other goals that you have as a department—things like getting girls into secondary education, for example—do you monitor what potential impact that might have on figures of child marriage or those types of issues, things where the goals might overlap? Are you getting to that stage?
Professor Watts: That is a really important question. It is one of the areas of new learning that we are committed to: to think about how some of our other sectoral interventions might potentially impact on violence against women. One of the new commitments we want to make is, in some of our other evaluations, to look at what is happening around women’s vulnerability to violence as a result of, say, keeping girls in school or economic‑focused interventions.
Nick Dyer: A good example was in the WASH conversation we had on the previous report where, when putting toilets into schools, we have been looking at the question of whether that has led to a reduction of violence against girls.
Q36 Chair: Can I just go back to the issue of value for money? Nick, you said that you have been making investment in looking at this issue in the last five years. We are looking here at a four‑year programme. We have already heard this morning that ICAI says that they had insufficient evidence to make an assessment of value for money, which is obviously a very important area of their work. Value for money is also something that the British taxpayer is very concerned about. Bearing in mind that you have been investing in this issue for five years, why was there not a focus on value for money designed into the portfolio right from the outset—a flagship portfolio that the Secretary of State has promoted as one of the key areas of DFID’s work?
Nick Dyer: I would approach this in three different ways. One is at the individual project level. As the Chief Commissioner set out, there would be consideration of value for money, in terms of the results the investment we are making will give us and the cost of achieving those results. That is one element of the value-for-money question.
Q37 Chair: Sorry, can I just stop you there? That is at the initiation of a project, so why does ICAI say that you do not systematically collect expenditure and value-for-money data? Is that wrong?
Nick Dyer: What I am hearing and what I hoped I was picking up earlier is that ICAI is giving us a challenge. There are three challenges. One is whether we are doing this at project level. Yes. Two, do you have sufficient guidance to guide people in their value-for-money considerations? What we are saying is that, in this new area, that guidance is still evolving and it is quite complicated, because there are different types of interventions. Three, are we doing a sufficient job of gathering and aggregating the information at a portfolio level on value for money? Are we creating databases for comparative measures? We have not done enough on that. That is a challenge that we will take away and look at, because it is a fair challenge. We are working on the guidance with What Works and we accept the challenge on aggregation.
Q38 Chair: Thank you. Obviously you have had the report for some weeks now, prior to publication. You have issued a management response, but it is really a very short paragraph on this area, isn’t it? Are you saying that you would be able to come back with much more detail on how you would respond to their recommendation here that DFID needs to set a path to including VAWG more fully in its data collection and measurement systems?
Nick Dyer: We are giving advice to our Secretary of State, so she will take the decision on how to respond to the report. We are also going to be having a learning conversation with the ICAI commissioners on VAWG, so I hope we can get into these conversations during that session.
Professor Watts: In terms of the value-for-money issue, it is really important that we learn and are integrating that into the What Works programme, which is what is happening. It is all very simple to say, “Let’s look at unit costs,” but for something like violence, prevention does not just reach women who are experiencing violence. It also reaches women who are not experiencing violence who might experience it in the future. In that cost per beneficiary, you have to think about who that is. If we are talking about social norm change, that is shifting whole societies. When you think about the unit cost linked to that, it is quite tricky.
Part of the challenge of trying to make those comparisons is whether we have a clear theory of change, so that we can really look at the unit with which we can then compare different programmes. That is the work that is happening now, so that we can go into those unit cost comparisons in a much more informed way, across different types of programmes.
Chair: What you are saying is that ICAI’s report will help you to focus more closely on something that you have been looking at for five years, but perhaps have not yet bottomed, if I can put it like that.
Professor Watts: Exactly, the cost data is there. It is much more how we look at the unit outcomes, so that we can make those comparisons across different options around investments.
Q39 Jeremy Lefroy: If I could just pick up a couple of things first from points that have been raised earlier, before I come on to my substantive question, to the last panel I raised the question of this project that the Chairman and I saw, along with Virendra Sharma, in Kananga last week, which had been stopped by DFID because of problems elsewhere. I fully understand that, but we felt, or certainly the local perception was, that it had been successful—and we had no reason to disagree—in terms of bringing the police and the community much closer together. One of the specific things that one of the women we spoke to spoke about was a reduction in domestic violence.
In taking a decision to close such a programme, which obviously has at least an indirect impact on a very major priority for the department, would the impact of work on violence against women and girls have been taken into consideration? This was going on in a different part of the country from that where the problems that led to the closure of the programme happened.
Nick Dyer: I do not know the details of the case. When we make decisions about closure of projects, we clearly need to look at the direct and the unintended consequences of that closure. I do not know whether that was done.
Q40 Jeremy Lefroy: Would you perhaps be able to write to the Chair about that and get back to us on that? I think it is a very important input into our report on Congo, because we felt we wanted to raise in our report whether that programme, at least in the area we were in, could be looked at again, because it clearly had had a very beneficial effect. A lot of people were quite disappointed that it had not been completed.
Nick Dyer: I would be happy to come back to you on that.
Q41 Jeremy Lefroy: Thank you very much. The second point came from something that Mr Howe raised about multilateral influencing. I was very interested to hear about this work with the World Bank. To declare my interest, I chair the Parliamentary Network for the World Bank and the IMF. Have you found the World Bank, in particular in this case, and other multilateral institutions to be receptive to mainstreaming this in their work? Clearly we work bilaterally with only 28 countries. The World Bank is in 185 to 190, and therefore that is a huge vehicle for influence.
Nick Dyer: In short, yes, we have found the part of the World Bank that deals with women and girls to be receptive. The gender strategy that they have been working on for a number of years does now include recognition of gender‑based violence in their strategy. They have integrated it into their strategy. The World Bank, as with all multilaterals and as with DFID, has different experiences in terms of the uptake of their strategies. I would say that the World Bank, every six months, through the senior advisory committee on women and girls that Jim Kim chairs, sends an important signal that they take this issue very seriously. We were particularly pleased to see that they did embed violence against women and girls into their strategy.
Gerard Howe: If I could just add a couple of things, firstly, clearly component three of the What Works programme, looking at making the case on the economic costs, is going to be really important in responding to what we know is a big incentive for decision‑making across the World Bank. Also, the World Bank is represented on the What Works board, so we have that integration there as well.
Q42 Jeremy Lefroy: I realise that it has not yet been published but will be very shortly. In the Multilateral Aid Review, did the question of the approach of the organisations to violence against women and girls play a significant role in your assessment? Do not give the specifics away, because you have not yet published it.
Nick Dyer: As part of the methodology for assessing the individual institutions, we did look at their approach to girls and women, and that would have been part of that.
Q43 Jeremy Lefroy: Multilateral partners, according to the report, sometimes see DFID as “more inclined to pursue its own priorities than align with others, and is not always supportive of harmonised processes”. Is this something that you recognise and, if so, what are you doing about it?
Nick Dyer: In some cases, we understand the challenge and we have taken the view that we will take a different approach. A good example is that, over the last number of years, it has been a challenge in the European Union to do division of labour. We have always said that we wanted to take an approach that is based on country priorities, rather than the priorities set out by the international system. We will respond to each challenge as it comes up.
Jeremy Lefroy: What you are saying is that you would sometimes disagree with their approach and perhaps you sometimes see the problem as them, rather than DFID, which is a perfectly understandable approach to take.
Nick Dyer: To some extent, but we are also happy to challenge the international system where we think it needs to be challenged. We think that is part of our role. That is part of the UK’s role.
Q44 Jeremy Lefroy: Are there particular parts of the international system that you feel are not taking women and girls, and more particularly violence against women and girls, seriously?
Nick Dyer: The Girl Summit was a good example of something we did not think the international system was taking seriously, which was why we gathered it together to try to raise the profile, identify the evidence and get movement in that area, both within Africa but also within the UN, with the World Bank and others. Yes, that was a very good example.
Q45 Jeremy Lefroy: Do you feel that you are pushing at a fairly open door or a door that has difficulty in opening, in many cases?
Nick Dyer: It is patchy. If you look at the follow‑on response to the Girl Summit, there have been other girls’ summits that have spontaneously happened in Africa, which we think is a good thing. Last week, we saw legislation to ban child marriage in Ghana and Tanzania, and there have been legal bans on FGM in some countries, since the Girl Summit. There is progress, but clearly it is patchy. You have individuals, leaders and particular communities that are very resistant.
Q46 Jeremy Lefroy: Which countries do you see as very good partners with us in this work, and very much encouraging us and standing alongside?
Nick Dyer: If I look at the list of countries that have had girls’ summits, in Nepal, Prince Harry was very heavily engaged. Uganda, Bangladesh and Ethiopia have all had individual girls’ summits, so you would intuitively think that they would be good partners in this space.
Q47 Jeremy Lefroy: Finally, one thing that Mr Howe mentioned is this community of practice. Clearly, economic development is an incredibly important part of DFID’s work. Work and livelihoods are where possibly some of the greatest challenges for women and girls arise, and the greatest opportunities. Is the economic development department very much involved in this community of practice? Are they strongly represented?
Gerard Howe: I do not have the details of all of that, but what I do know is that, with the Secretary of State on the High‑Level Panel on Women’s Economic Empowerment, there is a very robust discussion and close working between my department and our growth and resilience department, looking at the multiple factors that influence women’s economic empowerment. Violence has been part of that discussion as well, in terms of safe work spaces. Within What Works in Bangladesh, for example, we have work that looks at safe work spaces for women.
Professor Watts: In terms of future learning, we are looking at some of our research programmes that are evaluating economic interventions with women to understand if that also reduces their risk of violence. We are actively trying to look at that intersection on growth and women’s vulnerability to violence.
Q48 Mr Evans: Considering all the investment that you have made in this area and that there is much enthusiasm from the Secretary of State as well, what are you doing to disseminate the information and research that you have got to other people, so they can benefit from that?
Professor Watts: We have been actively disseminating some of the baseline findings from the What Works project. That is the major research component that needs to be promoted widely. For example, some baseline evidence summaries have been widely disseminated and used by agencies, such as UN Women, to inform some of their strategies. We have seen about 7,000 downloads of that evidence.
Moving forward, What Works is developing a research uptake plan. That draws on three different levels. There is uptake and engagement with stakeholders at the national level, particularly in the countries where the different studies are being conducted. There is also international‑level influence and dissemination plans that draw on the advisory committee for that What Works programme, with the aim that the information will be disseminated through those different stakeholder networks. On that advisory committee, we have representatives from WHO, the World Bank, UN Women and lots of the key agencies in this space. We have been active so far and there is a broader engagement and dissemination strategy that we are developing.
Q49 Mr Evans: I might as well fast‑ball you this one. In a couple of hours we are going to have a new Prime Minister. Do you foresee any change in policy, as far as international development?
Nick Dyer: I will leave that to Ministers to decide.
Mr Evans: There is no plan B, Nick.
Nick Dyer: We are currently working to the Conservative manifesto. Until such time as others direct us otherwise, we will continue to do that.
Mr Evans: Good response.
Q50 Stephen Twigg: Nick, can I take you back to the Girl Summit? You have spoken a little bit about the follow‑ups, in terms of other countries having girls’ summits. Are you monitoring how the countries that attended have progressed since? In particular, what progress is being made on the specific pledges that countries signed up to?
Nick Dyer: Let me just preface this by saying that the Girl Summit was quite interesting, because it was a very different event for us compared with the previous summits that we had. It was different in three respects. One was that it was deliberately universal. We were saying that we had an issue in the UK that we had to address, as well as an issue overseas, so we had to get our own house in order. It was not designed to be about money; it was more about leadership. The third was that it was using and bringing in different groups of people, so it had a very different feel about it. It was not just leaders; it was about youth groups, community groups and women’s rights groups. We very much integrated that.
There were a number of commitments. We engaged with the UN. There was a one‑year report, which reported on the commitments that were made at the event. We have been tracking the follow‑up issues and follow‑up events that have happened. For us, this was about triggering a movement. It was not so much about getting lots of commitments that would hold people to account. It was about trying to get a movement going and I think there was success in that, just from the responses that we have had in Africa and some of the responses we have had in the changes to legislation.
Q51 Stephen Twigg: You talked about it being a very different kind of summit. Does that then have implications in terms of future summits on issues to do with gender, but also more broadly for development summits?
Nick Dyer: We had the Girls’ Education Forum last week in London. Similarly, that was about bringing together different groups of people. There were a lot more youth groups there, so that again had a very similar feel to it. We are using that more as a model, because I think it works.
Q52 Chair: Can I turn back to the very important issue of learning, which is really the theme of the review? There were quite a lot of criticisms made by ICAI in the report about the way that DFID is learning in this area, even though there was an overall green rating about which some may have a comment, bearing in mind the number of concerns that DFID need to address.
Just looking at one or two of the comments, basically there is this overall remark that “DFID is not maximising learning from its own portfolio”. Also, “There is limited evidence that DFID is systematically capturing lessons from its own VAWG programmes and sharing them across countries.” There should be “an active process of drawing lessons that could be applied more broadly”. “We did not see evidence of DFID trying out different interventions in different contexts and adjusting quickly in response to early evidence of what works.” Why have you not already put in place some systematic and comprehensive learning processes? Why was it that ICAI had so many concerns when they came to look at what is a flagship area of DFID’s work?
Gerard Howe: Thank you, Chair. The first thing to say is that we recognise that DFID as a whole is on a journey on learning, after the ICAI report of 2014. It has put in place the How DFID Learns initiative. You may be aware that looks at three elements: leadership and culture, skills and capability, and tools and systems. There is a more vigorous debate, right across DFID, on learning than any of us have experienced before, and that is very welcome. It is absolutely right and we recognise the scale of the challenge. If we are to achieve the scale that we were talking about earlier on, it means maximising the learning from our knowledge, from our programmes and from our international influencing. That is moving from building collections of knowledge to thinking about those connections and networks of knowledge.
I talked earlier on about how we are rightly proud of the way we have used learning through the evidence reviews, What Works and the how-to notes. We have seen Rwanda setting a goal of being a learning office. Obviously that is a strong connection in terms of Rwanda’s own leadership on violence against women and girls. Our community of practice is something that we also want to take forward further, but we recognise that there is more to do here. It would be wrong not to say that.
Again, one of the specific commitments that we want to make is to introduce a new role of learning officer to our helpdesk. This is a new additional role that will help us better knit together that learning that the helpdesk is spotting and knit together our work with What Works and other programmes.
Another good example is our social norms guidance note, because that was an exercise in learning, where we had an advisory group from across DFID looking at how to produce that note and then share it more broadly.
The final additional commitment that we want to make is to repeat the mapping exercise on violence against women and girls. In 2014, we conducted one mapping exercise with OPM. ICAI very helpfully helped in that, in terms of the review, and we want to return to that in 2018‑19 and look again at what that mapping exercise looks like, in terms of the overall portfolio on violence against women and girls. We take the need to maximise our learning very seriously. We will put additional resources into it, because that is the kernel of ultimate impact.
Q53 Chair: Would you agree that there needs to be quite a sense of urgency about this? I am looking in particular at the fact that ICAI points out that there is no overarching evaluation strategy for the portfolio. You almost have evaluation decentralised to country offices, so country offices are working in silos, which is something that this Committee has referred to a number of times over the years as very concerning.
My question is: how do you plan to ensure that the central VAWG team has a greater role in ensuring that knowledge gaps are addressed through evaluation? Can I confirm this by quoting from the ICAI report? It says, “There is no overarching evaluation strategy for the VAWG portfolio. While there are a number of evaluations planned in the coming years, they are not necessarily focused on addressing knowledge gaps so as to inform the development of the VAWG portfolio as a whole. A stronger role for the central VAWG team in the planning of evaluations and the dissemination of results would enhance learning.” Would you agree that needs to be urgently looked at and, if so, what timeframe would you be looking at?
Professor Watts: The team is picking up on the way that evaluations are structured currently, which is very much a bottom‑up and demand‑led model of evaluation. For example, we have ongoing evaluations of VAWG programmes in Ghana, Uganda, Zambia, Ethiopia and India. They have all been initiated by country offices that want to learn about the impact of their own programming.
For us that raises two questions. One is how we ensure that we can learn from those evaluations. We are going to get to a critical juncture, where they are starting to produce results. It is going to be really important that we commit to ensuring that those lessons feed in across the organisation, drawing on the community of practice, and ensuring that those findings feed into the central theories of change of how we programme effectively on this issue.
The second point it raises is that, in the overall evaluation strategy that DFID has, what is our balance between those bottom‑up, demand‑driven evaluations and top‑down where, strategically, we identify that we need to be learning and conducting strategic evaluations on an issue? That is an area that I am picking up and will be focusing more on in the future, in terms of how we get that balance right in our evaluation portfolio.
Q54 Chair: You would agree that there is a lot of learning to be done in the area of learning, which therefore begs the question as to whether the green overall score was quite generous.
Nick Dyer: I agree that there is a lot of learning to be done in learning. We do have a wider piece of work that is looking at how we get learning moving across the organisation. In terms of the overall score, it is for ICAI and the Committee to decide. We are more interested in the individual commentary in the report, which we look at more closely.
Q55 Chair: Would you accept that there are a lot of issues in it that need addressing?
Nick Dyer: We accept that, on the learning side, we have in the past accepted that we need to look at this, and we are looking at it.
Professor Watts: On the programmes and learning in the violence field, looking at the different national programmes that are happening and the content of those, they really are drawing on evidence and best practice. There is this challenge of how we learn across the organisation but, in terms of the types of programming that DFID has supported, it very much draws on a synthesis of what works and what we know in the field about the types of programming that will really make a difference.
Chair: Thank you. We look forward to hearing more detail about some specific concerns in the future, beyond the management report that you have already put forward in response. Jeremy, you have the final question.
Q56 Jeremy Lefroy: ICAI found that this is an area where the things that we have referred to before in managing small programmes, which is often a problem for DFID, seem to be causing issues. When I raised this in a specific way to the previous panel, they also seemed to indicate that there were some very good examples of relatively small programmes that are covered by this work, which are effective and scaling up. I wondered what your perception is of the ability in this area, which is an area, almost above all, where small programmes can be incredibly effective and therefore should not be discounted. I gave one example earlier, which you heard. I wonder what your perception of this work is on small programmes in tackling violence against women and girls.
Gerard Howe: It is right that much of the progress on violence against women and girls is going to be led by small groups of fiercely committed people changing the sorts of social norms that Nick has set out. That is the bedrock of an overall portfolio response. As DFID, we need a portfolio response that looks at the overall performance of the global system, national systems and local systems too. Under What Works, we have funded 10 innovative small‑scale projects already and those have begun. We are starting to pull out learning from those and they are feeding back into some of our country‑based programmes, so into the DFID Ghana gender‑based violence programmes.
I also head our civil society work. The question of how one reaches small‑scale organisations is a really important one and is something we are testing through our programme AmplifyChange, in collaboration with others. How does one get money and resource to small‑scale organisations that may not have a track record of five years of audited accounts but are creating some of that change?
Unsurprisingly, I would point to a number of very good small‑scale examples. In the UN Trust Fund to End Violence against Women, there is a range of examples, including one in Gambia, where we are supporting, through a $93,000 grant, small groups of people to work on behaviour change for FGM and CEFM. Our work with the UN Trust Fund is helping to promote that as an effective part of a mixed portfolio, from small to large.
Similarly in Zambia, we are trying some really innovative stuff. There is a project called Serenity there, which is looking at the combination of childhood abuse, violence at home, alcohol and risk factors for girls and women in innovative ways. We will always need a front end of innovation being trialled and tested, and a reach through to smaller‑scale organisations. That is an enduring challenge for us, but that happens through our country programmes, as well as through our central programmes.
Q57 Jeremy Lefroy: It seems to me that one of the things that DFID could learn from this work is how even better to engage with small programmes because, as you rightly say, this is a classic example of where often small groups of incredibly committed people can make a huge difference. You could move that learning into other areas, not just violence against women and girls.
Gerard Howe: Yes, precisely, and we have agreed that what we want to do now is a review of what practices are coming through in four countries—South Sudan, Nigeria, Bangladesh and Malawi—and how effectively we are doing that. We will pick up some of the learning about how to reach and what factors help support a resilient, small‑scale context for women’s rights organisations and other civil society organisations.
Nick Dyer: I would make two general comments. One is, when you are dealing with new areas like this, lots of small innovation and testing is the right approach. On the more generic question, yes, it is good to be supporting innovation across the piece. It is transaction‑heavy to do it for the organisation, so we have to do it in a smart way and potentially think about doing it through other parties that can do it on our behalf. We have 1,600 projects in DFID but, if we were to do lots and lots of small projects, that would just expand. There is a challenge for us in terms of how we manage that.
Gerard Howe: Peter referred to the umbrella projects and umbrella approaches, which we have and we are developing, as in with the UN Trust Fund and others.
Q58 Jeremy Lefroy: I fully understand that. Is there not a difference between the day-to-day management, the financial management and the other aspects of that, which can be done one removed? I would be wary of DFID getting too far away from the actual work that is being done, because it was being managed through too many intermediary agencies and there was a problem in terms of learning from what is being done.
Nick Dyer: Absolutely, we have to understand what is going on for effective project management.
Chair: If there are no further questions, thank you very much for coming to be with us today.