International Development Sub Committee on the Work of the Independent Commission for Aid Impact 

Oral evidence: ICAI's review on the effects of DFID's cash transfer programmes on poverty and vulnerability, HC 1008.

Wednesday 22 February 2017

Ordered by the House of Commons to be published on 24 February 2017.

Watch the meeting 

Members present: Fiona Bruce (Chair); Stephen Doughty; Jeremy Lefroy; Mr Ivan Lewis; Paul Scully.

Questions 1-67

Witnesses

I. Dr Alison Evans, Chief Commissioner; Willem van Eekelen, Team Leader, Independent Commission for Aid Impact

II. Nick Dyer, Director General, Policy and Global Programmes; Jonathan Hargreaves, Head of Governance, Open Societies and Anti-Corruption; Heather Kindness, Team Leader, Social Protection, Department for International Development

Examination of witnesses

Witnesses: Dr Alison Evans and Willem van Eekelen.

Q1                Chair: Good morning and thank you for coming today.  Can I introduce to you a brand new member of our Select Committee, Ivan Lewis, who is also going to be on this Sub-Committee? 

I am going to start off with a general question.  You gave DFID an overall green-amber for its work in using cash transfers, which is the topic that we are looking at today.  This is your impact review report from January 2017, “The effects of DFID’s cash transfer programmes on poverty and vulnerability”.  Would you like, Dr Evans, to make an opening statement and lay out what you think the priority issues are for DFID to address?

Dr Alison Evans: Thank you very much and good morning.  This review examines the impact of DFID-supported cash transfer programmes between 2011 and 2015.  It is worth being clear from the outset that DFID’s cash transfer portfolio predominantly supports national cash transfer schemes where DFID is sometimes the main funder, sometimes one of many funders, but is always alongside national Governments.  The number of programmes delivered outside of Government is a relatively small percentage of the overall portfolio.  That is an important point.  That actually supports a somewhat distinctive feature of DFID’s support for cash transfer schemes, which is that funding of the schemes themselves goes along hand in hand with efforts to strengthen the national cash transfer systems that deliver them, which is why we looked at both in this review.

Overall we found clear and persuasive evidence of the scale and impact of DFID-supported cash transfers on poverty and vulnerability.  The beneficiaries of these schemes are small; sometimes as little as 50p per household per day cash transfer is the difference between going hungry or not, selling vital livestock or not, or being reduced to begging or not.  We did find some errors in the way DFID has recorded its results, which, to its credit, were quickly corrected, as well as some mixed results in a number of secondary areas such as the impact on health and nutrition, which we can perhaps come back to.  Overall, we consider this a smart use of aid that delivers good value for money and on that basis, where the conditions are right, we see a case for increased support. 

There are some important challenges and we mention these in the report.  It is worth stating the obvious: it is important not to see cash transfers as some kind of silver bullet.  They are not the answer everywhere.  The benefits of them are sometimes somewhat transitory.  The main purpose of some transfers is just to prevent the worst happening rather than to actively move people out of poverty.

We also found fairly convincing evidence that cash can sometimes do more when combined with other interventions and address broader livelihood and empowerment issues.  Finding the right combination of interventions that works requires some experimentation and piloting, and we note the BRAC case in the review as a really good example of that.  However, it is not the easiest to do when you are trying to work with national programmes, as DFID has found. 

Improving national systems is slow, painstaking work.  We acknowledge that DFID is in many ways taking the harder road to achieving impact through its cash transfer work than perhaps taking the quick fix of delivering cash transfers outside of Governments to particular target groups.  We support that harder, more challenging path but we identify some aspects of their work that we feel could be strengthened in the short run to support this.  I will quickly note those.

We found that DFID programmes are not always clear about what additional results targets they are focusing on and are therefore not always optimised to deliver results in those areas.  We note a lack of a strategic approach to technical assistance and capacity support, particularly a lack of a consistent approach to improving the financial sustainability of these national cash transfer systems.  We are also not entirely clear what the vision is for the level of financial contribution that DFID is providing to these schemes.  We did not really get a sense of a clear strategy behind that.  We note in several places a lack of systematic monitoring and evaluation of technical assistance and what it can deliver.

We have two final other points.  While we note that there are clear improvements taking place in programme delivery in many of the cash transfer programmes that we looked at, including evidence of strong checks being placed in relation to fraud and loss, we did note certain weaknesses persisting in certain areas of targeting, transfer size, and timeliness, which again tend to mean that these programmes do not produce the optimal level of results that could be achieved, and we think DFID can do better in that area.  Finally, while DFID has responded well to the Public Accounts Committee recommendation on VFM in this area from 2012, there is still room to push even harder on using value‑for‑money analysis to manage the portfolio, to scale up DFID support where appropriate and to tackle underperformance.

Q2                Chair: Thank you for that very helpful summary.  Mr Eekelen, would you like to add anything?  Can I say that you are the team leader, so thank you, because I found this a very readable report?

Willem van Eekelen: Thank you.  The only thing I would like to add is that we have been impressed on a number of occasions and we were wondering why the performance was good, and one of the reasons is that there is a very good community of practice within DFID, so whoever you talked to would be aware of what is happening where and what the lessons have been there, which were then often replicated in other countries.  That is the way that international development progresses.

Q3                Chair: You make four recommendations.  Are there any problems you found in the review that you did not follow through to a recommendation but you would specifically like to draw DFID’s attention to?

Dr Alison Evans: The recommendations as presented cover the priority areas as we see it.  We reference what we regard as a potential case for expanding DFID’s financial contributions to schemes that meet certain conditions.  It is certainly one of DFID’s aims to expand coverage of many of schemes because many of them are actually quite modest.  There are many, many extremely and chronically poor people who are not being reached by these schemes currently.  Our first recommendation really touches on that concern and we refer to issues of coverage in several places in the report. 

The remaining recommendations touch primarily on the need to be very clear about the relationship between the goals of cash transfer schemes and how that is operationalised through the programming and the targets that are set for particularly the secondary impacts, and then the areas around technical assistance and support for capacity of national transfer schemes.  Those were the primary areas.  Clearly, the focus on technical assistance is designed to pick up the issues around targeting transfer size and timeliness, which is a running theme through the review.

Q4                Chair: It is quite a positive review, though you say there is no room for complacency.  We are interested to know what you feel DFID should take away from it.  I am particularly interested to know also what you feel about their response, which was received after we prepared for this Committee, so it is difficult for us to look at that in the detail we would have liked. Do you feel that their response was satisfactory?  Do you feel they are addressing the issues that they should be?

Dr Alison Evans: We have had several conversations in this Committee about the evolving state of the management response from DFID to our reviews.  My own view is that we have received something that is more specific and more directly focused on our recommendations.  While we are somewhat disappointed that they do not fully accept our first recommendation, given that this is also a recommendation that follows on from previous recommendations made in this area by the Public Accounts Committee in particular, we recognise that this is an area where they are being relatively cautious and they need to ensure as they go forward that they are building their own financial contribution on robust programmes in country.  To some extent that explains the “partially accept” around recommendation 1. 

We would like to see perhaps even clearer delineation of a set of time‑bound actions linked to each of these recommendations.  That is the area that we continue to press in our dialogue: a desire to see what those time-bound actions look like.  The danger here is that we will do a follow-up in a year on their response to this report and we will not actually be able to track very systematically what they have done following this management response and that is clearly not desirable.

Q5                Paul Scully: Last year ODI published a comprehensive review of the evidence of cash transfers.  How did you use and build on that review?

Dr Alison Evans: I will say a word on that and then pass to Willem.  The ODI review was published just as we were coming to a conclusion on the evidence-gathering stage of this review but very kindly ODI had shared early drafts with us just to set out broadly speaking where they were coming out in terms of the review of evidence.  I have to say it is an incredibly comprehensive review of the independent evidence in this regard.  We were able to draw on that review to shape some of the themes that we were able to address in this review but also to identify a series of benchmarks against which it was reasonable to assess the functioning of the impact of DFID’s programmes.  We were able to do that because we got sight fairly early on of the early stages. 

Willem van Eekelen: An example that Alison just mentioned is education.  We found that although cash transfers can keep children in school and put them in a school uniform, it does not necessarily help them to learn if the education system is bad.  That would have been a disturbing finding had it not been that the ODI report, which is very relevant, reasoned and thorough, came to the same conclusion.  Cash transfer has limitations, one of which is that if the education system is bad you will not increase learning results just by giving somebody cash.  Those reference points were very useful. 

Q6                Paul Scully: Do you think you were able to add more to that detail that they had?  You were obviously coming to similar conclusions but does this review add more to the ODI review?

Dr Alison Evans: The most important thing is that we were looking exclusively at programmes supported by UK money.  The ODI review is a global evidence review.  We wanted to drill down into those particular programmes and to understand the relationship between the funding of the schemes themselves and this broader agenda around system strengthening.  That is actually not the angle of the global evidence review that ODI developed.  What we wanted to do was to understand the contribution that UK aid was making to that.  We see it as complementary but really more of a drill-down into a particular area of the programme.

Q7                Paul Scully: You had a lot of detail in the report.  You interviewed 36 DFID staff and 12 academics and peers from other organisations.  Who did you speak to on the country visits and how did you make sure that you were speaking to the people that you wanted to speak to rather than just the people DFID wanted you to speak to?

Dr Alison Evans: Just to clarify, overall we spoke to about 300 people, beneficiaries, non-beneficiaries, as well as professionals in this arena and government officials and others, upwards of 25 different types of organisations from the smallest local NGO in Rwanda all the way up to major multilaterals working in this space.  We had a very wide-ranging sample of people to talk to, including critics of cash transfers or cash transfer-type schemes.  Perhaps you can give a bit of a flavour of the types of people we spoke to.

Willem van Eekelen: The way it would go is like this: you take a map of the country and see where programmes are and then it is us, not DFID, deciding which regions to visit.  Within the region we would make a similar choice about which village and then in the morning we would change it a little bit to avoid the risk of staging, so you plan to go to one village but in fact go to another.  That way the ball is in our court to make these choices.  While DFID was really helpful with the logistics and stuff, it was not them who agreed with people to meet with us.  When it comes to ministries or departments or think-tanks, we would often speak with primary counterparts of DFID but also snowball from there and find other critical elements that we would speak with, including organisations that do not use cash transfers or that use micro-grants which is a one‑time big cash transfer instead of repeated small ones.  We talked with a range of other donors—the World Bank, UN agencies, the EU, USAID, the Australians—so really we had quite a range of perspectives, including perspectives that were not shared with DFID. 

Q8                Paul Scully: You have looked at cash transfers in a particular context, but you said you were going to leave the humanitarian context to a later review.  I am just wondering why that was and why you did not consider it at this time.

Dr Alison Evans: They are really rather different types of programmes.  Cash transfers in this developmental context, where the focus is very much on building nationally sustainable systems for social assistance, is a very different context from delivering cash in fast-moving humanitarian contexts where the targeting is all on need and not based on who needs to be lifted out of poverty at that very moment.  It is a very different enterprise.  It is hugely important.  It is growing and we understand also that the debate about the relevance of cash in a humanitarian context is very much of the moment.  We do hope to look at it but to try to do both together was simply too stretching to do thoroughly.  They are very different parts of DFID.

Q9                Jeremy Lefroy: Do you see cash transfer programmes as effectively being pilots that enable Governments to take them up if they see them as working?  Your recommendation 1 seems to imply that you would like to see larger coverage, which implies that you do not really see them just as pilots but as substantive in their own right. 

Dr Alison Evans: We can talk about this in the context of the social protection agenda more broadly and social assistance in the form of grants, pensions and various kinds of cash transfers is understood to be an important part of the social assistance pillar of social protection, as it is in most countries in the world.  These are systems that are now gradually being invested in by national Governments even in low-income countries, on the understanding that there are safety-net issues in all countries.  Even if you can move the majority of the population over the poverty line through growth and prosperity and investments, there are still ultra‑poor, chronically poor individuals who will need to be supported through some form of social safety net. 

The desire to expand coverage is really to get the fine-tuning of that safety net, so that it is absolutely with the right people, providing the right kind of transfer at the right kind of time—sometimes that is seasonal, for example—but is also able to do so on a financially sustainable basis, and that means national Governments funding it for themselves.  DFID is very committed to that road.  We see that as a very important path.

Q10            Jeremy Lefroy: The reason I ask that is in your very helpful annex 2, which is exactly the kind of information we like to see and which details all of the programmes, it is notable that most of the programmes, even in lower-income countries, have quite sizeable local government contributions.  In the case, for instance, of Rwanda £69 million as against £62 million from DFID.  In the case of Ethiopia, there are well over £1 billion, although £300 million is from the Government and £1.6 billion from other international donors in a very sizeable programme.  The one exception is Bangladesh.  It would be interesting to know why we feel that we should be supporting a programme in Bangladesh when there is not a national government commitment to that as far as your figures say.

Dr Alison Evans: There are elements here of piloting and finding out what works, but, as I said right at the outset, these are nationally supported programmes in the main—we will talk about Bangladesh in a moment—where Governments are already bought into the idea for themselves of providing cash transfers to the ultra-poor.  DFID is in there trying to make those better and work more effectively for those people, and we think they need to do even better on making them more financially sustainable for the future.

Q11            Jeremy Lefroy: Can I just ask a question about Bangladesh, so that Mr van Eekelen can reply to that.  There are one or two much smaller programmes, such as Myanmar, where there is not a local contribution.  In Bangladesh you identified some errors in the cash transfer programme that were leading to an over-claim.  Perhaps you can describe those errors and whether you would see any link between the fact that there is no national government direct contribution to the programme and the fact there were quite a lot of errors, or is that completely unlinked?

Willem van Eekelen: There are now three things I would like to respond to.  The first one is about the Government of Bangladesh and the contributions.  The Government of Bangladesh actually invest heavily in cash transfers.  Unfortunately those programmes are not of sufficient calibre at the moment for DFID to support them.  So, in parallel, the things that DFID has supported were NGO-implemented cash transfer programmes, so it is a different type of thing.  It is not a lack of commitment but a lack of quality.

Q12            Jeremy Lefroy: Could you just say how much the Government of Bangladesh, on your estimate, does invest in cash transfer programmes?

Willem van Eekelen: $2 billion a year.

Jeremy Lefroy: So substantial.

Willem van Eekelen: Very substantial indeed, yes, taking the shape of many different smaller and larger programmes.  What DFID do is provide technical assistance to the Government of Bangladesh in order to improve the quality and accountability of those programmes, but until that quality has increased significantly, DFID would not consider actually funding it.  They are working through BRAC and other national and international NGOs to do that.  That is one thing. 

I would like to go back to your pilot question.  What it is not is a set of pilots.  DFID has been very successful in sometimes demonstrating the effects of cash transfer through a pilot.  This is how the Government of Uganda got very interested.  Initially there was not a whole lot of interest but after a couple of very successful pilot years the interest grew and the contribution grew.  Sometimes it works the other way around; the Government of Zambia is very interested and DFID then jumps on the opportunity to help kick-start that process.  DFID is not doing pilots in order to check whether cash transfers work.  That is in the past.  It is now only tweaking things, such as, “Does a bank branchless type of debit card work?” and that kind of pilot. 

On the 475,000 people who were over-reported in Bangladesh, this has to do with the definition of cash transfer.

Q13            Jeremy Lefroy: Sorry, is it over-reporting, rather than over-claiming?  Are we talking about lots of extra money going to them, or is it just a reporting issue?

Willem van Eekelen: There was an over-reporting that was unnoticed by DFID.

Jeremy Lefroy: Over-reporting of the number of recipients.

Dr Alison Evans: Yes.

Jeremy Lefroy: Not of the amount of money.

Dr Alison Evans: No, they reported more beneficiaries than had actually received funds.

Q14            Jeremy Lefroy: How did that arise?

Willem van Eekelen: It is a definition issue.  A cash transfer is a regular payment to alleviate poverty and vulnerability.  A micro-grant is a single payment; it is a different thing with a different purpose.  An implementing partner in Bangladesh had provided micro-grants and reported that 475,000 people were reached with this.  DFID failed to notice in the reporting that this was not a cash transfer according to the protocol definition that DFID uses, so they had, without interrogating this reported figure sufficiently, added that to the total reach number.

Q15            Jeremy Lefroy: So there was not actually cash being sent in the wrong direction.

Willem van Eekelen: Not at all, and it may very well have been used very effectively but not as a cash transfer.  We then found this at the very beginning of the review, and it caused us to be particularly careful in scrutinising every other programme in our sample, and we can say with absolute confidence that this was the only major error.

Q16            Paul Scully: You say that the Bangladeshi national Government investment has lacked the calibre that DFID would be looking for.  I wonder if you could just outline the kind of things you would be looking for that were the shortcomings in that.

Dr Alison Evans: Willem can expand but the Bangladeshi Government invests significantly in cash transfers, so many of those schemes have history linked to previous political parties, or schemes associated with particular categories of workers who are considered to be eligible for cash transfers on retirement, say.  There are a lot of things in there that certainly DFID would not regard as a legitimate area for its own investment.  It is working very closely now, through its technical assistance programme, to try to help the Bangladeshi Government make changes to the targeting of those programmes. 

This is also where the point about political sensitivity comes in, because it is actually a very politically sensitive conversation to be had with a national Government about who it does or does not target with its own resources.  That is where the reference in the review comes to the political sensitivities that sometimes we feel puts DFID on the back foot around some of these questions about who benefits, whereas we would like them to be more on the front foot. 

Paul Scully: That is really helpful, thank you.

Q17            Mr Lewis: Where cash transfers have additional objectives, whether it be health or nutrition, education, women’s empowerment, there are clearly mixed results.  I would be interested to know if that is inevitable and inherent in terms of cash transfers, or are there very significant areas where DFID could improve as a consequence of that?

Dr Alison Evans: I will take an initial stab at that.  The answer is that they are not inherent.  The ODI review and many others in this space confirm that actually there are impacts that are achievable in health, nutrition and education, which set the expectation in relation to cash transfers.  It does come back to quality of design, quality and size of cash transfer, targeting and programme implementation.  We observe a range of effects, and it is really important to note that in observing a range of effects that means there are some really good, positive effects in there as well as some more muted and in one or two instances some negative outcomes. 

In relation to health and nutrition, we believe the effects are probably on average below what is achievable and what the global evidence would suggest is achievable.  We are appraised by DFID that they are working on that. 

In education it is a slightly more difficult picture because, as Willem indicated earlier, some results are simply not well linked to cash transfers, like improvements in learning outcomes.  A 3ie systematic review of the impact evaluation evidence on cash transfers in education pretty much confirms that.  You can get kids in school with cash transfers, you can get them with a school uniform on, but you cannot necessarily through cash alone improve their learning outcomes.

Willem van Eekelen: That is all correct and important.  It leads then to why the DFID programmes sometimes underperform.  Much of the explanation is the fundamentally wise choice to work through national systems.  In the short term, that means that there is limited capacity but in the long term it means that there is ownership.  That is the trade-off that DFID has chosen to accept and rightly so, because otherwise there is no exit scenario. 

On the occasion, such as in Bangladesh, where DFID funded directly and had direct control over cash transfer programmes, there were some examples where the impact was very impressive and above average.  There were also examples of programmes where we felt that DFID did not keep its eye on the ball sufficiently and there were occasions where programme parts were kind of meaningless.  There were transfer sizes that just did not do anything; 10p per elderly person does not help you, especially if it then stops after 18 months.  It included also occasions where actually implementing partners did very impressive things that could lead to real innovation that DFID failed to pick up.  So, in the direct implementation I felt there was scope for improvement in working with the Government, where on occasion DFID could have pushed harder to make improvements, but ultimately it is not under its control and it is a choice.

Q18            Mr Lewis: Is there any evidence that there are processes in place to disseminate and share best practice in the context of the cash transfer programme?  You have obviously been able to identify, in terms of additional objectives, some programmes that have worked, and you said that was largely where DFID had direct control or a greater level of control.  Is there evidence that there are systems within the Department to ensure that lessons are learned and best practice is shared?

Willem van Eekelen: Yes, there is an active community of practice that meets regularly, typically virtually but not only.  There is a system where one country learns from the other.  There are pilots that try certain modalities and then when they work or when they do not work that is shared with the rest.  What you sometimes see is that a model tried somewhere then pops up again on a whole different continent because two people have been talking with each other or one has made a visit where something seemed to work and seemed replicable in another situation.  There is an impressive community of practice, partly because people within DFID covering cash transfers have been there longer than most people in most fields.

Q19            Mr Lewis: In terms of clarity for both people operating the system and the recipients, is it clear at every stage which cash transfer schemes are about supplementing income and which cash transfer schemes are about achieving these additional or distinct objectives?  If I am a recipient is it clear to me that in these cases it is not just about me getting additional resources to supplement my income but I am expected to behave in a particular way as a consequence of that?  Is it also always clear in terms of the people operating the programme and the scheme, or is there some confusion?

Dr Alison Evans: The clarity comes where there is conditional cash transfers, which is rather different, as you know, from unconditional cash transfers.  In the case of conditional cash transfers—there are not that many across the portfolio that we looked at, but there are some—it is very clear then to recipients that this is more than supplementing income.  There is expectation of driving a certain behaviour or particular outcome, such as putting a child in school or taking nutritional supplements or whatever it may be.  Those are very clear.  Around unconditional cash transfers, it is a slightly different story, because by definition they are unconditional.  The expectation is that they supplement income and then good things will follow from that.

Willem van Eekelen: That is correct.  The only thing I would like to add is that one of the things that struck us when interviewing people in fields that received these cash transfers, they often considered it manna from heaven: “We do not know what happened but now we get this, and that is great”.  There was no sense of entitlement.  Ideally a cash transfer is not only regular but is also predictable, and it enables you to then make investments and other life choices based on the certainty that a year from now you would still have this as a safety net in case your entrepreneurial action does not work out.  Because there was no sense of entitlement, which is fair enough because these programmes are relatively new, as a consequence we did not see that long-term consideration of what this cash transfer can do for you.

Q20            Stephen Doughty: Given what you said about the community of practice and the sharing of information between programmes, are you surprised—the report is clear—that DFID does not have a clear strategy overall on the size or the conditions on its contributions to cash transfer programmes, and that there is such disparity in the targeting?  We have covered some of this already, but are you surprised that there are those disparities? Given the learning that has gone on and the staff that have been in place, should we not now have a much clearer agenda going forward?

Dr Alison Evans: As an initial response to that, there is perhaps a slight difference of opinion between us and DFID on this, which is that they very much approach their programming from the bottom up.  It is all about what we can do in a national context with the particular resourcing we have and our assessment of the suitability of national programmes for support, which is also how they derive their overall results target on what these programmes can achieve, which is very much from the bottom-up.  This means the idea that there should be some central capability that helps them drive strategy across the whole portfolio is something that they look at us slightly blankly on.  That is not how they go about it. 

However, where you are operating such a large portfolio and such a large commitment of resources, it would seem to us to make a lot of sense to have that anchored in some kind of central strategy, including providing the means to lesson-learn around technical assistance and what works.  There we saw more holes perhaps than in some of the aspects of what makes a good cash transfer programme. 

Willem van Eekelen: There was a hole we identified and feel confident about.  There is also a hole that we just did not get, all in all, and that is that some countries or programmes have a DFID contribution of 10% or 11%.  Other programmes have a DFID contribution of well over 50%.  There may be good reasons for that but we never found them.  There seems to be no coherent guidance that helps country officers make the decision on the percentage contribution to a programme.  We would argue that that guidance is relevant and important.  It is possible that DFID does not accept the first recommendation to scale up on account of not wanting to take space from the national Government out of fear of a substitution effect: that if we give an extra £1 million they invest £1 million less.  That would be more persuasive if we had found this kind of umbrella guidance that helps country officers make the choice about contribution. 

Q21            Stephen Doughty: Specifically with the evidence-based analysis on conditional versus unconditional cash transfers, do you think there should be a much clearer guidance set from the top about the validity of those or do you think it is inevitable that both styles will continue?

Willem van Eekelen: There are two things about that.  First, conditional and unconditional depends very much on circumstances.  In Latin America there is a lot of conditional because the education systems work, so if you then make a cash transfer conditional on school enrolment the learning outcomes will go up.  In parts of Africa that is not the case, so you can make it conditional but it does not do anything.  That is one reason for geographic differences.  The other thing is that DFID is not in the driving seat.  For example, DFID would prefer programmes to take the shape of direct support rather than public work programmes, but the Governments of Rwanda, Myanmar, Ethiopia and Nepal all like public work programmes.  So, the choice is not public work programmes or direct support.  The choice is: do you want to support the Government in that choice or not?  That should and does explain it more than internal guidance as to what is best.

Dr Alison Evans: Just to finish off that point, I would be slightly concerned about setting some kind of clear target for conditional versus unconditional.  That clearly is not desirable, but there is a case for much clearer assessment of both the size of DFID’s financial contribution and why, and also how that relates to the work that has been done on building financial sustainability within these national programmes and sense of the fiscal space for this kind of investment.  Those two things need to be addressed together and could do with clearer strategic guidance from the centre. 

Q22            Stephen Doughty: I was quite interested in what you had to say about the Pakistan programme and our involvement not just in terms of the contribution, but that our advocacy and involvement with other partners had helped change the overall programme as well as the direct impact on finances.

Dr Alison Evans: Very significantly.

Q23            Stephen Doughty: That is really helpful.  The recent media coverage around all of this includes people alleging we are exporting the dole or even this morning making some pretty big allegations about alleged fraud and other things.  Do you think that that coverage is fair?  Do you think it reflects where the programmes are overall?  Did you look specifically at the issue of the risk of fraud at all?

Dr Alison Evans: To the extent that any of that media coverage draws on this review—not all of it does, by any means—any suggestion that this review is pointing to a major problem of fraud or leakage in cash transfers is simply wrong.  This report does not say that.  In fact, we found good evidence that DFID in its support for cash transfer schemes and in its technical assistance is working very actively and proactively on any issues relating to fraud. 

Now, there is a slightly separate issue—and this is possibly where some of the media coverage has found an entry point—where we do make an observation about the extent to which DFID is able to be very proactive about talking about politically sensitive issues of targeting.  Some of those targeting areas mean that some individuals are left out of programmes where they should be included and some are in programmes that they should not be in.  Those are part and parcel of cash transfer schemes the world over.  DFID is working to try to minimise the effects of those.  However, the fact that we have observed that at least in one context, in Bangladesh, we felt that the country team were perhaps being a little diffident about some of that engagement over those targeting errors has got somehow turned into a comment from us that we think DFID is not clamping down on fraud.  It categorically does not say that.

Q24            Stephen Doughty: What awareness do you have?  Is there fraud?  Is there any greater fraud than you would expect generally in a programme of this size, or did you not look at that at all?

Willem van Eekelen: We found no evidence of any fraud in any of the programmes we looked at in our sample.  We found the checks and balances in place to be good.  Not only did we not find fraud, we found that if there were temporary sources of leakage the systems would be in place to identify that quickly and rectify it.  We saw examples of that. 

In addition, I would like to point out that cash transfer as a system is fairly incorruptible because, unlike food distribution and rural infrastructure, it is really hard to make money disappear.  You have got a list of people and an amount they get.  You go to the people and you ask them if they got it.  So, fraud is easy to detect and it is easier to detect now than 10 years ago because of debit cards, SIM cards and the modalities through which the cash transfer now is actually transferred.

Q25            Jeremy Lefroy: I know this was not the specific aim of the report but did you get any evidence or are you aware of any evidence that shows the impact of cash transfers on the creation of jobs and livelihoods, given that this is a high priority for DFID?  Obviously it would be a possible secondary result of this but I would be very interested to know.

Dr Alison Evans: We can give you some examples.

Willem van Eekelen: We went to Rwanda and we went to Bangladesh.  One of the reasons we went to Bangladesh was because the programmes there had cash transfer only as one of several components.  Those programmes in particular are creating sustainable sources of income, sustainable livelihoods.  Those programmes are impressive as well.  They originate in 2002 when BRAC piloted this, funded by DFID, after having made the observation that BRAC was reaching many people but not the poorest and most vulnerable members of society.  They also noticed that if they did reach them, they could not help them.  Out of that came an integrated range of services and products that together addressed the multiple overlapping vulnerabilities that the poorest members of society are typically subject to, and that then enables them to lift out of poverty or closer to the poverty line on a sustainable basis. 

It has been very thoroughly researched.  We were very impressed in particular with the BRAC evaluations in this field.  It has been tweaked over the years and it is a programme that BRAC but also DFID is now testing in other countries, where the circumstances, of course, are different.  If you want to create employment then that is a model that works.

Dr Alison Evans: We also saw some programmes in Africa too where there is downstream evidence of investment in agricultural assets, there is paying down of debt, which is often a great inhibitor to actually building a sustainable livelihood, and there is resilience to shocks over time; not having to sell cattle in a poor season means that actually you are able to breed your cattle and have more cattle in the next season, which builds the basis again of a stronger livelihood. 

In terms of jobs we did not see any particular programme that was looking to track and evaluate the impact on jobs but we certainly saw evidence of a number of the factors that enable a future livelihood that is sustainable and resilient.  Outside of this review I happen to know there is quite a lot of evidence now that is beginning to point to the relationship between cash transfers and jobs and livelihoods.

Q26            Jeremy Lefroy: It would be very helpful to have that information.  I suppose we ought to point out that BRAC is a Bangladeshi NGO that works in many other countries, sometimes supported by DFID, using the experience in Bangladesh; that is a great example of learning from their work in Bangladesh and DFID using that elsewhere.

Dr Alison Evans: They have done a very thorough evaluation of the transferability of that programme in seven countries outside of Bangladesh.

Jeremy Lefroy: Yes, we saw it in Tanzania a couple of years ago.

Q27            Chair: I have two final questions.  One is specific and relates to the fact that DFID’s value-for-money analysis looks at economy, efficiency and effectiveness.  ICAI has highlighted this fourth “E” of equity necessary to achieve the “leave no one behind” agenda.  In your report on marginalised girls’ education you noted DFID was weak on accounting for the equity of its spending in value-for-money analysis.  How do its cash transfer programmes compare?

Willem van Eekelen: The very mechanical answer is that the value-for-money assessments that have been conducted by DFID in the field of cash transfer never consider equity.  Only the Sahel programme did.  All the other value-for-money assessments did not consider this.  It is somewhat understandable because equity is part and parcel of the cash transfer idea so the notion is an equitable notion.  You identify those people who need it most and you provide them with the cash transfer, either indefinitely in cases of pensioners in Uganda, for example, or for a period of time until the harvest comes in, say, so it is fundamentally equitable. 

Within that fundamentally equitable programme there are nonetheless exclusions.  For example, we found public work programmes, both in Nepal and in Rwanda but not so much in Ethiopia, to be excluding certain groups by creating practical barriers; for example, mothers with small children were not able to go the distance they needed to walk in order to do that work.  DFID noticed that and is currently in the process of piloting alternative types of public work programmes together with the Government of Rwanda.  Why not, instead of building a road 10 kilometres from your home, help in a nursery closer to your home, and make that your public work? So, equity is very much in the minds of the people that are implementing these cash transfer programmes, partly because that is the nature of cash transfers and partly because they have a keen eye for exclusions.

Q28            Chair: The final question picks up on a theme that our Sub-Committee is concerned with because your reports have raised it a number of times, and this relates to a strategic approach.  In your section on building national cash transfer systems, under that you give an amber-red rating although the overall report is actually green-amber. You talk several times within your executive summary half-page about the lack of a strategic approach to strengthening national cash transfer systems.  Within these very few lines you say that “technical assistance programmes can become drawn too far into short-term problem-solving at the expense of a more strategic approach”, that “there is no clear strategy underlying the size or conditions of DFID’s financial contributions”, and that “DFID does not use performance triggers to drive reforms and it lacks a clear strategy for promoting financial sustainability”.  How would you like to see DFID set a strategy that would address this and who within the DFID structure should ultimately be responsible for ensuring that this happens?

Dr Alison Evans: We have come across this theme in a number of reviews, as you and this Committee has identified.  What we seem to be facing at the moment is a genuine tension between a desire of the Department to invest in a lot of what works, evidence-building and lesson‑learning attached to country programmes and within that programmes that are funded using UK aid, and the desire to look across the portfolio and then aggregate intentions and directions of travel, as well as evidence on what is working and how to build on that.  We feel that that is a missing piece and we feel that the Department feels that it is doing all of this but it is doing it at a much more disaggregated level. 

For our thinking, DFID has a central team working in social protection.  They have also a policy division that has a significant capability within it to be able to look across what is working, what is working particularly in a difficult field like technical assistance across its own portfolio.  We feel that there should be some commitment in part of the Department to bring that learning together and then inform a strategic approach that will have to evolve over time.  We are not suggesting it is a blueprint.  It can then be drawn down on for these country programmes.  That seems to fall within the responsibility of both the dedicated social protection team and of the policy division and under the Director General for Policy within DFID.

Chair: That is very helpful.  Thank you very much for your evidence today; it has been most helpful.  Thank you for coming.

Examination of witnesses

Witnesses: Nick Dyer, Jonathan Hargreaves and Heather Kindness.

Q29            Chair: We are pleased to invite Nick Dyer, who is Director General for Policy and Global Programmes, and accompanying him will be Jonathan Hargreaves, Head of Governance, and Heather Kindness, Team Leader, Social Protection.  Thank you very much for coming today, all of you.  We are set to finish at 11.00 this morning; we might go over a little further but if any colleagues need to leave then please just let me know.  Welcome everyone.  Our first question relates to the fact that the ICAI framework agreement with DFID sets a three-week deadline for DFID to respond to ICAI’s reports.  Yet again with this review, DFID has failed to respond within that time period.  Can you tell us why you were not able to provide a timely response in both this case and in the other cases?

Nick Dyer: Thank you, Chair. Let me start by saying that I absolutely appreciate the deep frustration of the Committee about the failure to meet the deadline, and I can assure you that we share that frustration that we find it difficult to and cannot meet the deadline, and we want to provide the best response possible with the Committee.  Mark Lowcock, Permanent Secretary, has just written to Mr Twigg on the issue of deadlines, saying that we do find it difficult to meet the deadline; quite a lot of the time is spent preparing submissions to our Ministers, a lot of time is spent making sure we get the clearances though the system, and a lot of the time is spent on responding to questions and queries, which takes us right back to the beginning, so we have to go through the whole process again.  We just find it very difficult to meet the three-week deadline. 

In the Permanent Secretary’s letter to Mr Twigg he is laying this out and saying that we would like to move to an eight-week period as we do with responding to the IDC reports.  We are proposing to have a conversation with ICAI around the framework agreement on that.  Let me just say that we do appreciate your frustration. 

Q30            Chair: You say you share our frustration but actually it is within your control to address it.  You may have seen my eyebrows rise at the suggestion of eight weeks.  You do have two weeks to fact-check before the three-week period sets in.  Could you not look at perhaps using that two-week period in a more constructive way?  You would have five weeks.

Nick Dyer: We not start responding to the report until we have the final report in front of us because the reports can and do change.  We have taken that choice internally to start the process, and certainly do not send it to Ministers until we get the final report.  Ministers want time to see the report and they want time to respond to the report and they want time to see the recommendations and they want time to respond to the recommendations.  Ministers are busy, they travel, and people are out of the office.  To squeeze all that in in a three-week period is very, very difficult.  We have said to the Committee a number of times that ultimately all our responses to the Committee are with the approval of and the direction of Ministers so we are working very clearly with Ministers in this process.  We just find it very difficult to meet the three‑week period. 

Chair: I am sure it is an issue we will return to.  Indeed, one of my colleagues wants to do so now.

Q31            Stephen Doughty: Does the suggestion of an eight-week timetable come from Ministers, or does it come from the Permanent Secretary, approved by Ministers?

Nick Dyer: I cannot answer that directly, Mr Doughty, but we have had conversations with Ministers around the timeframe and the frustrations of the Committee.  The letter that Mark provided to Mr Twigg was approved by the Secretary of State.

Q32            Chair: Thank you.  In that context, and bearing in mind that your response came on Thursday last week, 16 February—it is now 22 February—and that was too late for it to be disseminated to colleagues here on this Committee, could you summarise your response, and in particular why you only partially accept certain elements of ICAI’s report?  Could you also comment on the suggestion that perhaps there needs to be more specific points at which you will take action?  As the ICAI Chief Commissioner said earlier, there could be a timeline to which you could be held to account.

Nick Dyer: It was a very good report; we are always pleased to receive a green-amber report.  There are two broad conclusions that I would draw from the report.  The first is that cash transfers get aid to people who need it when they need it and it is done with good value for taxpayers’ money.  That is a really strong conclusion that we are very happy to see come out of the report. 

We also draw from the report the challenge, and we accept it, around learning more around our technical assistance work.  That is a fair challenge.  There is also a good challenge in the report which you started to get into with Dr Evans around this whole issue of how you make cash transfers have secondary benefits, beyond just poverty reduction and improvement in food security.  Maybe we will come back to that, but there is quite a lot of learning to be done in understanding how we sequence and improve the secondary benefits.  That is a message I take away. 

One thing I would like to make clear, which I was really pleased to hear Dr Evans say, is that we have strong checks on fraud and loss and that they found no evidence of fraud.  We absolutely categorically would say it is not true that we do not tackle issues of fraud when they are seen.  When it emerges we do tackle that. 

When it comes to the first recommendation on “partially accept”, our response to this is that we do not intend to provide indefinite support to cash transfer schemes.  We do not automatically take the view that we should be expanding the schemes and scaling up the schemes.  It is very much done on a case-by-case basis, and the pace and the shape of our support is a function of Government ownership, the strength of the systems and the finance the Government is prepared to provide and the financing that we have got available to provide as well.  It will be very much on a case-by-case basis.  There is no automaticity about this.

The one area that does lend itself to most potential for scale-up on cash transfers actually is the area that Mr Scully touched on but it was not covered in the report and that is humanitarian.  In humanitarian, the evidence is telling us that cash transfers compared to something like food is cheaper—our experience in Ethiopia is that getting the same benefit to people is about 25% to 30% cheaper—and more of the value gets to the beneficiaries; in Somalia we found that 85% of the value of the cash transfers was getting to the beneficiary whereas with food it was only about 35%.  It is also used well and used better, in fact.  70% of all Syrian refugees in Iraq sold their food and we know that, if you do household studies, cash is used for about 16 different purposes, so people have the choice on where to use it.  That has potential. 

In terms of the specific points in the recommendations and the action points and whether we are committing ourselves to deadlines, we have had this conversation with the Committee in the past.  Certainly with the previous Committee, we got ourselves into a situation of being loaded with literally hundreds of commitments, which we just found very difficult to track and follow up.  I commit that we will go away and have another look, certainly in this case, in terms of what we can find and additional commitments.

Chair: Thank you. You will have heard my concerns about the strategic approach needed, and that it is a theme that is coming to us again and again through the ICAI reports.  I will not labour that more at present; it may well be something that the Sub-Committee will return to.

Q33            Jeremy Lefroy: That is something I would like to pursue.  According to the report, the work lacks a strategic approach.  How do you plan to achieve financial sustainability without a strategy, or is there in fact a strategy that is not that clear?

Nick Dyer: You and I had this conversation around strategy when we were talking about the water and sanitation report.  I said then, and will repeat now, that when we write strategies in DFID they get out of date very quickly.  When I became the director of policy, back in 2009, DFID had 49 strategies, and no one had read them all; even I had not read them all.  No one was reading them because they would get out of date very, very quickly.  Policies change, Ministers have different views, and the evidence gets out of date very quickly; my own view is strategies get out of date very quickly.  However, what we do need to ensure is that we have good guidance, that we keep that guidance up to date, that we are investing in the evidence that tells us what works and what does not, and we are making that available to the staff who are involved with these schemes, like the community of practice. 

Now, on the sustainability of cash transfers, Jonathan may have a view on this.  As Dr Evans set out quite neatly, we do take a bottom‑up approach to this.  This does depend on factors such as the resources that we have available, the resources that the Government themselves have available, and the strengths of the schemes.  What kind of impact the scheme is having in relation to our particular country objectives for that country will determine how much we put into it, but also will determine, to a great extent, the commitments the Government will make for their investment in cash transfers.  As ICAI have shown in their report, Governments are increasing their commitments to cash transfers.  We know that the Government of Zambia increased their commitment to cash transfers by 800%.  We know in Kenya, as you said yourself, that the share they are now covering on cash transfers has grown to 60‑odd percent.  They are making more and more commitments.  Jonathan, did you want to add anything? 

Jonathan Hargreaves: I will say a little bit more about that.  Our strategy is based on some high‑level, consistent objectives around coverage, quality and, always, sustainability.  In fact, that was formalised in the Addis agreement on financing development, where everybody agreed to promote fiscally sustainable social protection systems, so that is very much the framework in which we do that. 

As Nick says, with that overall strategic umbrella, our strategy is then to look at the feasibility and appropriateness of the way to achieve that in each particular context.  We absolutely agree with the emphasis on financial sustainability, but clearly each different country has incredibly different challenges in terms of speed and scale and feasibility of taking over these sizes of programmes, depending on the fiscal space availability, on their capability and capacity, and indeed, on their own internal politics, as Dr Evans was saying earlier.

Our quite detailed value‑for‑money guidance, which we have produced over the years, is very, very clear that a central tenet of assessing the value for money when we do business cases for these programmes must profile and assess the issue of sustainability right up front, when we are attempting to use, support and promote national systems.  That is very, very clearly something that we require all our programmes to look at as an a priori factor.  However, of course, the pace and nature is very different in each case, and when we look at the individual cases, they are all, to some extent, a mix of technical assistance, systems strengthening, and cash. 

The balance, as ICAI pointed out, is very different in different cases.  That is quite often to do with some of the factors I have mentioned.  Quite often, we are trying to prove the concept initially, and that helps to make a political case.  In the Zambia case that Nick mentioned, Zambia are now at about 82% of the funding; they aim to have universal coverage in the next few years, and the politics behind that is when everybody—sometimes for the first time—has some connection with Government through these cash transfers, the political support can come in behind, but of course that is not necessarily possible to achieve right at the beginning.  In other countries, the politics of speed and scale is going to be very different.

It is a strategy to have that as an overarching and very consistent aim, but it is also a strategy to do it on a case‑by‑case basis, and ICAI have helpfully commended us on our ability to understand the political economy of each different case and respond to that appropriately.

Q34            Jeremy Lefroy: Thank you very much.  In the last Parliament, on a visit, we saw the beginnings of that Zambian scheme, which we were impressed by when we saw it.  Clearly, that is working and being scaled up.  I would tend to share, Mr Dyer, your slight scepticism about strategies.  It seems to me that we could do with fewer of them, they could be a lot shorter, and they could really focus on the fundamentals.  One of those, I would suggest—which actually seems to be the case—is that you would expect to see a substantial national government contribution to a scheme like this if the British taxpayer is being asked to put in that, and a few other significant points like that on one page would, to me, make a very good strategy that could then be updated regularly, as opposed to being on the shelf.  Maybe DFID could look at the length of its strategies and whether they could be much shorter and to‑the‑point. 

Chair: Mr Lefroy has taken the words out of my mouth, but I entirely agree.  Strategies are not looked at if they are in detail, but if they are concise, there is much more chance that they will actually be actioned.

Nick Dyer: Certainly, if we ever do get round to writing strategies, I very much agree: the shorter the better.

Q35            Chair: You say, “If we get round to”.  Whose responsibility would it be, then?  You called it “good guidance”; who is responsible for producing it, then, with regard to this issue?  At the end of the day, whose desk would it finish on? 

Nick Dyer: In terms of strategies, the DFID strategy is laid out in the multilateral development review and bilateral development review. 

Chair: I am talking about with regard to just this particular one.  Is it you? 

Nick Dyer: In terms of responsibilities for the guidance, ultimately it is me.  In fact, the head of the Social Protection team is Heather.  Heather runs the team that will be responsible for updating that guidance and ensuring that that community of practice is operating effectively. 

Q36            Chair: Thank you.  Very quickly, will you be updating it in light of this ICAI report, in particular, the amber-red section?

Heather Kindness: The guidance we have on this is actually the value‑for‑money guidance, and we have done two sets of that: one on value for money of social assistance programmes, and then, more recently, because we are starting to look more at social protection systems, we have a systems‑level value‑for‑money guidance.  In both sets of guidance, we talk about financial sustainability and good practice in terms of how to move towards that, so it is within those two sets of guidance.  The first set of value‑for‑money guidance was produced in 2011-12, in response to the NAO review and then the PAC hearing, looking at DFID’s cash assistance.  We were really pushed at that point to think about financial sustainability up‑front in every business case. 

Q37            Chair: Sorry, just to repeat my question, will you be updating your guidance in light of this report?  I just want a very short answer.

Heather Kindness: Not immediately.  They are very extensive documents.  DFID, at the moment, is going through a process of putting value‑for‑money guidance in place for every sector, and the two pieces of guidance we have are considered quite extensive and quite thorough already.  This issue is covered within it. 

Q38            Chair: Even though the building national cash transfer systems element of this report is amber-red?

Nick Dyer: The value‑for‑money guidance for social protection is 85 pages.  We will not rewrite all of it, but it is fair, in the light of this report, to go back and have another look on the TA side of this.

Chair: Thank you.  We look forward to that.

Q39            Jeremy Lefroy: Just coming back on that, at the risk of repeating myself, there are three or four major elements, such as what I would call national government buy‑in; you might call it financial sustainability.  We need to see it in that, together with, “Is it focusing on the poorest?”  Potentially, there should be other impacts on health, education, and job creation and so on.  That does not take long, but it would be great if every programme were to be assessed against those fairly fundamental points.  85 pages of guidance may be fine for the implementation of the programme, and very important—I do not deny that—but a very clear one‑page summary of what we expect to see in pretty much every programme of this nature would be very helpful for DFID staff, let alone for us.

Nick Dyer: You wanted to say something about technical assistance? 

Heather Kindness: On the sustainability side, as well, even though we agree that it is still amber-red and we are at an early stage with social protection systems, we think there is quite a good trajectory in terms of financial sustainability and the progress happening there.  We and others in the international community are looking at sustainability now.  An area where we probably have not done enough learning, and could give a bit more focus, is around capacity‑building—so helping to build the capacity of national and local government staff to manage and run these programmes.  As part of the sustainability story, that is an area where we are working with international partners and with national Governments to put in a bit more effort, so that would be part of our strategy and our approach.

Nick Dyer: It is a good idea, and we will take it away and reflect on that.

Chair: Thank you.  We have a lot more questions, and we are now under time pressure, so could I ask that perhaps questions and answers are kept succinct?  We have dealt with a really important issue there, because it does not just relate to this report.  Thank you for that, Mr Lefroy. 

Q40            Paul Scully: ICAI raised the concern that technical assistance is often fire‑fighting; it is looking at the short term, rather than taking the long‑term approach.  How are you planning on reducing the short‑term pressure on advisers to problem‑solve, so that they can focus on the long‑term capacity strengthening?

Heather Kindness: Our view is slightly different to ICAI’s on that.  The definition of technical assistance that we use includes long‑term and short‑term consultants, and some of the actual DFID advisers’ time in‑country, when they are working with partner Governments.  The type of TA that is given to programmes will very much depend on the stage that the programme is at: the maturity of the programme, the context, needs, what others are providing, and so on.  However, our sense is that the majority of the work that those technical assistants are doing is long term.  It is strategic; it is proactive; it is with a long‑term vision towards this greater coverage; it is more effective programmes; it is towards better quality and thinking about sustainability. 

Within that, problems and issues will arise, and there will be iterative problem‑solving, but our sense is that that happens but that is within a longer‑term strategy and a longer‑term view of where we should be going with that work.  A good example is the programme in Uganda.  When we started in Uganda in 2009, there was very little support for social protection and for cash transfers, so we put in two sets of TA.  One was around piloting a cash transfer programme, and there was TA to support implementation and getting that programme in place.  However, there was also very specific TA put in to look at policy influencing, and to build the capacity of national government counterparts to engage in policy dialogue, and then use the evidence coming out of the cash transfer programme to lobby again and influence a greater government commitment to social protection. 

Q41            Paul Scully: Was that DFID, or was that outside assistance, in Uganda?

Heather Kindness: DFID provided the funding for assistance that was provided through a—

Q42            Paul Scully: Provided from outside, yes.  The report also talked about the fact that DFID staff do not always have the technical expertise to monitor the performance of technical advisers.  Do you think you have a knowledge issue within your staff, the people overseeing it, and how do you plan to resolve it if you do?

Heather Kindness: On that, we absolutely agree with ICAI’s conclusion that we do not monitor and evaluate the effectiveness of our TAs robustly enough.  We do know that we have lots of examples of their performance impacting positively on programme performance, but we accept that it is an area where we have not done enough to monitor our TA well enough, and so in response to the recommendation—I think it was recommendation 4—we said we would do some work internally there, again using the community of practice that was mentioned, to lesson‑learn across our countries, look at approaches, and look at how we could monitor and learn on this better.

However, there is also a very effective international community working on social protection.  We were planning an internal piece of learning, but also a broader external piece to learn from others who are engaging and providing technical assistance to programs, too.  Obviously, also, we will be engaging with partner Governments and seeing from their perspective what is working, what they would like to see more of, and what we could do more of.

Q43            Paul Scully: Over what sort of timescales will you start seeing some positive changes from that learning process, then?

Heather Kindness: We are planning to do that process this year, in terms of learning and sharing experience.  In terms of putting that in place, we would bring those better monitoring practices into place as soon as we can.

Q44            Paul Scully: So this time next year, we might see some change.

Heather Kindness: At least we would have a much better understanding of good practice and how to do this, and be able to say that we have actually put that into our log frames and monitoring systems.

Nick Dyer: We do need to do this, because one of the messages coming out of the report is the challenge around learning on technical assistance and systems, so we will ensure that we do that.

Q45            Stephen Doughty: It is good to see you recognising the criticism that has been made there.  One of the points that ICAI make is that you are not actually collecting enough usable data to make the independent evaluations that take place worthwhile.  Can you just explain a little bit more about what you are doing to address this?  You mentioned the various examples, but they say that it is not being done systematically enough or clearly enough.  How are you making sure that those valuations are going to be worthwhile?

Heather Kindness: Yes, we are not independently evaluating them.  ICAI are absolutely right; there are not any, so this is where we are going to look at what we could do, whether it is an independent evaluation of technical assistance, or whether it is just more, better monitoring of the technical assistance within more regular monitoring systems or annual reviews.  That is where we need to do a bit of learning to see what is most appropriate and what we can do.

What we do know about technical assistance more generally is that it is quite a difficult area to measure, not just in social protection but more broadly, particularly around attribution.  Can you attribute some of the effects to the actual TA, or is it broader programmes or other people’s support to the programme?  That is where we need to understand a bit better how to do it well.

Q46            Stephen Doughty: Who are some of the people providing the technical assistance? 

Heather Kindness: It will be other big social protection funders and investors, or implementers.  The World Bank are another big player; UNICEF are another one, and then other bilateral donors, which include Germany, Sweden and Norway.

Q47            Stephen Doughty: Effectively, you are contracting in other donors and other partners to do the technical assistance?  Or do you get it on a reciprocal basis—pro bono work from Norway or the World Bank, or whatever?  How does that work in practice?

Heather Kindness: From the programmes that you will have seen, sometimes different donors are supporting the same programme, so different donors putting money in.  Sometimes there will be a co‑ordinated unit where everyone is funding the technical assistance.  That happens in Ethiopia, for example; there is a donor co‑ordination unit that everyone puts money in to support.  In other places, for example, one donor would provide particular technical assistance, and other donors may just put financial aid in.

Q48            Stephen Doughty: But this is not an area where you are using external private contractors at all to provide technical assistance.  Are you using any private contractors at all?

Heather Kindness: Yes, we will be in some places.

Q49            Stephen Doughty: Okay.  Given the wider investigations that the Committee are undertaking into that, and particularly given some of the criticisms that are being levelled at the moment at private contractors, it is therefore really essential that we get some really clear evidence of what they are doing and what value for money is.

Heather Kindness: Obviously, they are reporting, and we do have evidence of their performance.  What we are not particularly good at is linking the technical assistance.  We can see their performance through how the programmes are performing; that is where we measure it—much more in impact evaluations of the programme.

Q50            Stephen Doughty: But you cannot necessarily draw a line between a particular piece of advice and that improvement.

Heather Kindness: Yes, and we are not very good at documenting systematically that, because this technical assistant wrote that particular piece of advice or took that to the Government, it then resulted in this change. 

Nick Dyer: There is a question around line of sight and whether we do it.  We are not doing it; we need to do more of it.  Even that line-of-sight question is quite difficult, because of the attribution.  However, I do not think it is impossible.

Stephen Doughty: Thank you.  That is very helpful.

Q51            Chair: I have two questions about Bangladesh, and I have to ask you to be fairly brief.  We have talked about the fact that, for long‑term effects, cash transfers need to be accompanied by other interventions.  ICAI is positive about your Bangladesh pilot.  Would you like to very briefly tell us whether that is something that you are considering transferring to other countries?

Heather Kindness: The Bangladesh model, as was already explained, is a little bit different to the majority of our portfolio.  The programmes there are much more integrated livelihoods programmes that have a small cash transfer component, versus regular cash transfers that we do in most places.  As we have said, there has been a question over how transferrable they are, and so through an organisation called CGAP and also the Ford Foundation, they have been doing pilots in several countries.  We are a member and funder of CGAP, so we have been supporting that piloting and tracking the results, and it has been quite promising to date, but there are a few key questions around how transferrable that model is to other countries. 

One of the big questions is around cost, because cost is quite high per beneficiary.  There is a second one about how scalable the programmes are, and whether they could be implemented through national programmes, particularly in environments where capacity is a lot lower compared to very strong NGOs in Bangladesh.  There is a big third question: we know the whole model works very well, but which parts of the model are the most effective ones?  That is the bit we really want to unpack, because when we look at our regular cash transfers, what could we do on top and alongside those?  Are there fewer things at lower costs that we could do that would help move people out of poverty—cash plus other things?  Some countries are starting to test the full model, and other countries are starting looking at whether they can modify this model a little bit more, thinking about sustainable livelihoods, and testing some of those options.

Q52            Chair: My other question is not quite so positive, and it relates to the error in over‑reporting—we will call it that, after hearing ICAI’s evidence—of almost 500,000 people in Bangladesh.  It is also accompanied by a comment in the report that Bangladesh was the only country in which the office did not follow the approved methodology and have mechanisms in place to check data provided by their implementers.  What do you do to ensure that methodology is being followed?  How did this oversight happen, and how was it that ICAI discovered this, rather than DFID?  Half a million people is a lot, even if the claims were not actually made.

Nick Dyer: There are three main ways we do this to ensure consistency across the organisation.  The first is to have a methodology in the first place—so, we do have a methodology that we use to count these cash transfer impacts, which we have had peer‑reviewed and assessed externally, so we have a methodology that we can use.  Secondly, we do have a quarterly process of gathering the information that we want to aggregate at the central level, which we had for cash transfers in the last Parliament.  Thirdly, we have a series of statistical advisers, whose job is to ensure that we are following up and following the methodology.

Now, clearly, in this case, something went wrong.  As was explained, a single payment was counted but it was not in line with the methodology, so that slipped through the net.  I am reassured that ICAI were saying that, otherwise, the system seems to be working well, but when we do pick up these mistakes—and, most times, we pick them up before it gets to this level—we do correct them very quickly.

Q53            Chair: Why did this happen, and what have you done about it once you discovered it?

Nick Dyer: I can only speculate that in the process within Bangladesh, in terms of the people who were responsible for applying the methodology and checking it, it fell through the cracks.  We are not picking up more broadly that this is a broader problem within Bangladesh.  We have not seen that, in other circumstances, Bangladesh are making other mistakes in their application methodology for these accounting results, so my hope is that this is just a one‑off. 

Q54            Mr Lewis: It is clear from the report that the additional benefits of cash transfers are, to say the least, patchy, so it would be interesting to know two things.  First, where is the evidence that additional benefits actually have worked, so where is the best practice?  Secondly, what can you do more generally to ensure, if this is not working on the whole, that you can learn from that and improve the design of programmes in the future to make sure that additional benefits are being delivered?

Nick Dyer: Let me start, and maybe colleagues can add to it.  We do have good examples within the DFID portfolio of additional benefits—so, our work in Zambia has led to a 19% increase in the number of 15 to 18‑year‑old girls going to secondary school.  We have examples in Zimbabwe where cash transfers at the household level have had a discernible impact on early marriage and safe sex of girls in that household, so you do get additional impacts, and we have measured some of those additional impacts.

The issue—and this came out well in your conversation with ICAI—comes in the quality of the services and the extent of the services that already exist.  In a circumstance where, for instance, a lot of children are going to school already, you are not going to have much of a margin of impact on kids getting into school, but when they do get into school, if the quality of that learning is poor, they are not going to learn.  You might get them into school, but they are not going to learn. 

Similarly, in Ethiopia, with their cash transfer programme, the food security over a 12‑month period has gone up from 29% to 41% of the beneficiaries, but malnutrition has not moved.  What we are doing now is looking at how we can provide a complementary intervention, so that those children who are very malnourished who are part of the cash transfer programme can be part of the nutrition programme.  We are starting to think about how you can provide complementary services as a way of boosting the secondary impacts of the cash transfer.

Q55            Mr Lewis: Where there is not conditionality, are the recipients aware of the additional outcomes that the cash transfer is meant to deliver with them, or for them?  It is interesting; is it with them, or is it for them?  Are they aware?  Where it is conditional, it is clear: this is the contract between the recipient and the provider of the cash transfer.  However, where it is not conditional but there are examples of where you are seeking additional benefit, do the recipients know what those additional benefits are?  Is that explicit?

Heather Kindness: The principle of cash transfers being unconditional is that beneficiaries know far better than we do where their greatest needs are.

Mr Lewis: Of course.

Heather Kindness: In a sense, the primary objective we expect is poverty reduction, but some of the secondary benefits will very much depend.  There are some programmes that are more like that.  There are others where we see particular need alongside poverty, or food and security.  For example, we have a couple of very nutrition‑focused programmes, and so in those contexts, where malnutrition is very high, you can design the programme in a slightly different way and look at complementary interventions as well, to bring about those secondary benefits.  In those kinds of cases, yes, beneficiaries would be very clear what those secondary benefits are.

I would also say, just on the conditionality point, that there were all the reasons given for why conditionality sometimes is not a good idea, but what we are also beginning to see emerging evidence on—although it is still quite nascent—is that if you message with education, and message what the transfer is for alongside the transfer, we could possibly then see greater effects in certain areas.  That is an area we would want to look at a bit more in our programmes—a kind of nudge effect.

Q56            Mr Lewis: Yes.  When we sought to mainstream personal budgets in this country, in the context of social care, one of the things that was clear was that some people do not have the advocacy support, extra help, advice, information, guidance, etc, that is required to spend that money appropriately.  That is not being patronising; it is the reality.  Is there some work to be done around those issues, in terms of not simply giving individuals the cash, and not telling them how to spend their money either, but giving them that additional advocacy, information, guidance, etc?

Nick Dyer: Yes, and that is what we are doing in Ethiopia with this nutrition complementary intervention, because part of that is training the beneficiaries on nutrition knowledge and nutrition messages, as well as providing the nutrition programme.  Yes, there is more we can do to unlock those secondary benefits. 

However, one of the messages I take away from this whole literature is that the conditionality of cash transfers is quite contested.  It does not work in all cases, so an approach that says, “We should make it all conditional”, is one that we need to think quite carefully about.

Q57            Mr Lewis: My final point, very quickly, is that there is some evidence that cash transfers increase domestic abuse.  We all work to the principle of “do no harm” and on the basis that there is some evidence of that, are you going to begin any process of evaluating and monitoring whether this is widespread, or a growing problem?  Are there any plans to look at that specifically?

Nick Dyer: We need to be a little bit careful about generalising some of the very specific evidence, because the one example that came out of the literature is in relation to Mexico, whereas, actually, there are examples of the reverse where women are empowered by the cash transfer, and it is more of a benefit to them. 

Heather Kindness: I was going to say, similarly, that we take all risks very seriously in our programmes and at design stage, and carry on through implementation, and “do no harm to beneficiaries” is something we consider up‑front.  Our assessment across the portfolio is that this is quite a low risk, and that is what the evidence seems to be suggesting.  It is still a small size of evidence that we are drawing on, but the evidence seems to be pointing in the other direction.  There is also an issue, in that it is quite a difficult issue to measure.  It is very secretive, it is very sensitive, it is quite a hidden practice, and so we would be cautious about automatically saying we should be introducing this instantly in our impact evaluations. 

However, we do have plans relating to recommendation 3 on women and girls, and women’s empowerment.  We had already recognised that that was an area where we can and should do more, and so we were planning to introduce a programme looking at women and girls more broadly, looking at how to maximise benefits for them but also minimise any potential negative effects.  It is through that programme that we will look at this issue in a bit more depth, and get a better understanding of how high the risk is, and what we should do in our programmes to make sure we minimise it as far as possible.

Q58            Paul Scully: Do you think that DFID is too risk‑averse in addressing issues of national governance?

Nick Dyer: Again, this is the challenge from ICAI: that we are letting political sensitivity get in the way.  I think there was a confusion in their first draft, which was in relation to fraud, and so we would just repeat that we would categorically say that is not true.  However, in relation to sensitive issues around things like the Bangladesh example that they gave, which was around the system of targeting, we have a slightly different view of that, in that we believe that the way to change the targeting in Bangladesh was not to do a comprehensive review of the targeting in all the systems, but to make one or two specific proposals and changes to demonstrate that you can make a difference. 

This comes back to the question around understanding the political realities of the system that you are working in, and how you best advocate for change.  It may be that you want to look at a specific intervention, rather than generalise with intervention, to get the issue on the agenda.

Q59            Paul Scully: ICAI also talked about value for money, and their concern that you were using value for money effectively to say how well you were doing, rather than active management.  How do you think that you will use analysis of value for money to actively manage the projects, rather than just, as I say, approving how well you are doing?

Nick Dyer: The greatest assurance I can give you on that is the 82‑page value-for-money guidance that we produced for cash transfers following the NAO report, which does set out an explanation of the three elements of value for money: efficiency, effectiveness, and economy.  It does give benchmarks in terms of what are good value-for-money indicators or benchmarks, which people can follow. 

One of the areas of most opportunity in cash transfers is the whole area of electronic payments, in terms of value for money, because that can really reduce the leakages, and it can also reduce the costs of delivering.  There are some very good examples of new opportunities for value for money, such as in Tanzania and Zimbabwe, which we are actively working on now.  I think, in Rwanda, we have some terms of reference for an electronic payments scheme.  Everybody who works on the cash transfers has very strong guidance on the value-for-money question.

Q60            Chair: Page 33, the conclusions, says, “Across the portfolio, there are recurrent weaknesses in targeting, timeliness and payment size, which need to be more consistently and firmly addressed”.  What could DFID do to overcome these ongoing and long‑term problems regarding selection of, and targeting of, beneficiaries and late payments?

Heather Kindness: For us, these are the absolute core elements of any cash transfer programme: getting the cash to the right people, getting it on time so that they know it is coming, it is predictable, it is regular, and an adequate size.  There are issues around how big or small a cash transfer can be; there are constraints to that, but absolutely, those are the core.  We accept there are still weaknesses in programmes, but particularly when we look at our TA and our systems‑building support alongside the funding we put into transfers, it is where we put a lot of effort when we are helping establish programmes, strengthening systems and putting in additional money, or putting money into an existing programme.  We would put a lot of emphasis on that, in terms of improvement. 

There are definitely lots of examples of improvements having been made.  We recognise there is still a lot of work to do.  One of the things, as has been mentioned before, is that we have to accept the environments in which we are working, which are very challenging.  They are below capacity; some of these things will take time to get to the point where they are working very well, where they are very strong, but it is something that we do prioritise and we put ongoing effort into.

Q61            Chair: So, if ICAI come back to you—when they come back to you—regarding this report and how you have actioned some of these concerns, these issues will have been looked at carefully. 

Heather Kindness: They are areas that we can say we are looking at, definitely.

Q62            Chair: Can DFID build in better incentives for improved country performance?

Heather Kindness: Sorry, can you just explain the question?

Chair: Yes—incentives to ensure that in individual countries the performance is as good as it can be, and that the benefits are sustainable and long‑term.  Is there a way that you could incentivise schemes?

Nick Dyer: Yes.  Let me give you an example of a programme that was off‑track, which was our transfer programme in Ghana.  A couple of years ago, that was not performing as well as we wanted it to, so we had a very explicit conversation with the Government about how to improve performance.  We actually did have, in that case, time‑bound targets on reform, including transition to e‑payments—electronic payments—and making payments regular and predictable, and we do think that made a difference, because when we next went back to that scheme, it had improved.

This comes to the question of whether we should be introducing some kind of performance‑based funding for all these schemes, and as you know, DFID as an organisation is using performance‑based funding more regularly.  The challenge with performance‑based funding, as you also know, is that you are shifting all the risk onto the recipient, and in this case some very, very poor people.  If a Government misses a system change target, then you are very much punishing the people you are trying to help.  Now, that does not mean you should not do it, and within the broader context of DFID we are thinking about this more broadly, but we do have examples where we are very specifically using it.

Q63            Chair: So it might be something that you might consider, if best practice shows that it is something that could be shared.

Nick Dyer: Like all these things, if we can learn from it and demonstrate that it is working, we will want to think about using it more.

Q64            Chair: Thank you.  I just have a couple of further questions; one is very quick, because to an extent ICAI allayed concerns in their evidence this morning about the risks of fraud in cash transfer programmes.  They said that, actually, it was far less than in other areas.  However, is there anything that you are doing that you think could help mitigate these, however low they are?

Jonathan Hargreaves: I will respond to that.  It was, of course, excellent to hear ICAI categorically say that they had not found any evidence of fraud in our programmes.  That is, of course, what we would hope for, and our bottom line is that wherever there is any allegation of fraud or corruption, we act on it.  With the example that was used in Nigeria about the fake urine samples, which is raised in the report, our office responded to it immediately by putting in place a comprehensive fraud and corruption strategy, which randomly tested all of the recipients on a repeat basis.  There was also a community sensitisation in that, to make sure that everybody knew that that was happening, and that successfully weeded out 6% of the beneficiaries.  Sometimes, it was for reasons such as that people who had been eligible initially ceased to be eligible, and it was a question of rectifying those kinds of errors.  Wherever we find it, we deal with it, and that is our mantra.

Of course, we have very robust systems in place, ranging from the initial due diligence and risk assessment of our partners through to a lot of investment in targeting in the first place, transparency of the beneficiaries and the criteria, and the financial flows.  We invest a lot in management information systems that enable us, the Government and citizens to see what is happening within the programmes, and of course we have elaborate monitoring evaluation procedures and internal and external audits.  Our systems are robust.

Nick has referred to one interesting area around electronic payment systems, where there is some really interesting evidence of them being used more across the world.  For example, in Uganda, they have a system of a SIM card with an identity and a PIN, and oversight and third‑party monitoring of the agents distributing the components.  There are many other examples emerging across the world of using electronic payments, which we think are promising.  They are not a one‑size‑fits‑all, because they depend on countries having decent IT systems and decent communication across the country, a network of agents that are able to apply these systems, and also the financial capacity to be able to invest in them and maintain them.  However, where they are feasible—and we have seen them, as I say, used in Uganda, Kenya, Tanzania, and other places—they are a very promising area that we can certainly look at more and try to bring the collective evidence to bear across the world.

Q65            Chair: This is the last question from me, and then Mr Doughty wants to come in with our closing question.  This is a very positive report about cash transfers, and it is positive about DFID’s work as well.  However, it is in contrast to some really quite negative media reporting at the start of the year about cash transfers, and so I raise a question that our Committee has now been asking for some years: what can DFID do to communicate some of its really good work, in this instance with regard to the effectiveness of cash transfers, so that the British public—the taxpayer who is paying for this—really can be convinced that this is a good thing to do, and counter this negative publicity?  What can you do, and could you do more?

Jonathan Hargreaves: Thank you for the question.  Indeed, there has been widespread commentary on this report and the issues around it.  Generally, we welcome that debate and the scrutiny that it brings, and so to some extent, all publicity is interesting publicity when it comes to a debate on this kind of issue.

However, around the time of this publicity around the beginning of the year, of course, our media team was extremely active in briefing the journalists concerned and knocking down factual errors and inaccuracies.  We were grateful for ICAI clarifying their wording and sending a rectified report.  We now have a gov.uk page dedicated to DFID’s point of view on stories in the media, including this one, which enables us to showcase case studies and to put facts online, on the record, so that people seeking to know what we perceive as the truth behind these stories can go to both those good case studies and the facts.  It is also very helpful both when independent reviewers such as ICAI, NAO and PAC are able to put things into the public domain, and when other knowledgeable organisations such as NGOs and research organisations participate in the debate, as they have done very actively in this case. 

Generally, as you know, we are very actively trying to communicate much more meaningfully and simply what the real‑world benefits are of all the work we do, including, of course, the work in this area.  This is an area that lends itself to some fantastic evidence and stories of real‑world impact for the very poorest people in the world, so it is something that we would want to continue to do more of.

Chair: Thank you.  We do welcome that.  In that regard, can I suggest that the Department considers how it could communicate some of this good news to Members of Parliament, for us to then be able to have that?  Other Departments do that.  So many colleagues, as you know, do believe in UK aid, but to have that to be able to communicate to our constituents would be very helpful.

Q66            Stephen Doughty: Just to follow on from that, I have looked at a lot of media coverage around this issue, including the wild claims made about exporting the dole, which just does not bear any resemblance to the facts, or, indeed, the claims about fraud.  There are really robust responses from the DFID media team around this, which clearly reflect the briefing that was put in to journalists—very factual rebuttals of it.  That is great, but I struggle to find examples of where Ministers have been responding to these claims.  Can you just explain whether Ministers have responded on this issue, or whether it has always been a press officer, and if not, why you think that is the case?

Nick Dyer: I would need to go away and check, but of course press officers are speaking on behalf of the Department, and are reflecting the views of Ministers.  Even if Ministers are not speaking directly, the Department’s view is being reflected. 

Jonathan Hargreaves: It is also worth remembering that it was the Prime Minister’s spokesperson who responded in early January, and put on record the Prime Minister’s own support for the methodology of the use of cash transfers.  We have taken our cue from that, to some extent.

Q67            Stephen Doughty: Just in terms, again, of the ease of explaining some of these issues and the example you gave earlier on of the relative effectiveness of, say, a food handout versus a cash transfer, these things are very easy to attack, but do you think you could make more use of those very simple contrasts?  I can think of a constituent coming up to me and saying, “But surely handing out food is better than handing out cash?” and when you have to explain that actually it is not, that is quite a difficult thing to get across.  Do you think the Department could be making better use of more straightforward, common‑sense examples?

Nick Dyer: I would say that we do.  The challenge is the cut‑through of that, and as we all know with communications—and I am sure every member of the Committee knows better than we do—you just have to keep repeating the messages consistently for people to actually start hearing them.  What does the evidence tell you?  That you have to say something 12 times before somebody will actually hear it.  We might say it once, but then it gets lost, and there is an issue about not only simplicity but repetition, and how you get a simple set of really clear messages and then drive those.  That is something of a challenge for all of us.

Chair: Thank you very much for your evidence today; it has been extremely helpful.  Thank you to all our witnesses, and good morning.