International Development Committee

Oral evidence: Sub-Committee on the Work of the Independent Commission for Aid Impact Reports- How DFID Learns; and Private Sector Development Work, HC 652
Wednesday 10 September 2014

Ordered by the House of Commons to be published on 10 September 2014.

Watch the meeting Wednesday 10 September

Members present: Fabian Hamilton (Chair); Fiona Bruce; Sir Tony Cunningham; Jeremy Lefroy 

Questions 1-46

Witnesses: Graham Ward CBE, Chief Commissioner, Diana Good, Lead Commissioner for both evaluations, ICAI, Peter Davis, Team Leader, PSD, ICAI and Nigel Thornton, Team Leader, Learning, ICAI gave evidence 

Q1   Chair: Chair: Good morning, Peter, Graham, Diana and Nigel.  Welcome.  I apologise for the lack of members today, but you will be aware there is a certain referendum going on in Scotland and some of our Scots members are there, not many of our English members.  That is why it is slightly reduced today.  We hope to finish by 11.30, because members want to go into the Chamber, so let me just emphasise that not everybody needs to answer every question.  Although we want to hear your full responses and replies, we need to get through the questions as well.  Thank you very much.  Before we start, is there anything, Graham, or any of you want to say by way of introduction?

Graham Ward: I have nothing to add as a general introduction, Chairman. 

Q2   Chair: Thank you.  We have just published our Report on ICAI’s Annual Report 2013‑14—I am sure you have seen it—in which we expressed concern about ICAI’s transitional arrangements.  I wondered whether you have anything to say in response to that report. 

Graham Ward: Yes, Chairman.  Thank you very much.  First of all, may I say we welcome the report? We welcome the emphasis that you have given in that report to our continuing independence, and we believe that our continuing independence, at what I would view as the highest level we have at the moment, is essential to ICAI being able to provide proper support to the Sub‑Committee, your parent Committee and the Department, and indeed to producing the right results for the world’s poor, so we are completely behind you on that.

The transitional arrangements are still, to some extent, in flux.  We are concerned that there should be proper continuity, so that there is a proper handover period between the commissioners and incoming commissioners, and that we have continuity arrangements as far as the contractor is concerned.  A new contractor, of course, may or may not have people in common with the existing one, but that is a proper process of procurement that has to take place in the coming months.  We are also concerned that our staff are able to have a proper degree of continuity.  Otherwise, there is a danger of a hiatus that will result in a loss of service to you and your successors in the new parliamentary period. 

We have debated before the question of the location of ICAI in the new world, and you comment on that in your report.  Since we last gave evidence to the Committee, I have had a conversation with the Secretary of State, and the outcome of that conversation is that she also would be concerned if ICAI was to be located long term in DFID’s offices and believes that ICAI should have its own independent premises.  The current position is that there would only be a period of ICAI being located in 22 Whitehall if that proved to be absolutely necessary because we were thrown out of our existing premises and had too short notice to enable new premises to be found.  That really would be as short a period as possible, and everyone hopes that it will not happen at all.

Chair: That is a very encouraging development.  I am sure the current commissioners and you are very pleased that at least we have made some progress in that area.

Graham Ward: We are, Chairman.  The other thing I would say is we want to make sure over the transition period that we do have a staff complement that is sufficient to enable us to meet your specific requests, in particular to enable others to share in the conclusions and results of the work that we produce, and we are still in discussion with DFID about making sure that we have sufficient resource to enable that to happen.  We have made proposals that we believe mean that we would be able to provide an enhanced service to you at lower cost overall in the future, and we hope that that will be comprehensively signed up to by DFID.

Q3   Jeremy Lefroy: In the report How DFID Learns, some people, in responding to it, said that you are being over‑critical.  Have you considered other donors and how they learn?  Is DFID any worse than that?  Have you carried out any kind of comparative analysis and is this something that perhaps you think you should be doing within ICAI evaluations?

Graham Ward: It is always a fine line as to where we come in terms of the markings that we give, and we consider them very carefully indeed.  We do not believe that we were overly critical in respect of this report.  We have recognised, and indeed explicitly praised, the commitment of individuals within DFID to learning, which in many cases is extremely strong, but our criticism is centred on the lack of cohesion and coherence in terms of learning, and so it is about the way the organisation as a whole learns, rather than the way in which individuals are involved. 

Our practice has never been to try to do a poll of how other aid providers provide their aid and try to rate DFID within that.  It would be an enormous exercise, apart from anything else, to try to do it.  What we have always looked at is whether the way in which DFID is helping to meet the needs of the world’s poor is the most effective and efficient, and whether we believe that they could do better.  We are not doing a sort of popularity contest or a poll, in relation to the work that we carry out. 

Diana Good: I would just add that our findings were based on a number of interviews—92 interviews with DFID staff at all levels.  They were based on seven internal surveys that DFID had carried out, and what we heard is a great deal of frustration from DFID staff, in relation to the failures of organisational learning, an ability to learn across the global reach of DFID, and of course the findings that we put in our report in relation to staff feeling that they were under pressure to produce good results.  While there are statements about welcoming openness, in reality they feel that they cannot be as open about the things that are going wrong.

We were looking at the evidence that came from DFID themselves—from a very large number of their staff, when we combined the triangulation of our interviews with the huge number of people who had been involved in the internal surveys.  Plus, we based it on the 31 reports that we had done to date, in which we had noted the very great inconsistency in DFID’s capacity to learn.  We have seen extremely good examples of that and extremely good leadership in that respect.  Chris Whitty, who I believe is going to answer your questions later, exemplifies the kind of thing that we saw where, rather than putting in new systems and new reports, he has actually changed the culture in his department very effectively.  We have seen very good examples of that.  We saw an extremely good example of the Learning Partnership within the Civil Society department.  We have seen extremely good examples in-country. 

The problem is the fact that, because DFID is focusing on the learning of individuals, rather than getting the systems and the culture in place to encourage learning across the entirety of DFID, one sees some very bad examples, and a lack of consistency in sharing the lessons that need to be learned, both from the bad examples and the good examples.  We are looking at an organisation that wishes to be a leader in this area, and absolutely could be—we want to emphasise that—without huge additional systems, reporting and IT. |If they shifted the culture, they could get this really right.  Given that they are such a large commissioning agency and they work through multilaterals, through NGOs and through contractors, they could actually improve the learning of others in this sector by improving their own learning and encouraging others to do the same, and listening to them. 

Q4   Jeremy Lefroy: I have two very quick follow‑ups on that: this Committee has, on a number of occasions over the past four years, said that we believe that DFID should do more about learning from what it does around the world.  I remember saying so myself and I think colleagues have said the same.  Have you noticed any recent response to the points that we have been making about learning from work around the world?  Secondly, do you think there is any impact from the fact that DFID is increasingly commissioning its work from outside, as opposed to doing it in house?  Therefore, what a lot of DFID staff spend a lot of time doing is managing contracts, rather than actually—I am not saying that managing contracts is not important work—doing the work from which things could be learned. 

Diana Good: In terms of whether we have seen progress and learning from the comments that have been made, we have made a number of references in the report to things that we note them doing and praise them for: by way of example, the amount of research they commission.  That has been made publicly available.  As we have said in our report, we do not think that publishing research constitutes learning unless it turns into action, and that is the piece that we see missing.

Q5   Jeremy Lefroy: If I may just slightly interrupt—forgive me—what we are looking for is whether you have seen leadership in DFID saying, “This is a priority.  We must do this.  We recognise we can do better”?

Diana Good: We have seen statements from leaders within DFID saying, “We must make this a priority,” and we have seen very good examples of heads of country office who do this very well.  I mentioned earlier Chris Whitty’s role, which we specifically praise in the report.  Where there appears to be a disconnect is that, in reality, so many of the staff themselves feel frustrated; so many of them feel that they cannot be as open as they want to be; and so many know that they learn best from experience, but feel that their experience in the field, their encouragement to learn in action and from action is not what it should be on a consistent basis. 

Now, quite clearly, DFID is taking our report very seriously indeed.  The management response does not reflect at all, as we understand it, the amount of work that they are doing, and no doubt you will ask them about that.  We have not received a formal update on what in practice they are doing.  I think that, at every turn—this is very much the theme in our report—real learning is not the collection of information and putting it on IT systems; it is not creating more reports; and it is not complicating or spending more money on it.  It actually means learning in action. 

For instance, we have seen the smart rules, which are the end result of the end‑to‑end review they carried out.  It is 100 pages long and, with your approval, we propose doing a rapid review of the smart rules in the near future, which we think is merited.  We think that, as those 100 pages are read, there appears to be a missed opportunity in terms of learning.  The word “learning” is referred to a few times in these 100 pages, but the only section on it appears in the last 16 lines of the report.  We think this is just inadequate. 

It speaks volumes that we have been saying, and I believe you have been saying, that what DFID needs to do is learn from action.  It needs to go for adaptive programming.  It needs to learn during implementation.  It needs to work out, from cradle to grave, whether there is evidence for the plans that they have at the beginning; whether the plans work in practice; and whether course adaptation is needed throughout the process, rather than waiting till the end, and then maybe years later having a more scientific evaluation done.  I am not saying there is not value in that, but one actually needs course correction during the course of implementation, and that means observing what is happening and it means learning and listening to what is happening from, as you rightly say, the people they are commissioning—from the NGOs, from the contractors. 

We have also repeatedly heard from them, during the course of many of our reports, that they feel that DFID is not in listening mode enough, and therefore they get set off on a course that they know can be the wrong direction, but there is no culture that enables them to say to DFID, in plain terms, “This is pointing in the wrong direction.”  We need to listen to what the beneficiaries are saying.  We need to listen to what is happening in the field, and change it.

Q6   Jeremy Lefroy: Just a comment about commissioning itself, and whether that actually distracts from learning—the fact that so much time is spent on the commissioning. 

Diana Good: A great deal of time is spent on commissioning.  A great deal of time is spent on getting the business case right, and there is a sense, when one is observing these things out in the field, of a kind of exhaustion all round by the time the business case process has been gone around, and a sense that, in too many cases, the DFID staff then let the programme roll out and look to do an evaluation at a later stage, without actually engaging much more—whether you call it oversight, whether you call it listening and learning—with what is happening in practice. 

Nigel Thornton:  One of the things you have seen in the last five to six years is a shift internally in the culture of the organisation.  It is exactly what you say: DFID has externalised a lot of knowledge and a lot of experience.  One of the things that comes out very strongly through our work is that that split has become clearer.  The people inside the organisation are utterly dependent on those outside.  What was coming through, particularly from interviews with the senior staff, is that there is a sensitivity to that, but there is also a feeling that it is very hard to bring that knowledge back in, given the challenges that there are on the staff for managing the money.  I think your general thesis is absolutely right. 

Graham Ward: If I may just add, we have seen in a number of our reports instances where the people who have been commissioned have said, “Actually, there are things that we would like to share and we wish to share, so that the knowledge does not just sit with us.”  It is proving very difficult to enable that to happen.  It is not embedded; it appears in the day‑to‑day business of DFID that they will go out seeking this knowledge and seeking to learn from it in practice and in action, as Diana said. 

Q7   Jeremy Lefroy: Briefly, talking about failure as well as success, you say that staff sometimes use selective evidence to justify spending or support political priorities, which apparently is often driven by managers requiring support for decisions.  You say it is not a common practice, but it is happening with sufficient regularity to be a concern.  Obviously that greatly concerns us.  Could you expand on that, please, and perhaps give an example?

Diana Good: As I said earlier on, there is a great deal of evidence, both from their own internal surveys and from all the interviews we conducted, of staff feeling under pressure to produce results.  It manifests itself in a number of ways.  On the nutrition report, by way of example, we were concerned that, while there was a great deal of information available on the ground as to what was and was not working, DFID central was asking all the country offices to produce reach figures.  In order to state publicly that a mother and child had been reached, a mother only had to come into a health centre once for one intervention for it to be said that that mother and child had been reached, in terms of UK aid and development assistance.  In fact, that mother and child might never come back again and the child might die the next day. 

That is an indication of the centre wanting to produce figures that demonstrate considerable reach, making country offices produce information that actually does not reflect, in practice, what they are doing and what they are experiencing, and where a real difference could be made if one actually identified who is being assisted. 

We have seen other examples—and obviously you have looked at this before—in which the information that came through from the TMSA team simply did not reflect, in reality, what was or was not being done on the ground.  That then became a good story that was told right round DFID.  It became known as a flagship, but it did not actually reflect the reality.  I do not know if there are other examples you would like to refer to. 

Nigel Thornton: Perhaps another way of answering it is to look at what the internal survey material says from DFID itself.  The report refers to one particular survey, which was done by DFID, and we had 34 comments from that, from 34 different individuals, where they cited evidence that, if you like, their messages had been filtered.  There are a couple of things around that.  One is the issue about fear of failure, which you have mentioned.  There is also the positive bias that a focus on results brings in.  It is not the dominant culture, but it was a significant percentage of the respondents to this particular survey, and then we followed it up with interviews as well to indicate that is a pattern of behaviour that continues in some parts of the organisation. 

What was interesting about that is also the response of civil servants who were saying, “You can’t say that.  You can’t send that message further up the chain, because it won’t be acceptable.”  That was the thing that worried us most—that that filtering of information was going on—because it was perhaps not going to be acceptable to senior management or to Ministers to hear.  If people are afraid, if there is that fear of failure and if there is a positive bias, you are going to get that.  I think it would be fair to say that is not the dominant culture in the organisation, but it is a significant one that needs to be dealt with.

Diana Good: It is interesting to note Stefan Dercon’s reflections on the What Works review of DFID.  That is the Cabinet Office review.  At the end, he emphasises the need for leaders to be consistent, to demonstrate the right behaviour and learning, and use evidence.  The last sentence we think is instructive; he says, “Seeing senior leaders experimenting with what works, communicating what they have learned and admitting to failures would be very powerful.”  That “would be”, the conditional, is very important, because it recognises that it is not a culture in the organisation yet, and it is indicative that some of the messages we have been communicated are being said loudly internally, and they need to do something about it.

Q8   Fiona Bruce: Diana, you have gone into a great deal of detail about staff capacity and turnover and so forth.  Can I ask you to home in on staff turnover in fragile states, in particular for example in Afghanistan, where half of DFID’s staff are replaced every year?  We commented on this as a Committee and recommended the creation of a cadre of experts with knowledge, but DFID has rejected this.  Could you just comment on that and say how DFID can ensure that, in fragile states, capacity is retained? 

Diana Good: It is a very real problem, as you rightly identify.  Of course, given the nature of the countries in which they are operating, obviously the staff do need a break and a turnover.  That means all the more effort needs to be made to ensure continuity of learning and that it does not just stay in the head of the individual.  It has happened in many of the countries we have been to visit, and the reports we have done, that the person who knows everything about that particular programme is now somewhere else, or is on their break in between countries, and is not available.  It becomes more and more apparent that the fundamental stuff of organisational learning, from exit interviews to documentation to leadership on that, all too often is lacking.  It is a fundamental problem.

They have increased the number of their cadre advisers.  It has gone, since you made your recommendation that it should be increased, from 30 to 80, but our analysis—as you will have seen from the report—is that the vast majority of those people really are very junior and have had very limited experience in private-sector work, and even more limited experience at any senior level within the private sector.  That is taking us on to the private sector, but it is an example of where, oddly enough in the fragile states, the private sector is often one of the areas that thrives most. 

One of the things you will see in our fragile states report is that, actually, in Somalia, the private sector is one of the areas that is thriving, so that would suggest that one does need more private-sector cadre experience.  I think that it is a problem that we see also in non‑fragile states.  Where there are local staff in office, they are the ones who often are the continuity, and all too often—and we have referred to it in the report—we do not see local staff being listened to enough.  I have actually been in country where we have sat with everybody in the office in the boardroom, and it is all the non‑local staff who sit round the central table and the local staff all sit round the edge of the wall. I am not saying that happens consistently, but I have seen it happen and it has made me feel very uncomfortable.

Chair: I am not surprised. 

Q9   Sir Tony Cunningham: Were you surprised to hear that DFID does not know exactly how much it spends on evaluation?  You managed successfully to ascertain that DFID has committed, from central budgets alone, over £200 million to fund evaluation.  Did this strike you as excessive?  Finally, what do you perceive to be the key differences between your own work and DFID’s internal evaluation?

Diana Good: It was a significant exercise for us, mapping and trying to piece together how much DFID actually spends, because, as you note, they do not know themselves how much they spend. 

Q10   Sir Tony Cunningham: Did that come as a surprise?

Diana Good: Yes, it does come as a surprise.  Actually, it is something that we have seen in other areas as well in the private sector, which we are going to come on—they do not know how much they spend.  It seems to me that that is something that they ought to have a grip on.  What concerns me more than that is whether or not that expenditure is being used to turn into what we call learning knowhow, and synthesised, digestible, usable material for staff to use, in terms of planning and programming.  That is where the issue lies.  As we have said in our report, we only actually found one instance, and that was within the Civil Society department, of a department actually making an effort to set out the relationship between learning and results and performance.  That was part of the learning partnership. 

Sir Tony Cunningham: Only one?

Diana Good: Yes.  I do not think that we can comment on whether or not, whatever the actual expenditure is, it is too much or too little.  We are not commenting on that.  What we are saying is, if you are going to spend that much or however much you spend, you really ought to be using that evaluation work to ensure that it does translate through into learning and action, rather than sit on a shelf or sit on a rather inaccessible IT system.  That, to us, seems to be the issue; it is what really useful purpose is the evaluation expenditure serving, if it does not translate through into lessons that can be learned, preferably in real‑time but also, secondly, in a digestible and usable form across the board.

Then you ask us in terms of our work.  We are a very tiny part of the total budget.  We do not do long‑term academic post facto evaluations.  We are more looking at what is actually happening at the moment, and is it moving in the right course of direction and is it achieving the impact that is intended?  We believe that that does bring very real value to bear because, in so many instances often, when we have looked at programming, it is those programmes where people have stopped and reflected part‑way through and taken rapid action to do something about it where one can see the best results and impact. 

We referred previously to education in Nigeria.  There we were very critical of the UNICEF programming.  It took DFID a year to do an annual review and then put that programme on a PIPA project—improvement, performance.  Even now, the situation in Nigeria has not been resolved. 

The problem for us is that, where there is a contrast between actually doing an annual review or a post‑action review—which are things that we believe should be done as a matter of standard course, including listening to the contractors, the NGOs and indeed the beneficiaries themselves—action could be taken to ensure that the poorest people are getting benefit from the expenditure, rather than spending a great deal on an evaluation too late in the day to do anything about it, and for that then not to be utilised to create a real pattern of learning across the entire global reach of DFID.

Nigel Thornton: Can I just make two observations?  One is that a recent study was made of the World Bank’s evaluations, and it was found that 30% of all the evaluations that were then put online were never read—zero.  They can find that out because they can tell whether it was downloaded or not.  One of the interesting things about the difference in methodology between the ICAI methodology and DFID’s own evaluation methodology is that we know what actions are taken as a result of our reviews, because we follow them up.  You make sure that DFID is held to account for them, and we track that.  If you ask DFID what impact its evaluations have on what it does, it cannot tell you that, because it does not map that process.  Now, that is significant.  You asked for the difference between the two evaluations.  I think DFID wants to do that, and the new head of evaluations is thinking along those lines, but you asked for a difference and that is the key difference.

Chair: Can we move on now to DFID’s private sector development work?  I know that, Peter, you are the team leader on this and, Diana, you are the lead commissioner. 

Q11   Fiona Bruce: DFID plans to spend £1.8 billion on economic development by 2015, and this is a more than doubling of the 2012‑13 expenditure, and yet your report on DFID’s private-sector work is actually quite critical.  It is amber/red.  I have noticed you acknowledge that there are aspects where DFID does not have expertise.  Certain micro‑level projects are good, but there is a lack of a consistent portfolio of programmes overall.  There is perhaps a lack of clear acknowledgement of the scale and complexity of the challenge, and so forth.  Which of the reports’ findings were you particularly concerned by?

Graham Ward: I would kick off by saying that, to us, the most important concern relates to the availability of sufficient people with the right degree of experience and expertise to be able to advise on life in the private sector, and therefore on the actions that DFID best can take in order to stimulate the private sector operating well to the benefit of the poor in developing countries.  That really is the key to it.

Q12   Fiona Bruce: Just on that point, is thought about recruitment lateral enough?

Graham Ward: I suppose, looking at the outcome or the general lack of outcome at the moment—and I am not being critical of any particular individual in saying this—the answer must be no.  If the answer were yes, then we would actually see more people with the right degree of experience and expertise in post.

Diana Good: At the heart of our concerns is that what DFID needs to do with this very ambitious and very important development goal is to inject much more realism and focus, in terms of what they do.  In 2006, you recommended that they should produce a strategy.  Their ambitions are set very high indeed in terms of the theory of what private-sector development will do.  At the top end, you have got the ambition to transform economies.  At the more micro level, you have the theory of letting a thousand flowers bloom, and we certainly saw a lot of flowers blooming in terms of the micro‑work in microfinance, and the mid‑work in terms of markets for the poor. 

What we did not see was a coherent approach to how to make the thousand flowers bloom and the huge scale of ambition, and the assumptions that are made there, coherently add up.  We saw, probably as a result, a lack of clear guidance for those in-country, as to how they are meant to make this happen.  Combine with that the very special problems with developing the private sector.  DFID made the statement in 2011 that they wanted to help private enterprise work its miracles as the engine of development, but the private sector will do its thing anyhow. 

The issue for DFID is that, as an aid and development agency, what difference can they make in this space, when there is so much that is out of their control—what governments do, what economies do, what markets do, what the private sector itself does.  Combine with that the very lack of experience that we very much share your concern on.  Actually, DFID needs to focus and come up with something that is much more coherent and realistic, and simplify the scale of the ambition, so that it turns into something that makes the kind of difference that they are aiming for.

Peter Davis: Can I add one point there?  In a sense, it is the juxtaposition of these two things.  To come to your particular point, DFID is always going to struggle hiring people who would otherwise want to be in the private sector.  People who want to go into the private sector will go into the private sector.  DFID has to juxtapose what it can do with the people it has.  Rather than say, “Oh, that is a problem,” say, “What does that mean that we can do?”  As Diana pointed out, the creation of a thriving private sector is the work of many different entities coming together.  If DFID could focus much more on what is it that we can do that no one else can do, or others are not in a position to do— given the people we have, given the other capacities and capabilities that a government entity has—to ensure that private sectors develop in such a way that they are as beneficial as possible to the lives of the poor. 

Q13   Fiona Bruce: I am interested that you say people who want to go into the private sector will go into it.  You are quite right, but there are many people who, in a mature stage in their working life, will have been in the private sector.  It seems interesting to me that, while many of those people may then want to look at fresh challenges, what DFID appears to lack is high‑level expertise of managing private-sector projects.  Perhaps this is where we need to focus.

Diana Good: They are very different worlds.  One of the comments I heard when I was out in-country working on this report, from a businessman who is working on a programme funded by DFID, was that they find me very difficult because I am too business-like and I am too oriented to doing it now.  He said their processes are so cumbersome and so bureaucratic.  They are just different worlds.

Q14   Fiona Bruce: We all agree that the aim is worthy and also necessary, if countries are going to reduce aid dependency.  I do wonder whether DFID should be making such a large investment of this kind, at this stage, when it perhaps does not have the expertise, where you have highlighted so many concerns and where you have not highlighted a comparative advantage that it is offering.  Should there be, perhaps, a rethink about quite the pace at which it is hoping to move?  If it addresses the points that you have here, do you think this level of growth in its expenditure can be justified? 

Diana Good: It poses a very real question for the people in country offices who have to try to make this work.  We did hear repeatedly, “We cannot do everything,” and I think they feel that they are rather being asked to do everything.  When we asked offices to produce their strategy for this, in varying degrees what we were shown was a list of programmes, not a strategy as to how these added together.  It is because they are presented with an extremely difficult task, without the experience and the expertise to make this hang together. 

We certainly saw some very good microfinance programme work.  We saw some very good work encouraging farmers to increase their yield with better agricultural techniques and good markets for the poor.  Maybe they could achieve far more by really looking at how you can scale that up and join it up in a coherent fashion than by trying to transform economies in some of the most difficult countries in the world to do just that. 

Q15   Jeremy Lefroy: Following on from that, you point out that DFID has committed to spend 30% of its budget in fragile countries and conflict‑affected states, and also therefore highlight the need for a specific strategy for private-sector development in those states.  What sort of private-sector development do you think DFID can try to achieve in those countries?

Graham Ward: At the moment, we are embroiled in an in‑depth study on fragile states and scaling up in fragile states.  If we go back to what Diana was saying about the Horn of Africa, it is interesting that there the initiative, the drive, of the private sector is producing results and is producing an economy, despite the failures of the institutions.  One of the big things that government can do in developing countries, and arguably in developed countries as well, is to create the conditions for private investment to thrive, in terms of regulation, facilitating access to capital, the right sort of legal and regulatory environment for people and employment, and looking for gaps in the availability of the right conditions and helping to stimulate filling those gaps.  The private sector is given the best possible chance of doing what it does best. By definition, the role of government is not actually to run private-sector businesses itself.  Those are the directions that we would see it should go. 

Peter Davis: I think that is exactly right.  On the back of that, one of the countries that we visited was Bangladesh, which is on some donor agencies’ list of fragile states.  If you look there, you have to work with what is there.  For example, we saw an example of DFID working with Lal Teer Seed to help develop seed production that was able to allow small‑scale farmers to grow horticultural crops.  It is a question also of being very focused on working with what is there within that existing industrial level, rather than going back to this broader point about the need to focus—working at that macro level, but also working with industrial sectors that are there and seeing what you can do.  There is a lot of focus that could be made in that regard.

Q16   Jeremy Lefroy: You say in the report that DFID needs to give clear guidance to develop balanced country portfolios and projects, something which I entirely endorse.  How do you think DFID should go about this?  I fully agree with the conclusion that you have reached that that does not exist or does not exist fully at the moment.

Peter Davis: It is several levels.  The first thing that we did not see that we would like to have seen is some form of analysis of what the problem with the private sector is here that we are trying to solve.  At the top level, DFID says it is trying to transform the private sector to benefit the poor, so where is that analysis?  That analysis needs to happen.  This comes back to our point about providing training and support to in‑country staff: there needs to be a process of helping them to understand how different specific interventions can be drawn together. 

In our report, on page 15, we cite an example of how that might work, specifically looking at the case of land in Tanzania.  It is having an analysis of what the problem is that we are trying to solve, and obviously, laterally, which other agencies and which other entities are doing what else, then being able to develop a series of programmes that collect as a portfolio, and then managing those collectively, as you move forward—re‑engineering those and re‑orientating those as things move forward.  It is a mixture of analysis, but then also the skills and capabilities within the programmes and how those are managed together, at a country level.

Q17   Jeremy Lefroy: Given that private-sector development depends absolutely upon the government of that particular country, its policies and its desire to do that, have you noted any strong engagement between DFID and the countries so that the government and parliamentarians say, “These are the things that we want to concentrate on.  This is our programme for the future.  Please fit in with this,” or does DFID tend to go off on its own?

Diana Good: DFID does make a very real effort to work with governments, and I think that it is very challenging.  By way of example, in Tanzania, where they have a functioning fully elected government, there are so many different ministries, all of them with overlapping responsibilities and a degree of competition between them.  Actually, saying that one is going to work with government is difficult even when there are willing participants and there are politics at large.  As we heard very loud and clear, one part of government can do something that completely undermines what the other part of government is seeking to do. 

We heard a particular example of that, where a lot of DFID money is going into assisting a major rice farming programme in the central corridor, and out of the blue a different ministry lifted the import duty that rice producers from Asia normally have to pay, which completely undercut the production of domestic rice growing within the country and was a disaster for them.  That is just an illustration of how, however much one works with one part of government, another part of government might operate very differently. 

We felt that DFID could operate more as a trusted adviser and speak more plainly with governments than they do.  It is easier said than done, and we would not pretend that that was an easy activity.  However, it comes back to the learning point, which is that DFID has, if it really got its arms around it, an enormous amount of learning from around the world.  If it could turn to governments and ministries and say, “We have seen this going wrong or going right in other countries; let us share that with you and tell you in frank terms what does and does not work,” then I think there might be a better means of engagement with those governments to assist them in moving in the direction that both DFID and they wish to achieve.

Q18   Chair: ICAI has expressed concerns about the bureaucratic and time‑consuming business case procedures, which are a particular problem in private-sector development work.  The Permanent Secretary, when he gave evidence to us recently, told us that the business case procedures are being streamlined.  Have you seen any evidence that this is actually happening? 

Diana Good: They are the smart rules.  It is the 100 pages of smart rules, as the product at the end of the end‑to‑end review, which we are keen to do a rapid review of.

Chair: The 100 pages is the streamline.

Diana Good: That is the guidance.

Chair: Right.  Thank you very much indeed.  Thank you for your very comprehensive evidence to this Sub‑Committee.

Examination of Witnesses

Witnesses: Nick Dyer, Director General, Policy and Global Programmes, DFID, Stefan Dercon, Chief Economist, DFID, and Chris Whitty, Director of Research and Evidence, DFID, gave evidence.

Q19   Chair: Good morning.  It is good to see you here.  We are going to ask you, as we did the first panel, questions about How DFID Learns.  Welcome to this Sub‑Committee, Chris, Nick and Stefan.  ICAI’s report says that, aside from pockets of good practice, DFID does not learn well.  Now, the impression is of an organisation that does not listen to staff or partners as it should, that cannot confront failure, and that needs to use research more systematically and effectively.  I would like to say “discuss”, but I wonder how you would respond to that. 

Nick Dyer: The ICAI review came out at the same time as a second review that was done by the Cabinet Office, which looked at what works in how we generate, transmit and adopt evidence across the organisation.  Last time we spoke about this, I said that what those two reviews tell me is that, as an organisation, we have come quite a long way in terms of our use of evidence and our learning.  If you look at some of the indicators in terms of how good evidence is in business cases—the evidence survey in terms of whether staff feel, as an organisation, we are using more evidence—actually we have come a long way. 

What the reviews do tell us, though, is that it is patchy across the project cycle and it is patchy across the organisation.  There is a job of work for us to receive these two reviews and to ask ourselves how we are going to improve that consistency across the project cycle, across the organisation.  I am sure we will get into that now. 

Q20   Chair: Which of ICAI’s findings generated the most concern within DFID?

Nick Dyer: Well, the challenge that we are selective about our use of evidence and the broad challenges that we do not learn as an organisation. I would personally challenge that, because I do think we have made a huge effort over the last three years to scale up and to challenge the organisation to take evidence more seriously.  I am sure Chris and Stefan can talk about that more.  There is also the sense that we are selective about the choice of evidence. I would say there is a challenge in development to know what works and what does not work, and for people to understand when evidence has got sufficient quality about it to be able to rely on it.  There is an issue about how we know when evidence is good or bad and whether people are choosing the right type of evidence.  Again, I suggest that we have come a long way in trying to address that. 

Q21   Jeremy Lefroy: Do you accept that DFID is not sufficiently honest about failure and that people are reluctant to share problems, given that they are operating often in very difficult circumstances, where these problems are likely to arise?  One would be surprised if they did not arise. 

Nick Dyer: I will ask Stefan to comment on that in a second.  What I would say, just before he does, is that in all organisations there is going to be a fear of exposing yourself to things that do not work.  That is not healthy; it is not healthy in any organisation.  Certainly from a senior management perspective, it is something that we are keen to signal as not something that we want to see in DFID. 

Again last time we spoke, I talked about a concept that well‑managed failure is fine.  If you are following the rules, the systems and the procedures but, for whatever reason, the project or programme fails, that is okay; we just need to learn the lessons as to why that has happened.  Failure that is built on a failure to do things in the right way should not be tolerated.  That is a signal that, as an organisation, we are very clearly giving.  As a management team, we are now regularly tracking on a monthly basis the projects that are falling behind in terms of their performance and challenging the organisation in terms of how they are going to address those projects, and actually challenging the organisation to close them down if they are not working.

Stefan Dercon: If I may add, as Nick is saying, embedding this in an organisation is not something that you need to differentiate in terms of the difficulties of trying to do this.  There are tomes and tomes being written these days about learning to fail and how you do this. 

We have been taking a number of steps: the smart rules have been mentioned but, actually, there is quite a lot of detail there.  We should understand what smart rules are.  They are essentially trying to have a manual, in plain English, that helps to give clarity about the rules and the things that we need to do from the beginning of the conception of the idea all the way to the project’s completion.  What are the priorities?  How do you actually act and what behaviours would we expect from people working in the organisation?  Smart rules bring incentives.  They bring clarity in terms of the real things that need to be done in reporting and about the responsibility that individuals have.  Failure is something that actually has been very clearly discussed.  What are the procedures?  How do you escalate if you notice something is not going quite right and how will you bring that further into the organisation?

Of course we need to embed this further.  It is one thing to have a clear set of rules and incentives that people should be enacting.  Leadership behaviours, but also culture in the organisation, need to be following on top of this.  There is quite a lot of effort going on.  This is not something that you would institute but something you encourage people to embrace and try to be part of. 

I have just attended our yearly conference with all the economists who are working in the countries.  We have there staff members talking to each other about certain projects they have been managing.  What are the things they actually say?  “These things are failing.  This is how we close.  This is how we handle it if something is not going quite right, and we are learning from each other to do this well in an open space.”  That is the important part: creating a culture of openness and having procedures where managed failure is accepted as the steps to progress.  This is work in progress, of course, but I am very confident that we are doing a whole series of things that creates the ability for us to talk about failure in a much more constructive way than many organisations, maybe including us in the past, would do.

Q22   Sir Tony Cunningham: It would help if you gave us some concrete examples, some real examples.  Failure was reported in Tanzania, in Bangladesh, in Nigeria or wherever it was, and we did this and we did that, and action was taken.  Give us some examples. 

Stefan Dercon: I will give you one example and colleagues may want to add more—an example within the space of private sector and economic development.  In Nigeria, a particular programme called GEMS2 was trying to do an intervention in the construction sector, and trying to get that sector to create more value‑added for the workers and for the firms active in it.  It is an important sector that, in principle, employs a lot of people and it is important for people’s livelihoods.

One of the lessons from monitoring this programme was that it was not delivering well enough.  That created a situation where the office, with the implementing partner involved, had a careful dialogue.  At some point a programme improvement plan was designed, and people acted on that plan.  The improvement did not happen and it had to be closed.  That created some kind of fallout, but it can be handled, and other parts of this programme can continue.  The individuals involved acted properly by reporting concerns they had so that action could be taken, and that is a positive thing. 

Other teams that were operating similar programmes were listening very carefully.  They reflected, “Well, we are not entirely sure at this moment,” and they were learning from each other in terms of how they were now going to monitor certain other programmes that were taking place, and talking about the nature of the actions.  A lot of these things have a lot to do with the detail.  This is not simply procedural, and that sharing of experience seems to help staff an awful lot in terms of learning how to handle these things.

Q23   Fiona Bruce: How do you respond to ICAI’s criticism about DFID’s use of research, namely that the Department is not getting value for money for the research expenditure it is undertaking?

Chris Whitty: DFID broadly undertakes three different types of research.  One of the things people often conflate is those three different types of research.  One type is trying to research into new products, for example, new antimalarials, new bed nets, new vaccines and new seeds that are drought resistant.  That is just a matter of a global public good, and the question there is: do we get a proper return on investment?  Well, it is difficult to analyse, but studies that have been done on this suggest the return on investment from that kind of research is incredibly high.  For some of the work done on new rice in the Philippines, the return on investment is in the billions for research expenditure in the millions.  That sort of research is relatively straightforward both to conceptualise and test.

The second kind is research around testing out whether we are actually delivering things properly, because in many areas of development we know what to do, in a sense.  We have got the right kit, antimalarials or whatever it might be, but it is not getting to the right people.  That is the methodologies of things such as impact evaluations, trials and experimental methods.  Around 40% of our research budget goes on that, and that is the bit that should be headed very firmly both at DFID and its own staff, but also at the wider community.  We are trying to do things that say “this works” or “this does not work”.  Both of those are equally important.  To go back to the earlier question, there is often a belief in development that everything works a bit.  Actually some things fail completely and some things are actively harmful. 

It is clearly important to differentiate between things that are really well executed but do not work, and things that are a perfectly good idea but the execution has not worked through.  We need to learn lessons from both of those, but they are different sorts of lessons.  For example, the DART trial of antiretrovirals in Uganda for HIV demonstrated that doing expensive tests did not work.  For that relatively modest investment of the original experiment, we are then able to stop doing something, which means we can treat a third more people for the same amount of money.  That is an astonishing return on investment.  I am not claiming that every bit of research is in that group, but there are very large numbers of things where either you confirm a practice or you can stop doing things that are pointless and therefore waste money.

The third area of research is to understand the environment: what is driving drug resistance for malaria in South-East Asia?  Why are girls in Afghanistan not going to school?  This is generally the cheapest form of research, because it is observational, and helps us make rational decisions about where we should put our resources.  There is an argument for all three of those.  Not every piece of research will be good value, but overall, whenever it has been tested, it has been seen to have a very substantial return on investment.

Fiona Bruce: Yes, I am still concerned to understand the processes by which you decide to ensure that you get that return—what processes you use.  Could you just elaborate on that?

Chris Whitty: Okay, so there is a process to decide what research to do.  I am not going to dwell on that for very long because that is not really part of this ICAI Report.

Fiona Bruce: It is the return, yes.

Chris Whitty: It is how we use the results at the end.  In terms of the bit at the beginning, it is a matter of trying to use best efforts using experts in the field, getting proper peer review, trying to make sure we choose the right things and the right people to do it.  That, in a sense, is a mechanism for doing that.

What the ICAI Report asks, and it is a very fair challenge, is: “Are we using the results from this research in DFID’s work?”  For some of them, like the global public goods that I talked about the beginning, in a sense DFID is not the target audience.  For example, with a new seed that is drought resistant, DFID will only be one user among very many.  The really important research is the one that is trying to work out whether what we are doing is effective and cost-effective, so a lot of that is the experimental stuff.  On this, there is a very large body of research that is rolling through now, but it only really started in this kind of scale about three or four years ago because of the way budgets are allocated. Therefore the results of this are only beginning to come through now. 

It is true that there is a lot of resource going into research, and the results are now coming through regularly. We have got fantastic clinical trials done jointly with the Wellcome Trust and the MRC. We have invested in those for three years; the results are beginning to come out now, and it is from now on in that, if we are not using them, we really should be very ashamed of ourselves. But at this stage essentially we are only able to use the results of stuff that has come through to date.  I do think, though, DFID has been very good at using its own research compared with most other development agencies.  We should be proud of that rather than worried about it, but the challenge from ICAI is a fair one.  That is not true in every area of our development. 

The final point I would make is if you look across all of what DFID does, it is very patchy.  Health, for example, has always taken research evidence really seriously.  Some, like education, rather less so, but now does.  Historically, every field, including domestic departments, has been much less good at taking research evidence seriously, so it is patchy not just in different places but also in different disciplines in the organisation.   

Q24   Fiona Bruce: Can I just ask you to clarify?  ICAI says: “DFID may not be targeting its research efforts sufficiently on its key priorities.”  Is that one of the lessons that you are learning to ensure that value for money is achieved?

Chris Whitty: Where I think ICAI has a very fair point is particularly on evaluation, which is slightly to one side of research but is very firmly in that area.  The ICAI Report has helped us recognise that our perfectly good evaluations have often been targeted at areas where we already know quite a lot, and there are other areas where we do not know much and should be doing much more evaluation.  We are trying to do a complete rethink about how we construct the portfolio of evaluations to make sure we are better targeted.  It is a challenge ICAI have laid down and one we accept.

Q25   Fiona Bruce: Will value for money be a key element of that evaluation?

Chris Whitty: Yes.  One of the things we want to do is find out where we are spending this money and where the evidence is weakest, and target it at that.

Q26   Jeremy Lefroy: Something that has been raised in Parliament recently—and the Prime Minister has expressed great concern about, quite rightly—is resistance to antibiotics.  Clearly, this is not just a question for international development; this is a question for the world, but the initial impact is going to be felt in developing countries, where perhaps the use of antibiotics is the most profligate, and used on the wrong things.  Is DFID looking at this at all in conjunction with the Department of Health, the World Health Organisation and others?  I detect, and I may be completely wrong on this, a lack of urgency.  Given that, with things like the Medicines for Malaria Venture, we have seen a lot of very good medicines either developed or in the process of development to tackle the disease, which has really helped us over the last 15 to 20 years, would you say that something similar needs to be done on a wider scale for antibiotics? 

Chris Whitty: Historically, in the division of labour between DFID and the Department of Health, we have tended to concentrate heavily on the antiparasitics, things like malaria, sleeping sickness and leishmaniasis.  As Chair of the All-Party Parliamentary Group on Malaria and Tropical Diseases, Mr Lefroy, you will know there is a very major programme of trying to bring through new antimalarials and new drugs for the antiparasitic diseases to tackle exactly this issue.  For the anti-tuberculosis drugs, we have also got a big programme of mapping stuff out and better diagnostics.  For example, the GeneXpert diagnostic system in South Africa is something that DFID has heavily supported, and also new drugs. 

Historically we have done less on antibiotics, except through strengthening health services to do better diagnosis, which is a very key step in it.  It is very clear that we need to move further into antibiotics.  At the moment we are exploring how we can go about mapping out the antibiotic resistance to antibacterial drugs.  Whether DFID will be part of the large push to get new antibiotics is an open question; we are discussing it with the Department of Health, the Wellcome Trust and others at the moment, but my guess is our most important job is to get the systems right and to map out exactly where in the developing world this is a major problem, because it is a very major problem.     

Q27   Fiona Bruce: Coming back to my two specific points: firstly, you are creating two new regional research hubs in Delhi and Nairobi to bring implementation and research closer together.  Could you describe how they will work in practice?

Chris Whitty: We piloted this with a research hub in Delhi, where we have got a very experienced research lead and then some extremely good locally engaged staff who are also scientists.  The idea of this is to achieve two things. We have previously been justifiably criticised by this Committee and others that we are not transmitting our information right out to the regions and learning the lessons from the regions and reflecting them back.  Partly it is to provide a two-way traffic of information between the centre and the regions in evidence and research, and secondly to use Indian and other South Asian research expertise to address general development problems.  That was extremely successful.  We did an internal audit of it, which we thought was very good value for money, so we have now gone ahead and done a second one, which started last year in Nairobi, to do the same thing in East Africa.   

Q28   Fiona Bruce: Excellent, thank you.  How much of your research portfolio is undertaken by UK universities?  Are you making efforts to increase the amount of research done in the UK?

Chris Whitty: The first thing to say, very clearly, is that as with the rest of aid the research budget is untied, so we do not give any organisation money for research just because it is British.  The UK has one of the best and most intellectually and practically able groups of academic research and development in the world.  That is a historical fact, and it remains a fact in health, agriculture and a whole variety of other areas.  Although our aid is untied, because UK institutions are very strong, they win a very high proportion of the competitions that we run.  Additionally, we do a lot with the UK Research Councils, for example BVSRC in agriculture, MRC in health and ERC in environmental stuff.  Although it is open to all researchers in the world, because the UK institutions understand the systems, they often at least get through the process extremely well.  We do not try to just slap a Union Jack on it.  If the best research in the world is not in the UK, why should we be paying for the second best research for development just because there is a second rate institution in the UK?  However, frankly there are so many first rate institutions in the UK that they get a very high proportion on open global competition. 

Q29   Jeremy Lefroy: This is a question about the use of evaluation, and ICAI’s concerns that your rapid scaling up of evaluations means that you cannot pull together the information from them, and you do not use them strategically.  Perhaps Nick would like to comment on that.

Nick Dyer: Maybe Chris, as Director of Research and the Head of Evaluation, will take that as well.

Chris Whitty: I am not trying to do a monologue on this.  I think the criticism is a very fair one. We have looked at our evaluation portfolio.  People heard the message that we wanted to do more evaluation and understood that to mean we want many more evaluations.  The number ballooned but the quality did not balloon, so we are taking a step back.  I have agreed with my team that we are going to try to cut down significantly on the number we do, and concentrate much more both on the strategic fit, to go back to a previous question, but also on the quality.  There is no point having 400 not very good evaluations.  It is much better to have 30 or 40 really good ones that will have a high chance of changing practice at the end.  I totally take the ICAI point and agree, and we are trying to change it.  

Q30   Jeremy Lefroy: I think, Professor Whitty, we would all be delighted with that.  Just a follow-up question: with only two years left to run, just £24 million has been allocated to external evaluations under the Global Evaluations Framework Agreement, but it had a total indicative allocation of £150 million.  I do not see any reason why you should spend money if you do not need to spend money.  That is not in itself a criticism, but why was £150 million allocated in the first place?

Chris Whitty: To be clear on that, it was not an allocation of £150 million.  It was simply a signal to the market that up to that was an amount that we could spend if necessary.  This is a framework agreement; it does not actually have any money attached to it.  It is a way of meaning that we have pre-agreed suppliers so we can basically get evaluations quickly from a list of people we trust to do good evaluations.  There is a very large industry of not very good consultants who make good money out of doing very bad evaluations.  This is a way of trying to cut down on that rather than to indicate a budget per se.  

Jeremy Lefroy: Thank you for being so frank.

Chair: Thank you very much to this panel from DFID on How DFID Learns.  We have a separate panel to look at the private-sector development work, so thank you, Chris, Nick and Stefan.

Examination of Witnesses

Witnesses: David Kennedy, Director General, Economic Development, DFID, Alistair Fernie, Director, International Finance, DFID, and Meenakshi Nath, Head, Private Sector Development, DFID, gave evidence.

Chair: A warm welcome to Meenakshi Nath, David Kennedy and Alistair Fernie.  We are going to discuss the ICAI report on DFID’s private-sector development work. 

Q31   Fiona Bruce: I think we would all agree that private-sector development is critical, and we are all very supportive of the priority that the present Secretary of State gave to it when she came into office.  It is understandable that DFID has been on perhaps a fairly steep trajectory in prioritising this.  It is therefore perhaps also to a degree understandable that the report that ICAI has produced is somewhat critical in that it is an overall amber/red report, and contains quite a lot of comments that we want to probe with you.  First of all, the report says: “Whilst there are some very good micro-level projects, there is not a clear, coherent realistic joined-up country level portfolio strategy”—despite, I have to say, a huge amount of money having been put into this area.  Could you respond to that criticism? 

David Kennedy: I have recently joined DFID, and I had the same concern from the outside.  As the ICAI team was doing their research, as they produced their report, I had the same questions.  I had seen the strategy published in January this year and, for me, it sounded like DFID wanted to do everything in this space.  What was the comparative advantage, given that other institutions that I had worked for are very active here, for example, the World Bank?  Does DFID have the people?  You could put that all together and say I was worried about whether there would be a coherent approach or not.

As I have joined DFID, and understood what we are doing in this area and taken part in moving that approach forward, I am confident that we are putting in place a coherent approach. 

Q32   Fiona Bruce: What are you actually doing to address this practically?

David Kennedy: Let us split this into two parts.  There is 2015-16, which is the year that we have the £1.8 billion spending target for.  It is a bottom-up process where country and central teams have been asked to come forward with project proposals that would support economic development.  This is not from nothing; we do have good expertise in this area.  We have a very strong economics team who have a good understanding of the growth process.  We have a network of private-sector advisers at the level—and this has surprised me—of the other institutions I have worked in that have focused on the investment climate: the European Bank for Reconstruction and Development, and the World Bank.  DFID is not full of investment bankers, but it does have a strong capacity for identifying the challenges in economic development, and particularly what it is that would involve growth.  We have that set of people who are bringing forward proposals. 

We also have an evidence base.  If you look at the country level, for example, people are not working in a vacuum.  There is a lot of evidence for each of the countries where we are bringing forward proposals, whether that is World Bank assessments of economic development priorities or USAID growth diagnostics.  You put all that together: the people, the country knowledge; the evidence base; and a set of proposals.  At that level it is bottom up and, as I think was said before, you also need the top-down aspect.  In the near term, the top-down aspect is that we are now going through a very rigorous assessment process to say: “What do those programmes look like?  Where is the spend?  Is it in agriculture, the financial sector or is to support export-based industry?  How does that match with what we know are the priorities for economic development?  What are the processes that the teams have gone through in order to go from a long list to a short list of projects?”  That is something we are in the middle of.

For my taste, that is still a bit too much bottom up from an ideal perspective.  We do need to rebalance the bottom-up and top-down approaches.  At the same time we are introducing a diagnostic strategic approach, which is something we are working on at the centre.  My teams in economic development are working with the chief economist to say: “What is it we know about the growth process?  What is it we know in general about opportunities, barriers and responses in the economic development space?” and then, “What is it we can usefully do?”  We are looking at that in general across each of the sectors that will drive growth, whether it is industry, agriculture or infrastructure.

Having designed an approach, we will pilot that in five countries in each of our regions in the period to Christmas to then work out what is a set of things that DFID can usefully do.  We will roll that out, beyond Christmas, to all of our countries, so we will have a full diagnostic done of what the priority actions are, what others are doing and what it is that DFID can usefully do. 

Q33   Fiona Bruce: How many people with a fairly high-level private-sector background are working now to assist you with this?  How are you liaising with the countries that you are working with, the private sectors there, to ensure that you are prioritising where they want to see development?  What measurements are you going to put in to ensure that you do actually provide a return for the work that you say you are doing now?

David Kennedy: Let us take each of those in turn.  In terms of private-sector background, I would push back a little bit there and say I have worked on improving the conditions for investment for the private sector throughout my career in different institutions.  It is not the case that you have to bring in people from the private sector to do that work.  In the World Bank, for example, you do not find people who have come from energy companies who are improving the investment climate for the energy sector.

What you have to have is a very good understanding of the growth process.  There is a difference between that and doing deals and transactions, where you do actually have to have the transaction experience.  You have to have a good understanding of what the private sector needs.  We have got people from the private-sector background, and we can give you some numbers in a minute.  In our private-sector networks, for example, we do hire people who have a private-sector background, so that complements what we do.  We talk extensively with the private sector.  At the moment we have engaged PricewaterhouseCoopers to help us think more about how we can make use of the networks that we have built up.  The feedback there is that the private sector value talking to us.  There are good examples of interaction, for example, on extractives, but there is more we need to do to develop our approach of making good use of the private sector.

The Secretary of State has established a private-sector advisory group with very high-level people from the private sector where we can test out our ideas and approach.  In terms of talking to the countries, we talk very closely with them.  It is not the case that we design a programme in isolation from the governments and the countries who will have to execute that programme from us.  We talk continuously and we reach a programme proposal that has been agreed with the government, and it would be silly to do anything else because you would not be able to implement that programme if you did not have total government buy-in. 

Q34   Fiona Bruce: What distinguishes DFID’s role in this area as an aid agency? What is the distinctive contribution that DFID is providing?  I am quite concerned about your responses. I do not want to say that there is an element of complacency; that is wrong.  But I think the degree of concern that you should be expressing about this ICAI report is not coming across.  I note, for example, that, of all their recommendations, you only accepted the fourth, on working harder to understand barriers faced by business.  You only partially accepted any of the other recommendations.  My view is that, with the amount of money that is being put into this, the amount of prioritisation from the Secretary of State in terms of the resources being put into it, and the needs that the countries have to develop their private sectors, there should be a sense of urgency.  There should be a degree of, “Yes, we have a massive learning curve to undertake,” and that is not coming across to me.  There is no sense of openness about being willing to listen to this ICAI report, in particular.   

David Kennedy: I started by saying I am totally sympathetic to the gist of the report, which says we need a coherent approach.  Everybody agrees with that.  The question is: are we moving towards a coherent approach within DFID?  I think the answer is we are moving towards a coherent approach.  We have not got there yet, but there is a set of things that we are doing to try to bolster both the 2015-16 spending and what we do beyond that.  In terms of urgency, this is absolutely urgent.  We are working flat out on the set of things that have been raised by ICAI, and there are big challenges, so let’s not underplay those. 

However, in terms of whether there is a valuable role for DFID in this space—which is an important space, and that is not debated by anybody—there is an important role, and there are two things we can do.  There is work on improving the conditions for investment, and then actually making investments.  There is a question as to whether DFID should make investments itself or whether we should do that through investment vehicles, but the focus at the moment is very much on how we can improve the conditions for investment.  

Q35   Chair: David, you have not said why you do not accept the other recommendations that ICAI have made.  Can you explain a bit more about why you have not?  Is it because you feel you are already doing these things?

David Kennedy: I think there is a difference; I used that to illustrate the dichotomy.  I came from outside.  I understood the high-level approach from the outside, and there were a lot of questions, with only that high-level understanding.  Now having come inside the organisation, there is a lot of work that has gone on beyond that high level.  There is operationalising—the five-pillar strategy that was published in January.  Having come into the organisation and having had questions about the capacity of the people in the organisation, for example—whether it is the economists or the private-sector people—I have been pleasantly surprised by the capacity that we have.  I think is it very strong. Possibly we can build it up in certain areas.  There are questions about whether we have the right set of people in order to manage the governance aspects of our relationships with the CDC and PIDG and other things there, but we do have strong capacity in that area. 

Meenakshi Nath: In terms of the recommendations, we did partially accept three out of four.  The reason is we thought we had made some headway in each of those, and that we were not starting from a completely blank slate.

Q36   Chair: So ICAI was exaggerating its concerns?

Meenakshi Nath: That is right.  We feel that there is need for DFID to clearly define and articulate where it can add most value in PSD, and it should be realistic in its ambitions and impact.  We have published a framework, but there is still more to do.  If a framework was already published, the resource allocation round had already looked at key constraints where DFID should be at the country level.  The CPRDs had been undertaken and so on, so it was not as if we had done absolutely nothing on that.  Similarly, if you like, we can go through the other recommendations as well.  But if you look at the progress that has been made on each of these after the ICAI report, it has been quite accelerated, and that is where the ICAI influence comes into the picture. 

Q37   Chair: Alistair, did you want to add to that?

Alistair Fernie: I think I was the only one of the three of us who was actually talking with the Secretary of State when the report came out and agreeing with her what our response was—specifically on the recommendations that she wanted us to partially accept rather than accept, and we are conscious that the Committee has given views on when partial acceptance is appropriate.  I take it the first recommendation is about two things: focus and ambition, or realism.  We agree that there needs to be more focus in what we do, and the process that David has set out is part of a journey where we are trying to turn a thousand flowers blooming into a well-ordered flowerbed, if I can extend the metaphor.  On ambition, our Secretary of State feels quite strongly that this is one of her highest priority areas.  She did not want to send a signal to you in Parliament, to us as staff within DFID, or to our partners that our level of ambition is anything other than high.  We need to marry a greater degree of focus and some more coherence in the totality of what we are doing in individual countries with retaining that high level of ambition.

The second recommendation was about guidance.  We do have a slight difference of opinion with the team and the ICAI commissioners on how significant the issuing of guidance from headquarters to country offices is in helping them to shape how they put their portfolios together.  We do issue guidance, probably some that they did not see during the course of their study, and more has come out since.  But to go back to the other report that you were looking at this morning, we think that learning and continuing to learn from each other as we go on, which is a much more dynamic process than the issuing of guidance from headquarters, is probably going to have more significance in helping us to get coherent portfolios in the countries that we are working in. 

The report chimed on a number of key points—things that our Secretary of State and senior officials in DFID were concerned about—but we did not necessarily agree with everything that it said.  The key point, as David is conveying, is that there is an enormous amount of activity.  Justine Greening has identified this as a top priority.  She has brought in a director general for the first time ever, just to focus on one sectoral area.  We are recruiting significant numbers of new people.  We are bringing in private-sector expertise to help us.  There is an awful lot going on, so any sense of complacency that you might be picking up does not reflect David’s recruitment and all the effort we are putting in to respond to this high level of ministerial ambition.   

David Kennedy: Two big things have happened since the ICAI report, which is why I am probably more confident than they were that we will develop a coherent approach.  One, we have actually seen what the country programmes look like.  That information was not available at the time that the ICAI report was produced.  Secondly, we are putting in place a diagnostic, strategic approach, which will guide all of our activities in the future.  Those are two very big things that will underpin the coherence of what we do.  

Chair: This is something we certainly need to return to.

Q38   Sir Tony Cunningham: You touched on comparative advantage, and I want to probe a little bit more deeply on that.  ICAI’s report said you should be clear about your comparative advantage.  What would you say is DFID’s comparative advantage in this area?  What skills and attributes do you bring to bear relative to those within other donor agencies, including those with a more established record, like the World Bank, for example?  

David Kennedy: In terms of comparative advantage, if you look at the staff, we do have a very strong economics team, with a good understanding of the growth process.  That is at the central level, so in our HQ teams and our country teams as well.  We have our expanded network of private-sector advisers.  We have very good country knowledge just through our country offices, probably in a way that the World Bank, having worked there, does not.  On the ground, we are very well connected; we have a very good understanding of the situation.  If you put that together, that gives us a set of things that we can bring to the table. 

You can divide what we might do into two things: improving the conditions for investment, so for example, the laws, the rules and the institutions that will support market development and private-sector activity; then there is actual investment.  You have got to question whether, if all you did was investment-climate work, that would be enough. I do not think it is because you need somebody to come along and show that the investment-climate work can bear fruit—that you have catalytic investments that can then be replicated on a commercial basis by the private sector.  Within that, at the moment our focus is on improving the conditions for investment.  You are right; the World Bank is also very focused on this area, so we should not be duplicating what they do, for example.  USAID and other donors are very active in this area.  We need to work out not just what is important for the countries but what are others doing, and what is there left for us to do that matches well to our comparative advantage.  That is the process we are going through at the moment. 

Alistair Fernie: Can I just add a couple of other things that we hear other people tell us?  You can judge whether or not they are accurate.  One of the biggest concerns about the economic development agenda is whether we are going to see growth that does not reduce poverty.  There are a number of countries where growth levels are above 5%, and the growth appears to be not very inclusive.

Q39   Sir Tony Cunningham: Can you give me examples?

Alistair Fernie: I came back last year from four and a half years in Kenya, which has had growth rates of between 4% and 7% over the last few years.  At the moment, our Chief Economist’s projections are that the numbers of people living in absolute poverty are going to increase in Kenya, because the growth is concentrated in Nairobi and some other urban areas in a couple of provinces where there is a high level of business activity, but in remote areas—where agriculture is becoming more and more problematic and there are no jobs—we are seeing the number of people in poverty increase.  Kenya is not the only country that faces that potential risk.  A number of the countries that are high priorities for DFID are sitting on a hydrocarbon bonanza, where large amounts of coal, oil and gas is going to come out of the ground and potentially benefit the nation’s coffers.  Those industries typically do not create a lot of jobs, and we want to be as helpful as we possibly can in helping those governments make sure that the windfalls they get from those resources are invested in a way that benefits the country as far as possible.

There is a set of challenges about ensuring that growth is not only high enough but inclusive enough to reduce poverty.  What other people tell us is that the skill set we have within DFID in thinking about how activities, through often quite long causal chains, may or may not have an impact on poor people’s lives is better than the IFC’s, for example, which has a slightly different approach to looking at the development impact of some of its investments.  That is something we feel we have comparative advantage on.  We need to do more analytic work and build up the evidence.  That is one of the key points for us; we would not pretend that we have been big players in economic development for 20 or 30 years, as we have perhaps been in education or health.  Regarding the extent to which we can put into practice some of the principles you were talking about in the previous session about learning, we are starting with a small evidence base.  We need to be a little bit humble and draw on other people’s lessons, but we do think we have some skill sets and ways of working within DFID that enable us to add some value when compared with other members of the international community. 

Q40   Sir Tony Cunningham: Just one specific question: do you have a special approach for private-sector development work among fragile and conflict‑affected states?  Would there be value in developing this area?

David Kennedy: I would not say we have a fully worked‑out approach.  The challenges clearly for private-sector development in fragile states are much more pronounced than they are in other countries.  The growth story—the growth potential—is different; for example, a fragile state probably does not have a great opportunity any time soon of joining the global economy or attracting inward investment for export‑based manufacturing, so you have got to look at different things there.  You have got to look at the SME sector: what can you do with micro enterprises?  What can you do with the agriculture?  I would say those are live questions for us, and we have not got a fully worked-out approach.

Q41   Sir Tony Cunningham: The reason I ask, we heard earlier that in a country like Somalia, there is a tremendous growth in the private sector.

Meenakshi Nath: The fragile and conflict-affected states tend to be a really diverse group of countries, so you need to have a customised approach for each of them.  Some of them may have very low micro-trader levels, and some may have a higher level of private-sector activity.

Q42   Sir Tony Cunningham: But it is something you are working on.

David Kennedy: It is.  Within this growth diagnostic that we have talked about, which will frame all of the things we do in the future, there will be specific aspects that are for fragile states that pull out the different challenges and the potential responses.

Meenakshi Nath: But we are also looking at a framework, because it needs a specific lens; we should not exacerbate the conflict through our private-sector work.  Traditionally our focus has been on state building, which remains crucial, because you need the rule of law to get good quality economic activity, but also to look at some of those specific questions in context.

Q43   Jeremy Lefroy: ICAI found it impossible to identify how much DFID actually spends on private-sector development because it is not recorded as a category of spending in DFID’s financial system.  Is that being addressed now?  Clearly, if we are going to look at value for money and the results of the work, we need to know how much is actually committed to it.

David Kennedy: It is being addressed.   I look to either Alistair or Meenakshi to explain how we are doing the recoding within our system.

Alistair Fernie: It is important to make a distinction here, and explain what we mean by private-sector development and economic development.  When our current Secretary of State set out her priorities a few months after she came into office, it was clear that she was focused on economic development rather than private-sector development.  Included in that is investment-climate work, which usually involves providing technical assistance to government ministries.  That is not usually coded or counted internationally at the DAC, for example, as being an investment in the private sector, or working directly with the private sector, because it is technical assistance to government.  The £1.8 billion spending target that she announced in January refers to our economic development plans and includes our plans to invest and work with governments on issues that may create an environment that makes it easier for more private-sector investment to happen. 

Q44   Jeremy Lefroy: Does that include revenue collection?

Alistair Fernie: Revenue collection, I think, is not included in economic development, but we see it as being closely related and one of the benefits.

Q45   Jeremy Lefroy: The reason I ask is that it is clear to me that a proper good investment climate needs a proper revenue collection that is conducted in a straight and non‑corrupt way. Unfortunately, that is not the case in many of the countries in which DFID is operating, so I wondered if there was a case for actually bringing that in, because it is so related to economic development.

Alistair Fernie: We are trying to capture our Secretary of State’s policy priorities but also work within the internationally agreed codes, which the OECD DAC publishes.  On the policy point that you make, as your Committee is probably aware, we have been ramping up our domestic resource mobilisation work, including work with HMRC in terms of reducing aid dependency. When we look at countries—I know Pakistan is one dear to your heart—where the percentage of GNP collected as revenue is pretty low, we think there is scope not just for technical assistance but for some political will to move it forward.  In other countries, and Burundi is a good example, we have contributed through some relatively cheap technical assistance to a dramatic increase in revenue without apparently scaring away businesses.  That is one of the challenges in doing this kind of work.  Some people will say if you collect more tax or raise the tax rates you will stifle business activity, but there is plenty of evidence that you can significantly increase a country’s tax rate without stifling business activity.  

David Kennedy: Going back to what is our comparative advantage, one of the areas that we focus on that other institutions maybe do not is the link between growth and inclusive growth.  One of the roots for inclusive growth is to take the tax revenues from the value that is being generated and spend it wisely on public-service provision, for example.  That is a real area of focus for us.  

Q46   Jeremy Lefroy: Obviously economic development is absolutely vital, but as you referred to earlier, if this is not creating livelihoods and jobs, there is a real problem with inclusive growth.  What focus do you have on jobs and livelihoods within economic development?

David Kennedy: I will just give you a sense of the diagnostic process that we are going through to identify priorities. We say, “What are the opportunities in industry?” and within that we think of large‑scale industry and SMEs.  Then we say, “What are the opportunities within agriculture, where most people still live on the land and there is a big productivity gap; what is it you can do there?”  Then we say, “What are the opportunities in infrastructure?”  In each of those areas we will come to the conclusion, depending on the specific context, “Well, there is something lacking in the investment climate; if you could address that and other things were to happen as well, then you would unlock an investment story for large-scale manufacturing industry”, for example.  All of these areas, if you can get them to work, actually have a very important and profound impact on jobs and livelihoods, so anything you can do to boost industry, whether it is large-scale or SMEs, and anything you can do to boost productivity in agriculture will be of great benefit in terms of the matrix you have just talked about. 

There is a question of how you actually assess what the opportunities are and what the response is.  Then there is a question about what your results framework is—what is it you are trying to achieve there?  Jobs and livelihoods will be central to that whole set of analytical questions, including the results framework that we need to develop. 

Alistair Fernie: Obviously, we are aware that the Committee is doing an inquiry on jobs and livelihoods, and the Government will be submitting its evidence on that in a month or so.  One thing we have not mentioned so far is CDC, and the ICAI team were steered, I think, after a conversation with the NAO, not to look at CDC or the Private Infrastructure Development Group, both of which the NAO have been looking at this year.  In some ways that meant that their report did not cover a significant part of our private-sector approach.  As some of you will be aware, when CDC was re-strategised in 2011-12, it picked on jobs as its single predominant metric for assessing the development impact of what it does and is now supporting businesses around the world that have created or are sustaining over a million jobs. 

The really tricky methodological work here is looking at direct job creation, where we can say that something that donor money has supported has created a job which would not otherwise have existed.  There is then a whole series of other things to look at: indirect job creation, where supply chains are being strengthened because of businesses growing, where we can see that there is some kind of an economic multiplier effect in the community because a business is employing more people.  It is much more difficult to measure those things, but ultimately some of those effects will have more impact on poorer people’s livelihoods, because it is often not the poorest people who get the full-time, paid jobs when we look at the patterns of employment across Africa and India in particular. 

The number of people who have full-time, paid jobs where they take home a payslip at the end of the month is pretty low; it is generally in the 20% to 30% range.  Most people are eking out their livelihoods from some kind of informal grey-area activity.  They will not be getting jobs in CDC-funded businesses, but they may be benefiting from those businesses.  We are working with IFC and other development finance institutions to try to agree an international methodology where we can make some kind of credible claims as to what these secondary impacts on jobs and livelihoods are.  Our evidence to the Committee’s inquiry on jobs and livelihoods will set some of that out in more detail.   

Chair: Thank you, Alistair.  I am very sorry; we have run out of time. DFID’s private-sector development work is clearly something we will return to, and I hope we will return to it soon in October.  Although we have explored a great deal this morning, I do not think we have explored everything that we need to have done or that we could have done.  Thank you very much indeed for your attendance, Meenakshi Nath, David Kenney and Alistair Fernie.  That ends today’s ICAI Sub-Committee.  I am grateful for your attendance, and for everybody else who has given evidence today.  Thank you.  

              Oral evidence: The Work of the Independent Commission for Aid Impact, HC 652                            23