Sub-Committee on the work of the Independent Commission for Aid Impact

Oral evidence: Reports on (i) Business in Development, and (ii) DFID’s Approach to Delivering Impact, HC 597
Tuesday 10 November 2015

Ordered by the House of Commons to be published on 10 November 2015.

Watch the meeting: Tuesday 10 November 2015

Members present: Fabian Hamilton (Chair); Fiona Bruce; Mrs Helen Grant; Jeremy Lefroy 

Questions 1-45

Witnesses: Graham Ward CBE, Former Chief Commissioner; Diana Good, Commissioner, Dr Marc Stephens, Contractor, and Marcus Cox, Contractor, Independent Commission for Aid Impact (ICAI), gave evidence 

Q1   Chair: Chair: Forgive us for starting a little early, but because we are going to have several divisions I thought we had better get going as quickly as we can; it may be a little disrupted this afternoon.  Before we begin with questions, let me just thank you very much, all of you, for coming, especially Graham and Diana, as our former commissioners, although I realise Diana is now working for the Committee too, but, Graham, thank you for giving up your time again now you no longer have the responsibility of being Chief Commissioner. 

Graham Ward: It is a pleasure, Chairman.

Q2   Chair: Before we start, is there anything any of you want to say or shall we launch straight in to the first question?

Graham Ward: I am probably happy to launch straight in.  The order in which you are going to address our reports is what—BID followed by impact or is it the other way around? 

Chair: We will talk about BID first, yes, indeed.

Graham Ward: Right, okay.  I did have a couple of opening lines on BID specifically rather than other things, if that is all right.

Chair: Please go ahead.

Graham Ward: It is a pleasure for us to be here and to share our thoughts with you on these two reports.  If we look at BID, I think it is fair to say that our report comes at a pretty crucial juncture in international development in the context of the global goals, which were designed with a very strong involvement of the private sector.  In particular, with goal 8 on decent work and economic growth, there is direct relevance there that will strengthen the ability of business substantially to help to achieve those goals. 

We examined DFID’s work with business at a pretty early stage.  They clearly stated their intention to work with business more and there are great potential benefits, we believe, to come from that collaboration.  We are concerned, however, about how DFID will translate the goals into practical actions without more strategic oversight of the business engagement activities and without more concrete targets.  In order to work with business, we believe that DFID needs to provide practical guidance on how to engage and the business benefits of doing so. 

The investment involved for business in this activity should not be underestimated.  We believe that DFID needs to understand thoroughly how business works and to bring in the right people in the right numbers in order to deliver that relationship for the benefit of the world’s poor. 

Q3   Chair: Let me kick off by asking you this.  I will just stress, because of shortage of time, not everybody needs to answer every question, but please chip in if you feel it is important to say something.  ICAI’s Business in Development report found that DFID was performing relatively poorly overall in this particular area and I wondered if you could tell us how DFID could best focus its use of loans, equity investments and guarantees in order to help reduce poverty.

Graham Ward: Certainly.  Perhaps I should start by saying that the term “LEG”, loans, equity and guarantees, is not a term that has been used by DFID, but it is a term that was coined by us in order to aid public understanding of what is going on in this area.  I would like, if I may, to ask Marc to provide a brief explanation of why it was that we went there and then to move into the substance of your question.

Dr Stephens: We were interested in looking at loans, equity and guarantees because of the specific role they have in encouraging private investment into developing countries and also because of the special requirements they place on DFID staff, so what we were looking for was quite specific.  We found that the term DFID used, which is “development capital”, included things other than loans, equity and grants, for example all of the transfers to the multilaterals, many of which do not end up as loans, equity and grants.  For that reason, we felt compelled to come up with a term and we came up with a term that we felt was as clear and as evident to end recipients and users as possible, hence LEG.

Graham Ward: We could look then, first of all, at the question of who the poor are and effective targeting as far as this mode is concerned.  There are approaches in terms of, say, geographic focus.  In India, the fund managers have a very clear focus on low‑income states.  I saw personally some of the work that was being done there in this area.  A dairy farm in Rajasthan, a very poor state in India, obtained a loan from something called the Samridhi fund, which is part‑funded by DFID.  It helps local smallholders to improve the milk yields, to make sure that the milk is properly chilled on a timely basis in order to keep it healthy and fresh and then to have the milk aggregated and taken to market.  What is happening here is establishing a proper, healthy supply chain for the supply of milk and that seems to be working very successfully. 

One just needs to bear in mind that there is a bit of a downside, perhaps, to geographic targeting, in that it could place too many constraints on the flow of investment opportunities and it is pretty hard to make sure that the right flow of investment opportunities does appear.  We can also look in terms of the health markets, where there was a private provider called AHME providing health services to networks in the bottom quintile of wealth.  That is an area where LEG is not being used at the moment, but we believe that there is potential for that. 

We saw some issues in relation to large infrastructure projects where LEG financing was coming in.  One example would be to look at the Kpone independent power project in Ghana, part funded by an £11 million equity investment from something called InfraCo Africa, part of PIDG, but where the result in terms of individual poor people is quite hard to tie down, because once you are providing power into a power network, of course, you cannot say which electrons in the system come from which power station.  On the other hand, the improvement to the economy of the country as a whole is clearly something that is worthwhile and there are first‑order benefits coming from that sort of work in terms of jobs and spending by workers who are employed by the project and the work that is provided throughout the supply chain to the power station.  There are some benefits that come through that. 

We have seen benefits in relation to combining LEG with other inputs.  There is an affordable housing project in Jaipur, which again I looked at personally and saw the benefits coming out of that.  It is something similar to the housing association projects that take place here in the UK.  You could see that DFID was bringing some real value into this in terms of the convening power that it had, being able to bring together the state and private organisations, being able to obtain the political settlement that was so necessary to carrying out this sort of work, policy advice from DFID, prudentialisation coming from DFID. 

Indeed, this project was then being seen as something of a model project.  The state of Rajasthan had adopted this.  The Chief Minister had instructed the Chief Secretary, who told us with some trepidation that he had been instructed to build 20 million new homes by 2020, which is no joke.  Other states were then taking this on as well, seeing it as a model and looking to follow the pattern that DFID had brought in.  Those are some examples of how this can produce real benefit.

Q4   Chair: Thank you very much for that answer.  Let me just move on then to CDC Group.  Its funding is being dramatically increased over the next few years and I wondered whether you had any concerns at all about its ability to invest such a large amount of money responsibly.  As a supplementary, before anyone replies, does CDC Group need to rethink its portfolio in order to ensure that its focus is also on poverty reduction?

Graham Ward: I probably should start by saying that CDC itself was out of scope for this report, because at the time that we were agreeing the terms of reference with you—and, of course, this goes back a long way; it is very nearly two years ago that we were doing that—the National Audit Office had an examination of CDC on its agenda.  Of course, we work with the National Audit Office to make sure we do not duplicate work and waste taxpayers’ money and British resources in that duplication. 

On the other hand, I could say that, in the general case, the fact is that, for any investment vehicle, the absorptive capacity of the vehicle is absolutely crucial.  That is, in other words, having the right people within the vehicle to be able to make proper assessments of the investment opportunities that are coming through, to make sure that investment opportunities that are supported are in line with the policy of the investment vehicle and then to monitor the progress being made by investments that are taken up and followed through.  One would apply the natural scepticism that would apply to any investment vehicle to CDC, but, in terms of facts known to us, that is about as far as I can take it. 

What your question does is bring out the really important benefits of proper high‑level oversight of the investment portfolio in this area.  Back in India again, where they take their own investment decisions as opposed to what happens in the rest of the world as far as BID is concerned, we saw they have a proper LEG oversight committee with some very senior businesspeople and investors.  We met with some of these people, and they really do know what they are doing.  That meant that it was possible for them to identify learning across different investments.  They could spot portfolio‑wide risks, so they could guard against excessive concentration, say, in particular geographies, particular asset classes, particular stages of funding.  They were in a position to have a really informed, in‑depth and critical discussion with fund managers and other intermediaries.  They could spot synergies.  They could avoid overlaps.  There were lots of really important things there.

A concern that we have, Chairman, in relation to the DFID BID activity as a whole, is that we do not believe there is sufficient commercial investment experience being brought to bear in this area on the investment committee.  If one looks at the composition of the investment committee, there is only one non‑executive member with commercial investment experience as a fund manager out of 12 people.  There is another non‑executive director of DFID who has much experience as a reporting accountant and auditor, but not as a fund manager, and the remainder are drawn from what one might call the normal DFID staff, albeit at a very senior level, but nevertheless from that area of life rather than from the investment management area. 

There is a risk here, but, especially in the non‑fiscal area, which is a new and rapidly expanding area of work for DFID, there are new skills required to be able to manage these things effectively.  There are risks and uncertainties that those new skills need to be brought to bear upon and we are not confident that there are enough of those in the right place right now.  I know Marc has some other points in relation to this.

Dr Stephens: The essence of the LEG investment is that this has to be additional and over and above what the private sector on its own can achieve.  Ensuring that that is the case is the key.  This is related to the question of the sheer ramp‑up and speed of increase of spending in this area.  The danger is that DFID could lose sight of that core imperative of additionality if the pressure to spend becomes too great, because, as the pressure to spend increases, there is a risk that easy investments are the ones that receive funding.  As soon as it is the easy investments that are receiving DFID funding, the core goal of additionality goes out of the window.  Whether it is DFID itself that is searching for opportunities to spend and feeling pressured to slip from investing where others fear to tread into easier investments, or whether it is funding that is then transferred from DFID to other organisations, be it CDC or other vehicles, either way the risk is that that core goal of additionality is lost sight of.

Q5   Fiona Bruce: That is very interesting, because I do have a question following on from that, which is: have you managed to discern what the characteristics are of a project that effectively promotes additionality?  Can I also just come back to Graham?  Do you think DFID acknowledges this skills gap that you have referred to?  Is it actively addressing it?

Graham Ward: I have seen different reactions within DFID.  There are a number of people with whom we have had conversations who do understand that the skills gap is there.  There are others who do not seem to take it as seriously.  If we think back to evidence that was given to the Sub‑Committee in the previous Parliament by somebody from DFID, he did not seem to think that it was terribly important, but I must say we disagree with that view as expressed by that individual during that hearing.  What we do believe, then and now, is what I said just now.  Marc has some comments on your first question, if he may.

Dr Stephens: In essence, they are investments that are taking place in areas, be they geographies, sectors or at stages of funding where mainstream private sector investors are loth to invest.  Whether it is the eight low‑income states in India, where one of the projects we saw was the one that Graham described, the dairy farming project, or whether it is northern parts of Ghana, where it is very hard to find private sector investment—we spoke to AgDevCo there and heard that the person in charge of the fund had had to visit 100 possible investments before being able to make one investment—those are precisely the areas that are extremely difficult to invest in but which are, by definition, the ones where you are going to be additional, because private sector investors are not going to be there.

Chair: I have to interrupt you, because there is a division in the House now.  There may be two votes, so we will try to reconvene at 5 o’clock, provided there is a quorum in the room.  I apologise for this and we will see you very shortly.

              Sitting suspended for a Division in the House.

              On resuming—

Q6   Fiona Bruce: This picks up on a comment that Graham made earlier about the need for concrete targets.  If I may, I will elaborate a little on the question.  A major concern appears to be that the results agenda has led to the value for money agenda suffering from a lack of credible methods for measuring and comparing the real impacts of aid.  How has this affected ICAI’s ability to measure value for money?

Diana Good: I am addressing this from the point of view of our impact report.  ICAI has always used DFID’s results data as a starting point for its analysis and reviews and then has tested that data, looking behind it and beyond it to see what impact has been achieved.  The results agenda has undoubtedly served a very useful purpose: it has created accountability; it has been very influential with DFID’s implementers.  But it has had the unintended consequences of creating incentives and, indeed, tools and processes that have focused on short‑term results, big numbers and impressive numbers rather than looking at longer‑term transformational impact. 

By way of example, we looked at DFID’s reach figures for nutrition, which report centrally, as a matter of aggregated results data, how many women and children have been reached in terms of their nutrition programming.  “Reach” can mean literally a child receiving one deworming tablet on one day and then never going back into the health clinic and being extremely ill, but they would still count as being reached.  That is numbers, not quality.

In terms of education, in our education reports we identified the problem of large numbers of children stated as being enrolled, but what was the quality of the education?  Did they have any teachers?  Were the teachers trained?  Did they pass any exams?  Were they literate?  Were they numerate?  No.  Now there is a much greater focus on ensuring quality in that area.

We have repeatedly, with DFID’s implementers, come across them, whether they are UN agencies, contractors or NGOs, saying that they feel under pressure to produce large numbers in the short term and repeatedly we have found implementers saying, “You are asking us about impact, but DFID does not usually ask us about that.  They ask about how much has been spent, whether the money has been spent, geographic spread, numbers.” 

By way of example, when we looked at the justice and security programming in both Malawi and Bangladesh, implementers were talking about geographic spread.  They would say that they had covered the entire nation with improving village tribunal work and getting women onto tribunals, but they had only provided one day of training to women who had never sat on a tribunal before, to teach them about constitutional human rights, diversity issues, land tenure at a one‑off, one‑day training.  Our view was that it would have been much better in terms of impact to focus on a smaller area, really deepen that, get really good training going and only then scale it up.  This goes back to the scale and spend point. 

Certainly, when we look at the Departmental Results Framework, it tends to generate an analysis of impressive numbers rather than long‑term, beneficial, sustainable impact.  In Pakistan, there was a particular example that struck us where DFID has its biggest education programme of anywhere it is operating.  It is doing some very important things, like training teachers  but, in terms of the results framework, the way in which they analyse and report what they are doing is to take the amount of the DFID spend, compare it with the national government spend and come up with figures.  Although a critical part of the DFID programme was to ensure that the national Government spent more itself, because they analysed it on the basis of a percentage of the original total, when the Punjab increased its own budget on education DFID had to revise down the figures. 

This just goes to show that it is not a way of actually analysing what impact you are having.  We are not suggesting for a moment that the results agenda should be thrown away.  Clearly, accountability is hugely important, but it needs to step up, actually start to measure something and thereby create incentives on the implementers and on DFID staff themselves to be working on things that are much longer term and much more transformational.

Graham Ward: Marc has an important additional point to raise in answer to your question.

Dr Stephens: In terms of measuring value for money in BID‑type projects, one of the things that will add value will be if the wider catalytic impact of those investments is captured.  If you are only looking at the direct benefits, that is one thing, but the whole picture is maybe going to be nine‑tenths.  One of the things we said in our report was that we felt more could be done, first of all, to try to ensure that those wider catalytic demonstration effects occur and, secondly, to try to capture and document when they have happened.  That is one really important difference about the value for money in the BID‑type projects.

A second specific issue is that, with the scale of projects that are being funded through BID, such as, for example, the Kpone power project in Ghana being funded through the PIDG sub‑fund, the benefits of that, as Graham said, are sometimes quite hard to be pin down.  In that case, what is required is complementary efforts just to make sure that those big investments, for example to increase the capacity of energy production, are complemented by initiatives that make sure the benefits accrue to your target population, i.e. the poor.  That might require regulatory changes; it might require initiatives to ensure that people have access to the grid as well as increasing capacity.  That is the second thing.

The third point is around measurement and being specific about whom we mean by “the poor”.  In value for money, what do we mean by “value”?  In reaching out to the target population of concern, the poor, whom do we mean by “the poor”?  We saw a couple of really good examples of where that was done well.  One was the African Health Markets project in Ghana that was being done in collaboration with Marie Stopes.  They had taken a major investment to do a baseline survey and specify that it was the bottom quintile of the population they were trying to reach.  They found that they were not getting access to that population.  Hats off to them for having done that work.  They are now trying to reorient the project so that they do target it. 

The second good example was a project by Discovery Channel, a US broadcasting company investing in Ghana alongside DFID, providing IT‑based educational materials.  Again, they were targeting marginalised girls.  They had really invested in a baseline survey, defining what they meant by “marginalised girls” and then following up.  That is the standard that one would like to see.  It is not always there in all the projects that we saw.

Q7   Mrs Helen Grant: Diana, you referred to incentives to encourage donors, providers, DFID, whomever, to focus on impact and not just numbers.  What sort of incentives would you be thinking about?

Diana Good: What I really mean is it is a frame of mind and a design and use of tools and processes with longer‑term, sustainable, beneficial work in mind, whereas, with the results agenda, I do not think anybody ever intended that it was going to have this effect, but it has had the unintended consequence that the message within DFID staff and with their implementers is that they need to be able to come up with short‑term, big, announceable figures.  I do believe that there is an absolute desire and I understand that our report is having quite an impact within DFID, because it is striking a chord with them.  It is not that they do not want to achieve the harder to reach, longer term or difficult to measure. 

What needs to happen is the message needs to get out and it would assist if the tools, processes and results framework were re‑geared to build in longer‑term stepping stones, milestones along the way.  The sense at the moment is that senior management will approve of what you are doing if you are pressing the value for money button, rather than the longer term.  Payment by results contracting is an example of that.  The NAO produced their very critical report in June.  We have been concerned about it and I know the IDC has been concerned about it.  Payment by results is meant to achieve better value for money and impact, but, unless it is being used in a sophisticated way appropriate to the particular context, the programme and what is achievable in that country, it can create some very perverse incentives. 

For instance, it encourages implementers to work with those who are easier to access near to the capital city rather than those who are much further away, harder to reach and more costly to reach.  It can build in rigidity.  As Marc has just referred to, Marie Stopes intend to reach the bottom quintile, realise that they are not doing it, do a course correction and decide to shift the programme.  If the payment by results contracting is too rigid, it can lead implementers to pursue courses of programming that they know are not achieving what ought to be achieved in terms of impact.

Q8   Mrs Helen Grant: Do you think maybe a more specific indicator of quality, success and impact would help?  There could be a tick in the box if some very good best practice, for example, was spotted, and, in reporting on that, people like you and us looking at the work of, say, DFID, at least could give credit.

Diana Good: Absolutely, and it may well be that it involves more complex messaging, more complex communication.  Quite clearly, if one unwraps the communication in terms of a big number ostensibly achieved speedily and in fact discovers that it is not reflecting the work that they are doing or it is creating an incentive to, for instance, get as many children enrolled on day one of the term but not receiving a quality learning experience, then that needs to shift.

Graham Ward: If the objective is long‑term sustainable impact, there needs to be a path to long‑term sustainable impact and what one needs to do along the way is to test if the project is on the intended path; if it is, to see whether there are good things to do more of; if there are bad things, to eliminate those and do something else.  If you are in this world of what gets measured gets done, then the key is to be really smart about what is going to be measured and to know the contribution that that has to the final objective.

Q9   Mrs Helen Grant: Yes, so, if the children are enrolled at school, we need to know if they continue to go to school in the months and years thereafter.

Diana Good: Yes.  One of the things I want to stress is that what we are talking about here is inconsistency.  It is not that there is not good practice; there absolutely is and obviously the more senior the manager, the more experienced, the more likely the tools are not going to get in their way.  If you are less experienced and less secure, the tools need to be designed to create the right incentive.  We have seen annual reviews, by way of example, which are done very well.  We have seen other annual reviews that are really very poor. 

Whatever the tools are, DFID staff need to be questioning the data in a very active way.  For instance, in Bangladesh, we saw UNDP cherry‑picking the data to produce good news results in terms of the data that was good and DFID staff were simply not seeing that.  In Malawi, we saw UNICEF producing data that looked very good, but, if you questioned it, it was very easy to see that they were producing good news stories rather than identifying that the numbers of women going into victim support units appeared to be dropping and were very few in the first place.  It is a question of experience and determination, not just accepting the data you are provided with but really interrogating it.

Q10   Fiona Bruce: I wanted to ask about learning across the organisation and how far you see, when you make recommendations like this, learning and good practice are shared.  Is that something that there needs to be much more of, because obviously you are saying there are some good managers, but there are clearly areas where it is not happening?  You have done other reports about learning, but how is DFID now going to take forward what you are saying and ensure that it is spread across the entire organisation?

Diana Good: Our report identifies a series of things that we think would make a real difference.  One was getting the goal right, making sure it is a context‑relevant, achievable goal.  Another was setting the longer‑term view, the kind of thing that we saw in Odisha with livelihoods and in Ethiopia with productive safety nets.  A third is coherence: break down the silo approach, look at a combination of health, education, livelihoods, if that is the area you are talking about. 

It is about conscious learning, to really address your point.  An enormous amount of effort has gone into programme design.  If you talk to DFID people, the bulk of their time is spent on what can be very burdensome design processes.  There needs to be more real‑time assessment and shorter learning loops.  The fact is that DFID has struggled to turn learning into action.  On BID, one of the things we had to do at the very beginning with our Business in Development report was to find out how much DFID was spending and what they were spending it on, because they did not have that information.  It took us two months to do that because they were not able to provide it. 

If you are not capturing these things, how are you going to start measuring them and learning from them?  That is why one finds a series of pockets of very good practice, but sharing that good practice horizontally and vertically—the centre and countries—is challenging.  That is not to say that this is not difficult; it is difficult, but one really has to have processes in place that work at this.

Graham Ward: We could have a BID example from Marc, if there is time, Chairman.

Dr Stephens: On the vertical integration, the link between the centre and the country offices, which is an ongoing challenge for not just DFID but practically every organisation that has an international spread, we did come across repeated examples.  For example, in Ghana, fantastic projects were going on on the ground, such as a multi‑donor water and sanitation project, WASH‑UP, involving Unilever, private sector, DFID funding and so forth.  This is a priority sector for Ghana and it had not been visited by the country office.  Similarly, the Discovery Channel investment in education, up in the north of the country near Tamale, doing great work, had not been visited a year after it started when we went there. 

We were hearing from the businesses involved that they would have liked to have that contact because, for them, part of DFID’s USP is precisely that contact with the local office.  Obviously, the response was, “We cannot do everything locally”, but it just means that corporately there are plenty of examples where it seems that the glass is half full, but it is not completely full because of that lack of join up. 

As Diana says, in terms of information across the organisation, yes, LEG and BID are not necessarily classified as a core DFID work stream, but it was very hard to get information about what was going on in health, education and in economic development and get an overview of all the activity.  Without that basic information, both in terms of spend and activity, it is very hard to draw lessons.  I suspect our report is the first time it has all been put together in that sort of form.

Diana Good: I will quickly mention two other things.  One, which has absolutely been a theme of our reports and our findings and certainly the impact report, is the importance of really good beneficiary engagement throughout the process, not just in some early consultation phase but consistent involvement.  Since our impact report, DFID has produced an interesting report on beneficiary feedback that confirms that—I will just quote a bit from it—DFID’s evaluation policy does not “provide an adequate framework for beneficiary feedback” and it concludes that it is time to move to “explicit and systematic application” of beneficiary feedback throughout the evaluation process.  It is vital learning.  These are the essential stakeholders.  We think that is very important and it is interesting that the DFID report is confirming that. 

Finally, and this is very much part of the SDGs, the global goals and the importance of data, really vital to all of this is that there have to be baselines.  Too often, what we have seen in DFID reporting, in annual reviews and so on, involves a lump sum figure of how many people that programme has assisted in this year, without a trajectory, without looking at what percentage of the total number of girls out of school is that lump sum number.  What was the figure last year?  What was it two years ago?  What is the trajectory?  That is the kind of information that it is absolutely vital that national Governments have.  We would regard it as an absolute priority for DFID to assist countries in creating those baselines. 

Q11   Fiona Bruce: Thank you for all of that.  It has been really important evidence.  Do I take it that very substantial sums of money could be put into a project but no one from DFID visits for a year or more?

Dr Stephens: No one from the country office will have visited.  There are multiple examples, whether they are the challenge funds or whether they are what we would call business partnerships.  There are many examples of where good work being done on the ground could be even better if several projects were joined up in one sector, all funded from the same challenge fund, or access to Government officials came through the local office, but it is not happening because of division of labour and burden of work and so forth.  It necessarily means that the full potential of projects is not being realised.

Q12   Fiona Bruce: These are quite substantial sums of money, are they not?

Dr Stephens: It varies.  Some of it could be a small challenge fund grant; some of it could be larger.

Q13   Fiona Bruce: When you say “small”, what do you mean?

Dr Stephens: The example of the Discovery Channel investment in Ghana is about £11 million, so that is not that small.

Fiona Bruce: Thank you.  That answers my question.

Chair: I would like to thank you all very much and thank you for bearing with us through the vote.  Marcus, Diana, Graham, Marc, that was very, very helpful to us.  I am sorry, we need to move on to the next panel now, but thank you for your time and for your very good answers.

Examination of Witnesses

Witnesses: David Kennedy, Director General for Economic Development, Tony Burdon, Head of Private Sector Department, and Malcolm Greer, Deputy Head of Private Sector Department, Department for International Development, gave evidence.

Q14   Chair: Welcome, gentlemen.  Our first question is from Helen Grant.

Mrs Helen Grant: Hello, everyone.  I have four questions in succession and it might be easier, because they are quite detailed, to take them one at a time.  The first question is: in which countries is DFID’s engagement with business working best and are there any lessons to be learned?  I do not mind who answers or whether you all want to say something.

David Kennedy: Let me make a start.  Can we come at that from a slightly different angle, in the sense that there are lots of different ways we interact with business?

Mrs Helen Grant: As long as you answer the question, you can take it from whatever angle.

David Kennedy: The first thing to say is we have relationships with business in this country, the big corporate relationships.  Is that working well?  In some circumstances, yes, and in others not.  For example, we have partnerships with Unilever that are resulting in action and then there are a lot of partnerships that we want to find a way to energise and give purpose.  That is one of the things we are doing here.

The second thing we are doing is engaging business here through a set of roundtables and, again, some of those work well and some of them do not.  The extractors roundtable, the energy roundtable and the agriculture roundtable have all resulted in action, but some of them, frankly, are talking shops and that is not what we want.  We are thinking about how we can do better there.  There are networks here that we want to engage with.  You can take the CBI as an example.  You can take Mark Malloch‑Brown’s new group, which is business working to implement the global goals.  Frankly, there is more that we can do there.

If you get into countries and what we do in countries, we work very closely with business, probably in a much more extensive way than is covered in this report, across the range of our programmes, whether that is economic development, which is what I lead, or whether it goes beyond economic development into health and education.  There is extensive interaction.  I do not think we can say if it is working better in Ghana versus Nigeria versus whatever country.  There is extensive interaction. 

One of the points raised in the report from ICAI is whether we have a good way of measuring how that interaction is working in practice and what results it is giving rise to as well, which is what we have just been talking about with the previous panel.  The answer is that we need to be much more systematic in that.  We can give you some good examples of where we are working with business, if that is what you would like to hear from us.

Q15   Mrs Helen Grant: It is about what elements make it work well.  It may well be the interaction, the communication or just the gelling.  I do not know.  I am just interested to know what factors make it work rather than the fact that some work and some do not.

David Kennedy: What does not work well is that we are expected to talk to the private sector, so we invite them in, have a very transactional conversation, tick the box that we talked to the private sector and then forget about it.  That does not work well, clearly.  What we are trying to do is, as I have said, engage with purpose.  We know what the private sector values from us, and we know this because we hired PwC and asked them to talk with a whole range of investors about what they are looking for from DFID.  They want to understand what we are doing.  They want to help inform what we are doing.  They want to be able to tap into our networks and they want to be able to work as a partner with us. 

If you use those things as a guide to how we should interact with meaning and purpose and you transplant that to the country level, we need to go out and find the private sector partners.  We need to not talk at them but talk to them, to be in listening mode and then to work in partnership to find what we can do usefully together.  That is basically the approach, but again, if you come back to whether it always works in practice, we cannot confidently say it always works in practice, so we need to be more systematic: systematic both in design of the approach and also then in monitoring the approach and the results, with all the complexities that measuring results entails.

Q16   Mrs Helen Grant: Making sure that there is something in it for both sides.

David Kennedy: Yes.

Q17   Mrs Helen Grant: What level of detail do DFID’s existing operational plans go into and is there scope for making them more detailed perhaps?

David Kennedy: Last time I came before this Committee to give evidence, we talked about the coherent strategic approach that we were trying to develop and we mentioned the growth diagnostic, which is our key analytical tool, which flows right through to the operational plans.  We have made a lot of progress now.  We had a tool the last time I spoke to you.  We had not used the tool; we had not rolled it out, but now this tool is being used by all of our country offices and all of our departments working on economic development to identify the priorities and where we can add value in this space.  That is the starting point. 

You go from the growth diagnostic to the period we are in at the moment, which is the bilateral aid review and the multilateral aid review, which are the strategic processes to say how we are going to spend the money over the next period.  We are right in the middle of that, so that will conclude in the next several months.  Coming out of that, we will have detailed operational plans about what we are going to do across the whole set of sectors where we are involved, including economic development and working with the private sector.  Those will be at a level of detail.  They are necessarily still reasonably high‑level, the operational plans, because they are strategies and strategies, by their nature, are high‑level documents.  Those need to be translated then into the detailed programming that we will do.

Q18   Mrs Helen Grant: Are you saying they are detailed enough really or does more detail need to be put in?  I am not sure what the conclusion was on that, David.

David Kennedy: Tony, do you want to come in and say what level of detail we go into in the operational plans?

Tony Burdon: I think sufficient.  It is a broad level of detail, but underpinning it is lots of analysis.  When we did the inclusive growth diagnostics, we looked at what was stopping growth in these countries, how we create more and better jobs and raise the incomes of poor people, what the barriers are.  Then we identified what we can do about that, what the opportunities are, what other donors are doing, what the Government is doing and, therefore, the niche for DFID.  In the spending review process, we will be given our allocations and a steer on direction, and that detailed analysis then forms the foundation of our business plan for each country or central department working on economic development issues, for instance.  That is quite a strong level of detail.

Mrs Helen Grant: Yes, thank you.

David Kennedy: It will say, “These are the areas we are going to focus on in the countries that have come through this process.”  Let us take an imaginary country: “We will focus on commercial agriculture, small business development, through whatever levers we will pull there, and energy, working with the Government and the regulators.”  It will be at that level of detail, the kind of objectives we want to achieve and then the programming will go into much more detail, including the results that would come from that programming. 

Q19   Mrs Helen Grant: You are happy with the working of your diagnostic.  It is working well, useful, good.

David Kennedy: Yes.  It would have been a leap of faith last time to say we were happy, because we had not rolled out the diagnostic.  One thing that was raised in the previous discussion was that DFID can be a bit siloed.  This was a process that had to pull together all of the different departments at the centre and in the countries.  In fact, it has worked well and it is a really good model, and we have external evidence for that.  For example, at the World Bank, top economists have looked at this and said it is state of the art, cutting edge, analytical, strategic thinking.

Q20   Mrs Helen Grant: Great, thank you.  I have a couple more.  You have pledged a large amount of additional funding to the CDC Group.  Why has this been done?

David Kennedy: Basically, we think it offers really good value for money.  It is important to say that the CDC has been on a journey over the last few years.  If it was five years ago, I do not think we would have wanted to recapitalise the CDC, but over the last few years we have completely replaced the board, first of all, then replaced the management and then replaced all of the staff.  There are very few CDC staff now who were there five years ago, so it is a new organisation in that sense.  It is a new organisation because it has a different geographical focus.  It works much more in difficult countries and not in China, for example.  It is no longer a funder of funds, which was its main business.  It does direct investments.  It does those both in equity and debt.  It is a completely transformed organisation.

Q21   Mrs Helen Grant: Just on that point, the last panel of witnesses mentioned, in relation to CDC, they felt that there was perhaps a lack of commercial investment experience.  What do you say to that?

David Kennedy: I think the point was that there is a lack of commercial investment experience in DFID rather than CDC.

Q22   Mrs Helen Grant: You are quite happy that, on the CDC board, there is sufficient commercial investment experience.

David Kennedy: Yes.  CDC is a commercial bank with a development mandate, but it is staffed by commercial bankers and the people on the board all come from a commercial banking background, so there is no shortage of commercial experience there.  I think there was a question there about how much expertise there was on our investment committee, which is the DFID investment committee.  There was also a question raised last time about how much financial and transaction expertise we have in DFID.  That is a really important point.

Q23   Mrs Helen Grant: In DFID, do you feel you have sufficient?

David Kennedy: It is not true to say we do not have any expertise; we do.  We have hired a small team of people with a financial background and we are in a good positon now to be able to oversee our investment in CDC.  There is a question going forward, which is: if we step up very significantly our development capital investment—and that is a likely outcome of the spending review, so the Treasury will look for us to do that—do we have the capacity in DFID to oversee that step up?  I think the answer is no; that is pretty clear. 

Then there is the question: what do we do about it?  Do we hire a load of investment bankers to work in DFID?  I think the answer is no; that is not practical.  We will contract out the transactions.  There is a question over whether we have sufficient capacity to do oversight of that contracting out and there are different ways of doing oversight: we could contract out the oversight of those transactions; we could hire people, and we probably will do a bit of both.  That is a little bit on hold until we know what is happening in the spending review, which we will find out in the next two or three weeks.

Q24   Mrs Helen Grant: Tony or Malcom, do you have anything to add to that?

Malcolm Geere: Part of my role as deputy head in the private sector department is resourcing the team and, as David mentioned, I can list at least four people over the last 12 months whom we have brought in directly from the financial sector, from Barclays Capital, from Standard Chartered, from PwC and from Ernst & Young, and it is not that easy.  We do not compete with the financial sector on terms and conditions in the civil service.  Having said that, when we have gone out to recruit, we have attracted good lists of candidates, so we know we can do it.  It takes a little longer than recruiting from within the civil service, but we know we can do it. 

We have also been finding other ways of bringing in expertise when we have needed to.  A director from GSK has been working in our health department on medicines.  We have an EY director who has been helping us develop the governance framework for our development capital portfolio.  We currently have under contract one of the members of the ICAI team who wrote part of the report on LEG and who is working on reviewing CDC’s strategy for us.  We know we can access skills at short notice, but also longer term we know we can bring people in.

David Kennedy: Do you want to mention Ernst & Young when we asked them if we have the capacity to oversee what we are doing?

Malcolm Geere: Last year, Ernst & Young conducted a skills audit of the whole of DFID, to identify where there were people with specialist financial sector expertise or investment expertise, or suitable accountancy or audit experience.  They looked right across the organisation and identified the people, and it was not a vast number—it was in the dozens rather than in the hundreds—but there were people within the private sector department, in our central finance section, in the climate and environment department, which also manages some investment programmes, and in DFID India, where they do not have the difficulty we have in recruiting people on civil service terms away from the financial sector. 

That report concluded that we were suitably staffed for the stage of rollout of our development capital plans, which we took to mean that, if we had gone through an extensive recruitment process at that point and brought a load of financial sector experts into DFID, they would have been underemployed; they would not have had a lot to do at that stage.  We are aware that, if plans change after the spending review, we are likely to need to recruit more people, but we know that there are ways that we can do that.

David Kennedy: If we get a big target in this area from the spending review, you would want to make sure that we are resourcing adequately to that and at the moment we are saying, frankly, we are not adequately resourced for that step up in the future.

Q25   Mrs Helen Grant: But you would accommodate that, should you need to.

David Kennedy: We would not want to step up if we did not have the capability to oversee what we are doing, no.

Mrs Helen Grant: No, I know, but I said if you need to.

David Kennedy: Right.

Q26   Mrs Helen Grant: In relation to the work that DFID is doing with business, is there added value happening in terms of what business would have done in the first place?  I am probably not explaining myself very well.  Obviously DFID works with business; yes, there is added value, but is there value over and above what business might have done initially?

David Kennedy: Let me have a quick go and then I can open it up to Tony and Malcolm.  If we are talking about additionality, the specific thing that you touched on in the previous session, it is absolutely central to all of our thinking about development capital investment or what the report called LEG—loans, equity and guarantees.  Clearly, if the private sector will do something, then we should not be there.  We should not be displacing private sector activity.  That would not add any value from our perspective and so we are very careful.  For example, a lot of our discussions with CDC are about additionality. 

Now, it happens to be the case that, if you push CDC to invest in more and more difficult countries, the likelihood is they will be additional, because there is a lack of patient capital in the more difficult countries and that is how we are pushing CDC.  That kind of says not that you can be complacent about additionality, but probably, in many cases, you will be additional.  A lot of the discussion we have with CDC is about how we can measure our additionality.  How can we have checkable stories that say, unless we were present, this would not have happened?  We have had the same discussions with PIDG and we have a set of criteria that are used for all of the CDC investments.  That discussion on additionality is not finished.  We are going through a CDC strategy review and one of the key elements of that will be additionality and ensuring that they are only doing things that the private sector would not do. 

There is a different question, which is about how what we do unlocks private sector activity that would not have otherwise happened, which is all the work we do in our country programmes.

Tony Burdon: One of the biggest constraints is the lack of investible projects in many of the countries that we work in.  Our job, if we want to end poverty in those countries, is to get more private sector activity.  With CDC or with development capital, we can try to push it into more risky areas where the private sector is not present now or is unwilling to go, show that it is possible and then attract the private sector into those areas.  That is the challenge.  Alongside that, there is our other complementary activity, which is improving the regulatory environment, cutting red tape, helping Governments get the right environment for the private sector, tackling governance challenges, helping to get the infrastructure in place so that the private sector is more readily able to go into those areas.  There is a range of actions across the piece that we need to do, but we are working in some of the hardest countries in the world where the private sector is very loth to be active, so our job is to try to pull them in through example.

Malcolm Geere: We have been talking about additionality in the investment space, but there is also a really good example in terms of some of the other ways that we have been engaging with business.  The much touted example of M‑Pesa in Kenya came from a challenge fund grant to innovators within Vodafone who had an idea but did not have the space within their organisation to roll that out.  That was a major success story.  That was 15 years ago or so.  Things have changed now.  The space has changed.  There has been a big autonomous movement within multinationals and other businesses to develop profitable solutions to development challenges as a way of accessing new markets. 

That does not necessarily require grants through challenge funds.  That requires a different approach, so we have moved away from the challenge fund approach and are now reconsidering what our niche is in terms of encouraging innovation, encouraging firms to scale up that innovation and then other firms replicating it.  We recognise that that is a changing space and the additionality question today is not going to look the same in five years’ time.

Q27   Chair: Could I ask you a final question before we move to the next panel and the next report?  Are you confident that your strategy in working with business in development is overcoming and continues to overcome extreme poverty, because in the end that is what you are about?

David Kennedy: I am confident that it delivers that.  Am I confident that it is as good as it could be?  The answer is no.  We have to do better and there are a number of things that we are very actively involved in at the moment.  We are developing a new business engagement strategy, which we will launch in the next month.  That is the intention and that is looking at how we can engage again with meaning and purpose that leads to impact rather than talking shops and box‑ticking.  That is the first thing. 

Secondly, we will publish an economic development strategy including the results framework.  The intention is to do that in the spring.  Again, we are working through what the portfolio shifts are that will give us better value for money, so moving to more of energy, urban, agriculture.  There are a number of active areas where we are trying to push the boundaries and get better in order that we can have more impact.  We are not there yet.  We want to do better, but we are in a reasonable starting position to do that.

Chair: Malcolm, David, Tony, thank you very much indeed.  I am sorry we are out of time.  Can we move on to our next and final panel, please, on DFID’s approach to delivering impact?

Examination of Witnesses

Witnesses: Les Campbell, Director for Value for Money, and Peter Taylor, Deputy Head of Better Delivery Department, Department for International Development, gave evidence.

Q28   Chair: Gentlemen, thank you very much for joining us today.  This is our last session before we have to vote at 6 o’clock and Fiona Bruce is going to ask you the first question.

Fiona Bruce: In its report, ICAI says that you need to have a results framework that better captures and measures transformative impact.  Assuming you accept that and, therefore, the need to review your processes to encourage longer‑term and harder‑to‑measure results, I wonder whether each of you could give me some examples of where, since the report was produced, you have done this and tell me something about the way that you have changed the wording of the goals involved.

Les Campbell: Thank you for that and good afternoon.  Perhaps I could put the results framework in context.  The overarching context is the single departmental plan, which we are in the process of revising and reviewing as part of the spending review.  That sets out the objectives for the next five years.  That is the very big picture, and then the results framework is something that we have in place, which we are revising just now, to look at how we can aggregate specific measures.  That is largely numerical, because it is adding things up that we can aggregate. 

Then, if you want to drill down into the impact, you have to get into the specific programmes and projects.  That is where some of the wording can change and pick up more of the qualitative and the long‑term nature of what we are doing.  That is more within the projects and the programmes element, as well as in that hierarchy of moving up through the results framework into the overall single departmental plan.  We are looking at the results framework and taking on board the comments from ICAI, and we will be launching that over the next few months, once we have the single departmental plan in place, so it is still a bit of a work in progress at the moment.

Q29   Fiona Bruce: That report was in June, so you are saying you still have not changed your processes as a result.

Les Campbell: No.  We have been reviewing the process and we welcome the report and the comments on the results framework.  We have not finalised the outcome of that framework until we have the single departmental plan in place. 

Q30   Fiona Bruce: You say you welcome them; do you accept them?  Do you accept the comments made by ICAI in the report?

Les Campbell: Broadly, we do.  We accept that we need both the long‑term qualitative as well as the long‑term quantitative.  I do not think it is an either/or; we need both.  We also need to focus on the results and the short term, because we have the process of adaptive programming, so we are constantly looking for feedback and changing what we do in the short term, but with a view to moving and achieving our long‑term objectives.  There is a bit of both, and that is what we are changing.  That is what we are moving towards, and you will see that more clearly when we issue our new results framework in the new year.

Q31   Fiona Bruce: When exactly in the new year?

Les Campbell: It will probably be March, April time.  We will issue our single departmental plan probably in December together with other Government Departments and then we are working through, as David Kennedy referred to earlier, the bilateral aid review and the multilateral aid review.  The outcome of that will come together and will help to inform the new results framework.

Q32   Fiona Bruce: That is nine to 10 months from when this report was published.  Do you think that is an adequate sense of urgency?

Les Campbell: In terms of what we are doing to amend the process and the thinking, we are doing that now, but we do only issue a new framework relatively infrequently.  We were not planning on revising and reissuing our results framework, but the thinking that goes into it and looking at bringing in the qualitative and the quantitative is happening now.  It is just that you will not see the outcome in the new framework until next year.

Q33   Fiona Bruce: How will you ensure that there is a less patchy quality of oversight, which you will have heard the evidence refer to earlier?  How will there be learning?

Les Campbell: Over the last year, 18 months, we have brought in Smart Rules, which I think you have heard reference to before.  That is our operating framework at the programme level, and I will ask Peter to talk a little more about that.  There is an extensive array of training and communication for our senior responsible owners who are running each of the programmes to make sure that the quality is sufficient, to make sure that learning does transmit across the organisation.  We are putting in improved training and there is internal communication; we are picking up learning, picking up beneficiary feedback, and that is a constant process.  

Peter Taylor: As you say, it is a long‑term process.  We are trying to get the systems right, the rules right.  Les mentioned the Smart Rules.  Underneath the Smart Rules, there is a whole series of separate guidance, for example, to help DFID staff write business cases, to think about how they monitor programmes. Some of the former commissioners mentioned the recent documents that have come out to try to improve the way that we do things: for example, to improve getting beneficiary feedback through evaluations.  We are also trying to step up our training at all different levels—senior responsible owners, all our programme managers as well—and to provide real learning across the organisation, as you have touched on already, so that we can pick out pockets of best practice, help to identify areas where things are not going quite so well and work in a more intensive way with those people.  It is a question of structures, incentives, processes, materials, support, and it is an ongoing process. 

Q34   Fiona Bruce: Do you think it was acceptable that an investment of £11 million was not visited for over a year and that that was one of the smaller investments that was not visited for over a year?

Peter Taylor: It is worth saying, as you are probably aware, DFID has 1,600 or so live programmes at any time and, within that, there will be subcomponents.  For example, particularly with the centrally managed programmes—if I understood correctly, this was an example of a centrally managed programme—there may be dozens of individual projects underneath that programme.  It is difficult for DFID staff to go out to every single one of those more than, say, once a year.  Every single programme will be subject to a rigorous annual review, which will have either DFID staff involved directly in getting the feedback and going out to visit the project or the programme or that being subcontracted and then DFID staff reviewing the quality of that review. 

Yes, in an ideal world, DFID staff would get out to see every single programme on a regular basis.  For programmes managed in‑country, that happens much more frequently.  Most country offices will have a clear monitoring strategy with an agreed level of frequency of visits to programmes, but obviously for, centrally managed programmes where there may be dozens and dozens of individual projects, perhaps, within even one country, it is harder for DFID staff to do.

Q35   Fiona Bruce: How are you ensuring that, where there are these centrally run programmes, DFID really does have a hands‑on understanding of what is happening?  You are saying that you are sending out subcontractors, but that is not the impression I had from the previous witnesses.  The impression I had was that the programme had not been not visited at all.

Les Campbell: There is a range of ways that we monitor and evaluate the programmes, and each programme, in the business case, will have some thought given to how we do that.  To an extent, it is horses for courses and what is most appropriate, but each one has a way of monitoring and evaluating it.  That might be visits; it might be meeting whoever is running the centrally managed programme at the centre; or it might be getting information and feedback from beneficiaries.  There is a range of ways of getting that data in and monitoring it.  There are formal annual reviews where we, in effect, force the programme owner to report on that as well as doing that during the period.

Q36   Fiona Bruce: Are you satisfied that that is happening in an acceptable way or are you reviewing that as well, following this report?

Les Campbell: We are reviewing.  We are in a learning mode.  As Peter said, we are running 1,600 programmes.  The Smart Rules is a relatively new process for us; we are 18 months into that.  We are constantly reviewing that to see if the processes are working, learning from that and feeding that back in again.

Q37   Fiona Bruce: How many programmes have you reviewed and found to be completely unacceptable, so you have gone in and looked at reviewing fundamentally how they are undertaken?

Les Campbell: We have closed a number of programmes—I do not know if Peter has the detail—as a result of that.  Some programmes are just not acceptable.  They are not delivering or they are not functioning in the way that we want them to function.  Part of the value for money approach is to say there might come a point where we just have to say enough is enough, it is just not working and we close it.

Peter Taylor: As I say, every programme will be subject to a full and rigorous formal annual review.  There is a five‑point scoring system for that.  Anything that scores particularly the lower two scores within that five‑point scale will be subject to increased scrutiny and the senior responsible owner for that programme will be asked to come up with some improvement measures, which will have to be set out in what is called a delivery plan, which monitors the implementation of the programme. 

If a programme continues to score badly, continues to be off track in terms of the stated objectives and milestones against the outputs, a decision can be taken to close it.  If a programme no longer represents value for money or we consider that it will not continue to represent value for money, we will close it.  There are plenty of examples of that.

Q38   Fiona Bruce: Are there traits that you have discovered, if you are closing programmes?  I am interested to know how many you have closed.  Do you know how many you have closed on this basis and what the traits are?  Obviously, one needs to learn in the sense that you do not want to have more programmes of that nature put in place, do you?

Peter Taylor: We are just in the process of reviewing the figures at the moment.  It depends what you mean by early closure.  Sometimes they can be closed a month or two early; sometimes they can be closed a year or two early.  It depends what your cut‑off point is.  In terms of closed before the expected closure date when the business case was agreed, I think it is several hundred in last few years.  We are just in the process of going through those.  Some of them are quite marginal, as I say, by a month or two, so it is probably not a significant early closure.  There are others.  If you go to more than six months, then we are probably talking about dozens rather than hundreds.  We are going through the numbers at the moment.

Q39   Fiona Bruce: How much money has been invested that you have decided has not produced value for money, say over the last year?

Peter Taylor: I would have to check those figures for exactly what that would show, but the initial preliminary analysis we have done suggests that we have stopped spending several hundred million pounds that we might otherwise have spent if those programmes had continued, because we decided that it was not good value for money. 

Are there any traits, you asked, that may characterise those?  It varies.  A lot of these tend to be in the most fragile, conflict‑affected states.  For example, it might be somewhere like South Sudan where suddenly conflict flares up and what you thought you could deliver you can no longer deliver, so it does not make sense to continue with that.  There are lots of reasons.  Sometimes programme closure is because implementing partners are not performing.  That is pretty rare, but it can happen.  Otherwise it is external circumstances that mean you cannot deliver what you originally set out to deliver. 

Q40   Fiona Bruce: You are in the process of producing a report on this.

Peter Taylor: We are just doing some internal analysis of that to get a better sense.

Fiona Bruce: Chair, it would be very interesting to receive a copy of that analysis.

Q41   Chair: I hope that you would let us see a copy, but I wonder also whether you could perhaps write to us when you have the figures collected.  I appreciate the qualifications you have put on the reason for closing programmes, but nonetheless it would be interesting to this Committee and to the full International Development Committee.

Can I ask you perhaps a last question here, unless my colleagues have anything else to add?  We understand that DFID is exploring options for more flexible, adaptive programming.  I wondered how that will fit in with existing result management processes.

Les Campbell: This is part of our Smart Rules approach, which is that adaptive programming.  When we are designing a business case, we are constantly looking to see how we might amend that as the project takes place.  We are looking at what milestones we might look at, which is the results aspect of it.  At what point would we get an indication that something is going off track?  What measures would we see?  That is the results part of it that then allows us to adapt and to tweak what we are doing, to change what we are doing to keep it on track, recognising that there is normally an endgame in the project that we want to get to but there may well be different ways of achieving that. 

That is at the programme level, not at the results framework level, which is a higher level, but within the programme.  Whether that is qualitative or quantitative results, what are those milestones that might give a signal that it is not working in some way?  It is not the annual review, because the annual review quite often, frankly, would be too late for that.  We would be looking at milestones in a shorter period, as part of that monitoring and evaluation process that I mentioned earlier, when we are designing the programme, to say what it is, how we would monitor that, what signals would alert us to needing to change and to put in some adaptation to the programme that we have set up.  It is trying to get the link between the results and the adaptation.

Q42   Chair: Can you just tell us what log frames are?

Les Campbell: That is logical framework.  Your next question is: what is a logical framework?  That is the results chain.  How does a results chain work for a particular programme?  Ultimately, it is getting to what the impact is, but then breaking that down to say what the outcomes and outputs are.  It is based on the theory of change.  We are going into some programme for a reason, because we think this will have an impact on the lives of poor people.  Well, in what way?  The log frame then sets that out. 

The impact might be to improve infant mortality, and you could have measures for that.  The outcome might be number of children inoculated and the output might be number of nurses trained to do that.  It is working through that chain from the inputs we are putting in, which is primarily money, the activities taking place, the outputs that are delivered, the outcomes they achieve and then, ultimately, at some high level, the impact.  Most of that is related to DFID, but when you get to the impact stage there are attribution issues.  It might not just be DFID’s activities that are having an impact; it could be other partner countries or other partners.  That is the log frame.

Chair: Thank you very much for explaining that. 

Q43   Fiona Bruce: I just want to go back to evidence that you gave earlier.  You said that over several years you had abandoned quite a large number of programmes.  Presumably, bearing in mind that a small programme was described as £11 million, we are speaking of hundreds of millions of pounds of abandoned programmes.  Is that right?

Les Campbell: That probably will be the case and it would be correct.  Each year in our annual report and accounts we have to disclose what are termed as “losses”, and that is a much smaller number.  That is wasted money, if you like, and that would not be in the hundreds of millions; that would be a much smaller number that will go public in the notes to our accounts each year.

Q44   Chair: Abandoned programmes are not necessarily wasted money.

Les Campbell: Some of it might be.  I would be surprised if it was entirely wasted.  It is simply going off track or has not achieved what we had hoped it would achieve.  It would be wrong to say that is entirely wasted.  That would be unusual.  It maybe did not deliver as much as we had hoped it would and we do not see any sensible way of rescuing it, so at that point we would draw a line, start again and think about what we need to do in this country or in this sector. 

Q45   Chair: Presumably, as Peter explained earlier, it is not necessarily because they were badly thought out programmes; it is because the circumstances on the ground may have changed with conflict and with political change in those countries.

Les Campbell: That is more likely.  Out of the 28 countries that we operate in, 21 are defined as fragile and conflict‑affected states and we are moving more and more into that type of work in those areas.

Fiona Bruce: Perhaps this is something that we could revisit after the report.

Chair: Yes.  Peter, Les, thank you very much indeed for your time this afternoon.  Thank you to all our witnesses today.  Apologies that we have to finish slightly early, but we are very grateful.  It has been most helpful.  Thank you to Daniel Whitford and Paul Hampson as well.

              Oral evidence: [Inquiry name], HC [XXX]                            22