International Development Committee
Oral evidence: Sub-Committee on the Work of the Independent Commission for Aid Impact – ICAI’s Review on Conflict, Stability and Security Fund's Aid Spending Review, HC 1024
Wednesday 20 June 2018
Ordered by the House of Commons to be published on 20 June 2018.
Members present: Paul Scully (Chair); Richard Burden; Pauline Latham OBE; Ivan Lewis; Lloyd Russell-Moyle; Henry Smith; Stephen Twigg.
Questions 1 - 63
Witnesses
I: Richard Gledhill, Lead Commissioner, ICAI; Willem van Eekelen, Team Leader, ICAI; Melinda Simmons, Director of Implementation for Cross-Government Funds, National Security Secretariat; Dr Christian Dennys, Acting Head of Joint Programme Hub, National Security Secretariat.
Witnesses: Richard Gledhill, Willem van Eekelen, Melinda Simmons and Dr Christian Dennys.
Q1 Chair: Good morning, everybody. Thank you very much for coming. As has become the norm in this Committee now, we tend to work with ICAI and the respondents from the Government together. It just makes for a more meaningful conversation. I think this is the first time, Melinda and Christian, you have come to one of these Sub-Committees since I have been chairing it in recent months, so thank you very much for coming. Welcome again to Willem and Richard.
I wanted to ask a question about methodology of the report, but if you do want to make any introductory comments as you go through, please do feel free. Richard, your methodology involved looking at 34 projects across the six country and regional programmes. On what basis were they chosen, those projects and countries?
Richard Gledhill: Perhaps I could start with a bit of context. CSSF is a huge fund. It is spending over £1 billion a year, of which around 35% to 40% is classified as aid. It is working on complex, often high-risk issues in very challenging environments. It is also a new fund—it was set up in 2015. Importantly, it is building on a long track record of cross-Government funds addressing conflict and fragility, most recently Conflict Pool. We decided to do a full review on the performance of the fund and to score this. The score we gave was amber/red. This contrasts with other, newer cross-Government funds where we did a rapid review, which was not scored. That approach was very much justified because of the scale of spending and the track record that the Government have on working in these areas.
Because we were restricted to ODA and because we could not look at the full range of activities of the fund because of its scale, we decided to adopt a case study approach, based on countries and regions. We looked at the country portfolios of Colombia, Iraq, Jordan and Pakistan, and the regional portfolios of the Caucasus and the Sahel, which together represented around 12% of the CSSF’s programmable ODA expenditure in the two years covered by the review—the first two years of operation of the fund. We felt that it gave reasonable coverage of the diversity of the CSSF programming, both in terms of geography but also the issues it was addressing and the delivery channels that it was using.
Q2 Chair: Do you not think the fact that you only visited a few of the countries compromised your findings?
Richard Gledhill: Clearly our findings relate primarily to the countries we visited but we also looked at the fund’s systems and processes. I am confident that the conclusions we reached and the recommendations we made are valid for the fund as a whole—for the ODA component of the fund. Willem, do you want to comment?
Willem van Eekelen: Yes, a little bit. There are also lots of findings that did not make it into the report and those are typically country-specific. Everything that is in the report is well evidenced from multiple places. Yes, we are sure those findings are robust. In addition to that, when we go to a place, we send a little report after it reflecting on what we found in that particular country, which would not normally find its way into the report.
Q3 Lloyd Russell-Moyle: ICAI found that CSSF often focused on providing training, including in situations where training is not actually required. Why does CSSF focus so heavily on providing training like this?
Dr Dennys: Thank you very much for the question. The CSSF highlighted a number of ways in which we deliver some of our programming. The training components are often identified as a result of needs identified by the host Government, often in conjunction with allies and partners across the Security Council or across the European Union, to deliver support to those specific countries, for example in Mali. The challenge with training is providing support to those countries to enable them to deliver either their policing or their military capabilities without having that training and support given in the first instance. We accept that, but because it is often a very difficult environment in which we are delivering, the training inevitably provides a valuable resource for the recipients.
Richard Gledhill: First of all, the over‑reliance on trade is not just a CSSF issue. This is a problem with many donors and multilaterals. What you often find, and certainly what we found in Jordan and Mali, is that you get a small, select group, often from the elite, who are getting trained on everything by everybody, and the vast majority do not receive any training at all. That does not necessarily reflect the needs of those individuals or the organisations. From what we saw, often training is used as a quick fix. We saw some good examples of training. For example, in Jordan we saw where training was used in community policing in refugee camps, where it is part of an overall package of approach involving mentoring, new procedures and institutional reforms. In that sort of context, training is great and important but, in other places, it is just used as a quick fix without really having a sense of where you are going or indeed measuring whether you have got there.
Q4 Lloyd Russell-Moyle: Would it be fair to say that the entire counter-extremism strategy is about trying to talk to groups and train groups rather than actually dealing with the underlying motivations those groups might have for injustice, discrimination and poor governance, or does the training deal with those concrete issues?
Richard Gledhill: I should probably ask CSSF to respond to that.
Dr Dennys: The majority of the training that ICAI would have looked at in those countries is not focused on countering extremism. To pick up on the Jordan example, which is about building the capacity of community police within refugee camps, which are clearly very vulnerable and at-risk populations, other types of training was provided to the Jordanian police to help them address civil unrest in that programme.
Q5 Lloyd Russell-Moyle: I am not just interested in the examples; I am interested in the whole programme. One of the CSSF’s objectives is to counter entrenched views in areas such as violent extremism, gender and social security. ICAI found that the influencing work was “insufficient to stand a chance of countering deeply entrenched views in counterpart institutions”. That is what I am trying to get at. Is it able to change the underlying causes of entrenched views or is it very superficial? ICAI seems to suggest it is very superficial.
Melinda Simmons: It aims to hit both. It aims to hit every level but how it does it is going to change from country to country because the context for the sources of extremism will vary from country to country. The way in which the programme chooses, whether it is training, media broadcasting or engaging with communities on governance activity, will be informed by the analysis that those teams do—usually the JACS analysis, which has now become the norm in the CSSF. That is where the choice has come from.
Where training is picked that looks like it is quick-impact, it is usually related to diplomatic activity going on elsewhere, or it is piloting something in order to create the space for the UK to do something on top of it or in addition to it. There are no random choices on either training or other elements of counter extremism programming. They are grown from the analysis that teams need to do in order to make the bid for the programme.
Dr Dennys: Specifically on countering violent extremism, ICAI recognises that analysis produced by the CSSF has informed HMG approaches in general. Specifically on CVE, work has been undertaken in some countries that has concluded that there is not a programmable option in that country in that context. There are active choices not to do things on the basis of analysis or evidence, as it is identified.
Richard Gledhill: Looking beyond the issue of training, it is fair to say that some of the better programming we saw was on strategic communications and the use of social media, which takes them into countering violent extremism. Interestingly, we were not allowed to publish two of the best examples we saw of this because of security concerns for those involved in the delivery—we have two examples in the report but we saw other examples as well. This was an area of strength for the CSSF. We did not see that level of quality in design or delivery across the piece though.
Willem van Eekelen: I have three things to say on training. The second and third link to other issues as well. First, there was just an over‑reliance on it. The CSSF actually recognises that in its internal reports. The learning reports say, “Guys, there was too much training. Make sure that you actually need it”.
Secondly, the reason that there is so much training is in part that that is not actually the objective of the programme and the intervention—the objective is to open diplomatic doors, which is conceivably a good reason to do training. You then would need to think, “Do you want to train 10 people, 100 people or 1,000 people?” but conceivably if it does open diplomatic doors, that could be a useful thing to do. Then it is an implicit objective, so it is not normally there in any of the documentation we could see—“This training is not really meant to achieve this; it is really meant to achieve so and so. There is no attempt as far as we could see to measure the results, which was a concern. Training is used for a purpose that is implicit and that is not being verified.
The third thing I want to say is that, on the issue of counter-extremism and counter-terrorism, you may need training, but you definitely need some sort of policy or framework within which you design your programmes. That framework is not there. You then have Pavlovian responses. You need to do something, you do not have a real framework within which you do it, so you resort to what you know, which is training.
Q6 Lloyd Russell-Moyle: You talk about that in terms of theory of change—a lack of decent theory of change and a lack of decent programme. Some of the bidders for the programme report to me that they are not even allowed to see the priorities in country, and have to guess what the priorities are in writing their application. There is almost a Kafkaesque kind of situation where they are not told, they have to guess or they have to get it through whispers. Is that something you saw on the ground as well in terms of the priorities you are talking about?
Richard Gledhill: The issue of transparency is a challenge in CSSF. Clearly they are dealing with sensitive issues and classified documents, but in our experience and in the case studies, the decision as to whether information should be shared is taken by officials at a local level—they tend to err on the side of caution using confidentiality as a bit of a comfort rag, which means that information that could be shared is not always shared. We saw good examples where declassified versions of documents were shared. If there is an issue about sensitivity of information, then that is a good approach but it was not being widely adopted.
Melinda Simmons: Can I comment on that?
Lloyd Russell-Moyle: There will be some more discussion around that later on, but if you want to now very briefly you can, but there will be a longer bit about confidentiality stuff.
Melinda Simmons: Sure. It is a simple thing that country strategies, regional strategies or thematic strategies that set out what the CSSF will do are not just about CSSF—they cover the whole of Government activity. That is why, for the most part, they are classified. In fact, suppliers do see programme documentation. What they find it more difficult to see is that whole-of-Government strategy. In the last year, we have started to see how we can disaggregate CSSF-specific activity by country so that there is more transparency for suppliers.
The broader point I want to make, which is something that was picked up by ICAI, is that this is newer than ICAI has portrayed it. It is very different from the Conflict Pool and I am sure we will talk about this later. In setting up something that both built on ICAI’s recommendations on the Conflict Pool and fulfilled this wider remit, we took it in three stages. We set up the systems, which is challenging enough when it is involving five Departments and seven agencies. We built the capability and now we have over 400 staff trained, and we got to a point where, in the last year or so, we have begun to look more forensically at the programme cycle.
The issues that have come out that ICAI have highlighted for around the programme cycle, which I know we will go into in more detail, indicate the point at which we are now at. Having built this, there is a really strong cultural issue for Departments, including around sharing what they consider to be classified material. The idea for people who are not just potentially used to spending ODA but who are not used to running programmes in a cross-Whitehall way means they do have to challenge their own perceptions around things like transparency and accountability. That is why in this last year we have seen more shared even than at the beginning of the ICAI review. We are on trend at the moment.
Q7 Lloyd Russell-Moyle: I do not want to go too far down the rabbit hole about sharing information, because other people will come to that. ICAI found that one of the central commitments is gender but it is sometimes not mirrored at a programme level, and is sometimes lost during implementation. Rather than just talking to men with guns that we support about other men with guns that we do not support, which is sometimes what it seems the programme does, how are we making sure that women’s roles in peace and security is maintained throughout the programme and really embedded in detailed programme activities?
Melinda Simmons: This may sound counterintuitive given your question, but I am quite proud of the gender work we have been doing. We encounter the same issues as, for example, DFID does in doing this work. It needs sustained leadership. Our ambassadors, our Ministers, our directors general and our national security adviser need to be messaging to our programme leads that this is an important thing. Of course, in the case of the CSSF, we have a legal obligation under the International Development Act to ensure that we are programming in a gender-sensitive way. We have done well in the last year in getting that message across. The issue then is to keep reinforcing it.
There are parts of our portfolio where that has really been quite transformational. The Middle East is a particularly good example. Gender is now taken at every Middle East board when they are looking either at the whole portfolio or they are interrogating a head of mission and ambassador over his programme. It has become a routine thing. The leadership piece has done well.
The programming piece is about a combination of getting the right capability and the right form of technical advice into the right place to inform programming at the right time and then to ensure that that technical expertise is available through the programme cycle. The stabilisation unit, for example, which gives an awful lot of help on gender, does not have enough breadth—literally enough people—to be able to provide that in a systemic way through the programme cycle. This is something we are having to hire for additionally.
In the meantime, we have been undertaking gender audits—Eastern Europe has recently done one and the Middle East has done one for itself. The Eastern Europe one has been really useful for highlighting the gaps that the previous analysis has not quite picked up and has helped to set out for them the scale of the issue. This is not just in ODA programming of course, but across the complete range of what we do—military training, et cetera—where the stabilisation unit is now helping them with a plan and the joint funds unit will help by sourcing the expertise.
We have come quite a long way, actually, in getting across to people that this is an integrated piece of what they need to do, not just the next new thing. The challenge now is to square the capability and put it in at the right time so it is sustainable through the programmes.
Richard Gledhill: I would support a lot of that. There is a strong commitment at the centre of leadership from the centre, which we comment on in the report. The weakness where we saw it was at the programme level. There is a common theme from the response to the two last questions about there being a lot to do, it being difficult and it taking time. I have a problem with that because this is a really big fund that is spending a lot of money. A lot of the experience—not all of it, but a lot of the experience—from past work in conflict-affected states is relevant. There is a huge track record in DFID and elsewhere in Government of managing programmes. If we are going to spend this amount of money we should be drawing on that, using that and using it now, not just thinking about using it once we have systems and processes right in one or two years’ time.
Q8 Lloyd Russell-Moyle: In your analysis, if an external programme to Government run by Oxfam or someone else had such a large amount and similar reports to what you are getting, would the Department shut the programme down?
Richard Gledhill: Whether to shut programmes down is an issue for Ministers, not for me. We raise a lot of important issues in this report and made some important recommendations. There is a need for more urgency in the response to those recommendations, but I am sure you will come onto those in further questions.
Q9 Lloyd Russell-Moyle: ICAI found that in countries where CSSF works, the Governments in country are often unwilling to take on the projects after the funding ends. Given this, how do you ensure that the programmes are sustainable rather than just flash-in-the-pans?
Dr Dennys: We have to remember that these are very fragile countries and the Governments often do not have the capacity, the capability or at times the money in order to be able to sustain some of the activity directly. As we have discussed already, some of those activities are there to enable other parts of Government to achieve ODA-eligible outcomes or for allies and partners to help build capacity or the stability for that Government.
The sustainability of a CSSF programme is not just about whether that particular piece of training emerges in the Government. It is also about whether other parts of the international community come in behind us and provide additional financing. We have seen additional funding in Egypt where the World Bank has come in to announce £500 million worth of funding on education after a CSSF programme. It is also about how that is embedded within the wider international community’s response to addressing conflict and fragility in that country. That will include somewhere like Somalia where there is an international compact against which a number of donors and partners are working, which the CSSF contributes towards.
Q10 Lloyd Russell-Moyle: You are suggesting that it is a financial reluctance to continue or inability to continue? Some of the discussions I have had with NGOs and representatives of Government are that it is an ideological reason that they wish not to continue—they do not believe that the programmes are achieving the change that is necessary on the ground. Is that something you have heard from NGOs or Governments?
Melinda Simmons: I have not heard that in my discussions with NGOs and we have met with them quite frequently, actually, via Bond formally, in their conferences, but also informally.
Q11 Lloyd Russell-Moyle: Bond was very damning of the programme, were they not, in their latest communique?
Melinda Simmons: The Bond group?
Lloyd Russell-Moyle: Yes.
Melinda Simmons: It is interesting, because I have spoken at their donor group and I spoke at their annual conference to quite a packed room. On both times, they were pretty constructive, quite wide-ranging and quite honest discussions. We also had a transparency panel including Bond but also with some of the large development NGOs. We have been in constant touch with them throughout the year. We have come at this with our NGO suppliers as a shared challenge around the question of how, once you establish the programme cycle systems, you ensure that your partners are able to access the conversation around the choices of programme in the process. That has been quite a live conversation.
When the JCNSS put out their review, Tamsyn Barton, who is the head of Bond, blogged quite constructively on that review, of which, by the way, we have implemented nearly all of the recommendations now. I have seen that as quite a positive partnership with them, and one that we want to continue.
As for the ideology, I do not recognise that—I do not recognise an ideological a reason behind not continuing the programme. CSSF differs from departmental activity, which it is meant to complement. It is not a standalone mechanism; it is there to do things that Departments do not have as high a risk appetite for or where they need to collaborate, because it is supposed to be a mechanism that drives collaboration. It goes into countries where DFID potentially may not have a large presence, for example, so it needs to be complementary.
It also needs to be very agile so it will quite frequently flex itself, where it can see that potentially the political environment makes it difficult to achieve strategic effect with the programmes that you are trying to deliver. It may change. That said, over 80% of the CSSF is programmed for multi-year, so that would be a small minority of the portfolio that will be flexing itself within in-year.
Q12 Lloyd Russell-Moyle: In the internal briefing of Bond, it said it literally cannot tell whether the programme is doing any good whatsoever. That to me is damning from the NGOs. They are effectively saying that you could do more work pissing in the wind.
Richard Gledhill: Could I comment on the Bond views? I obviously was not at your meeting so I cannot comment on that, but we held a meeting with them a week or two ago to review the findings of the report, and they were very engaged, because many of them are bidding for future work. They expressed some quite strong views and said, “Please pass these views back to CSSF but do not say it was us that said them”, as individuals, because they were concerned it might impact on how they were received from the point of view of bidding. That is an issue and it is something that CSSF needs to work on.
On the multi-year point, I can see the value-for-money arguments about having break clauses on multi-year contracts but from the perspective of the NGOs that are providing these contracts, they effectively only have one year of certainty of funding, and that is impacting on behaviour and on short-term focus, and it encourages them to perhaps hide bad news. For many of the issues that CSSF is looking at, they are really challenging long-term issues that require longer-term funding. That break clause point is causing an issue that potentially could undermine value for money, even though it has been put in place for good value-for-money reasons.
Q13 Mr Lewis: Good morning. I have one observation and then I have some questions about specific programmes, the impact and ICAI’s findings. The challenge for Ministers, for senior civil servants and for leaders of organisations in all sectors in the end is the link between strategy and policy and delivery and implementation. In the end, we are all judged by delivery and implementation. It seems to me there are a lot of issues and challenges that have already been identified this morning about the actual frontline delivery. That is what matters, because that is how we are judged in the end. You can have all the architecture that leads to the frontline, but if the frontline is not adding value and is not delivering, then all of that architecture and wiring is either wired up wrongly or there is something going seriously wrong. That is just an observation.
In terms of the specific programme findings, as I understand it, in Mali ICAI visited a prison where CSSF-funded physical access infrastructure was poorly designed and not used as intended. Can you perhaps develop a little bit more and explain what you meant by that?
Richard Gledhill: Perhaps I can ask Willem to do this. He led the mission to Mali. I was not there.
Willem van Eekelen: It is an interesting example because it is layered and almost like a metaphor for the fund as a whole. This is the situation. This was Conflict Pool funding but it was under the responsibility of the CSSF when we visited. First, what happened was there was a breakout in this prison—Bamako Prison—and as a consequence the authorities had checked with the UNDP and the UNDP had agreed to use CSSF funding or Conflict Pool funding to upgrade the infrastructure. Those were the basics.
The infrastructure itself was poorly done. There was a CCTV camera but the wiring was so low it was easy to sabotage—Christian, you can correct me if I am wrong. There was no air conditioning in the CCTV room so it is probably no longer working because it is a dusty, hot place. There was a certain amount of staging as well. The man looking at the CCTV set of cameras, when asked for an example of how this was used, could instantly give an example of that morning but when asked for a second example of his year of doing that, he could not think of one. That is typically a small alarm bell of somebody who has one rehearsed answer. It was not really being used, we felt, but if it was used it was not going to be used for very long because this was going to break down. That was one of several infrastructure issues.
That was not the bigger issue, though. The bigger issue was that safety and security in a prison depends on standard operating procedures and those procedures were not there. At one point in time we were pushed into a small door, with behind it a number of alleged terrorist detainees, together with the women of the UNDP with no back cover. It was clearly not safe. Stuff could have happened then. It does not matter if you have CCTV in such circumstances; it matters that you follow protocol and you secure staff.
That was not the real problem either. It was also a problem of relevance in the sense that this prison was misplaced. It was in the centre of the town. It was an unsafe place to imprison alleged terrorists. They were indeed planning to move it out of town. Why do you then invest so much money and time in the infrastructure of a prison that will no longer house the group that you are focused on?
That brings me on to the last and the most important point: these detainees—there were many of them—had been arrested on accusations of terrorism but they had not been tried. They did not have a timeline for trial either. Several of them had been there for multiple years. Clearly, there is a grave human rights violation. You have 100 people who left their wives and children to go into prison without a trial or prospect of a trial. By building that prison you are complicit in that abuse.
Dr Dennys: It is a good example but it is the example of the Conflict Pool. That funding was given to the UN before the CSSF existed and one of the issues that transpired was that the UN held on to that money and did not spend it. Two or three years later they then come back to the British Embassy, the CSSF team is the major programme team in that country and they are asked to have a conversation with the UN about the disbursement of that funding for which the UN had already agreed on what to spend it. It is a very good example but it confirms the criticisms that ICAI made of the Conflict Pool.
The CSSF programme manager and in fact the CSSF programme in Mali has been working with the UN team since to ensure that there is a project board that looks at the disbursement of any UK funds in any project in Mali by the UN. That is the difference between having a CSSF and having a Conflict Pool.
It was happenstance that a few weeks before Willem and I travelled out—I was accompanying Willem to do the review—the UN got hold of the British Embassy to say, “We have this money and we are delivering it to support the refurbishment of a prison”. The criticisms of the individual project are valid and we would agree, because that is not how we think those programmes and projects should be delivered, but we need to be very, very clear that that was under the Conflict Pool under a previous settlement, under a previous team.
Q14 Mr Lewis: You do not take any responsibility for the failings identified by ICAI in the context of this prison?
Dr Dennys: We were not involved in the decision-making to provide that funding. We were not aware—it was a single-year disbursement to the UN—that that project would continue as it was transferred from the Conflict Pool of the CSSF. To the CSSF, there was no project and it turns up two years later. There have clearly been failures—that is not in argument—but it is not the CSSF.
Richard Gledhill: One of the points I remember very clearly from early meetings with the CSSF was them saying that they had actually closed down a lot of Conflict Pool programmes because they were not happy. That is a great and sensible action. Why this was continued when others were closed, I do not know. There may be good reason for that. What you see here, irrespective of the design failings that might possibly be laid at the door of the Conflict Pool, is that there were failings in monitoring, which must be the responsibility of the CSSF.
Willem van Eekelen: In fact, the Conflict Pool gave the money for work in the centre and in the north of Mali for an entirely different purpose. It was changed in purpose in 2017, which is under the watch of the CSSF.
Q15 Mr Lewis: To be clear, it was changed in purpose; do you mean in terms of being deployed in this particular prison?
Willem van Eekelen: Correct and, in fact, on prisons in general. Originally, the purpose of this Conflict Pool donation was to upgrade the legal system in the north of the country. That could not happen because it was inaccessible. It then lay dormant for a while and then in 2017 the decision was made to use it to upgrade the security of the infrastructure of that prison. This was under the CSSF watch, so either there is a case of poor monitoring or there is a case of poor decision-making.
Q16 Mr Lewis: Let me just get clarity, Chair. The CSSF made the decision to deploy the resource in this particular prison. That was the first thing. It was their decision.
Melinda Simmons: No.
Dr Dennys: No.
Q17 Mr Lewis: Hold on. I am being devil’s advocate for a second. In terms of the nature of the work that was done under the banner of those resources and that programme, it was CSSF’s responsibility to oversee and monitor the way that money was being spent in the programme being delivered in that prison. Is that correct?
Melinda Simmons: It was a contribution to the UN that had been made some years previously under the Conflict Pool and the team in Mali only became aware of it recently. You are quite right that having been made aware of it, the team in Mali then needs to report on it, but it was not a contribution to do that particular work; it was a contribution some years back that they decided to use two or three years later. Yes, we are all quite irritated, including the Mali team, only to have found out about it recently. That is because it was down as part of a prior portfolio.
Q18 Mr Lewis: To be clear, for money that was obviously earmarked for criminal justice reform or whatever, you never made the decision to spend it in this particular institution.
Melinda Simmons: For this particular allocation that the review team have been looking at, that was part of a previous settlement.
Q19 Mr Lewis: We have diametrically opposed views. Do you accept you are responsible for monitoring the delivery of the work in the programme?
Melinda Simmons: What I am accepting is the logical thing that follows. Richard is quite right: it was me who was explaining to him that in the first year of the CSSF that we did spend some time closing down what amounted to over £1 million of projects. This was not one that we were aware of, because it was a contribution to a UN facility in-country.
Q20 Mr Lewis: Unless I am wrong, there is clearly a significant difference of opinion about where responsibility and accountability lies for the failings of this particular project. There either is or there is not.
Willem van Eekelen: I think we both agree that this happened: the Conflict Pool gives money, the purpose changes and it happens in 2017. In 2017, the CSSF is responsible for the remaining Conflict Pool funding. That decision, made perhaps by UNDP with or without consultation with the CSSF, is made under the CSSF watch. Either there was insufficient attention to these remaining funds or there was an agreement to fund it. It is one of these two. There is no third option.
Q21 Mr Lewis: Were you consulted and engaged with by the UN?
Melinda Simmons: That would have been via the team in Bamako.
Dr Dennys: The selection identification of projects was driven by a UN team. As I talked about earlier, as this issue was made clear to the Embassy, the Embassy and the CSSF team then started a process to form up a project board that would look at all future programming options and all future activities. It is challenging to see how we can be held to account for something we did not actively decide on and had no knowledge of until significantly later.
Q22 Stephen Twigg: The CSSF is the successor body to the Conflict Pool. Clearly you seek to learn lessons from what might have gone wrong with the Conflict Pool but surely you inherit the responsibility. It is all UK Government funding, so surely you should be looking at what the Conflict Pool had funded and be monitoring that, rather than saying, “We are the CSSF; it is nothing to do with us because it was the Conflict Pool”.
Dr Dennys: Absolutely, but in this instance it was a single contribution to a UN trust fund, which is a basket fund of multiple donors. That contribution, from the perspective of the Conflict Pool, was seen to have been closed. Do not forget that under the Conflict Pool, there were no annual reviews; there were no ongoing monitoring systems on delivery. That is exactly the system that we have been putting in place now.
Q23 Mr Lewis: Essentially you are saying that you were unaware that this specific amount of money was being spent in this specific institution.
Melinda Simmons: In that year. It had been deemed according to Conflict Pool paperwork that a contribution had been made to a trust fund, and then it only became clear to the team in Mali, who then alerted us, at around the time the ICAI team were doing their review immediately when they became aware, that this money had been held over by the UN and was going to be spent in that year on that thing.
Q24 Stephen Twigg: If ICAI had not got in, we would not even know about this, would we?
Melinda Simmons: That is questionable.
Dr Dennys: The team would have.
Q25 Stephen Twigg: Which team?
Dr Dennys: The CSSF team.
Q26 Mr Lewis: You just said you did not know.
Dr Dennys: No, they were alerted prior to the visit that we undertook. It was not the case that ICAI found—
Q27 Mr Lewis: The team were told because ICAI was about to visit?
Dr Dennys: No, no. The UN through its own systems had then got in touch saying, “We have had a request from the Malian Government to do this. We are planning on delivering this. You gave us this money three years ago”.
Q28 Mrs Latham: Was there no checking that it had been spent in-year?
Melinda Simmons: It had been accounted for financially as the contribution to a UN trust fund, but one of the issues in Mali is that until the CSSF was set up there was not a project team in Bamako. It was always a small UK presence anyway. The CSSF project team is a year and half there. It is quite a new capability across the board, including for other Government Departments, including for DFID. I imagine that at the time of that—and of course none of us were there and it was at the time the Conflict Pool—the payment was made and accounted for, but there would not have been a bespoke team in Mali whose job it was to follow up on where that particular contribution went. It is recorded as having been made, and it so happened that about a month before the ICAI team came out, the UN team, UNDP, alerted the new team in Bamako that this was—
Mr Lewis: Can I ask Willem to come in? I am then going to move onto the next question.
Willem van Eekelen: Is it useful if I give another example of the same country that does not have this Conflict Pool complication, with the same kind of principle of the risk?
Mr Lewis: Yes, very quickly.
Willem van Eekelen: Chris and I will go to the centre of the country to meet with a couple of groups that are representative of beneficiary communities who do local conflict resolution. In the course of our hours there, it dawns upon us we are talking with a husband, his wife, his uncle, his sister, a nephew and the mayor of the town. These people have monopolised aid in their sub-region. They can talk effortlessly and flawlessly about all the various peace-building programmes, and in fact gender programmes and livelihood programmes in that sub-area.
Richard Gledhill: Not just CSSF.
Willem van Eekelen: Not just CSSF but across the board. Because it is across the board, you might think it is less concerning because other donors have those same issues, but also if you cannot travel somewhere, one of the key risks is elite capture of aid. One of the key risks is if you cannot see where it is going, it may be going to the same people all the time. You need to be aware of that risk. Not being aware of that risk opens you and makes you vulnerable to causing harm, which is entrenching the political power dynamics and the people who get all the resources and give them more.
Mr Lewis: Can I suggest, Chair, that we ask for something in writing to clarify a timeline of who knew what when? This is about the decision to deploy the resource, the design of the programme and then the oversight of the programme’s delivery.
Chair: Yes. That is very sensible.
Richard Burden: Does that apply to that second bit as well as the first?
Q29 Mr Lewis: Just the first at the moment. I have several more to come on to. I am going to move on. In Colombia, you describe CSSF-funded reconciliation activities that were “not much more than opportunities for elites to give speeches”. Can you explain that, please, and how do I access that funding?
Willem van Eekelen: One of the tough choices that the CSSF has to make always, and in the case of Colombia made very sensibly, is to do bilateral projects or contribute to multilateral funds. This was to multi-donor funds that were immediately after the peace agreement, had a quick-impact component and then a more durable component. This quick-impact component was meant to quickly do some basic reconciliation.
I had the pleasure of going to a place where there was such programming in place. In fact, it started on the day that we came there. There were two things that were striking. The first was that there had not been a civil war there for the past decade. The reconciliation was not so urgent. It was a conflict of eight years ago. The second problem was that the set up was a school with a theatre-style open marketplace kind of thing with—I do not know the English word for this—an elevated area where the important men sit and that is where the important men sat. Then you had a couple of hundred women and others listening to them reflecting on how things were bad but better now.
Q30 Mr Lewis: This is a one-off reconciliation event that you attended.
Willem van Eekelen: No, it is one of several. Earlier that morning we went to another place.
Q31 Mr Lewis: We have all visited such activities and events around the world—those of us who have been involved in development for quite some considerable time. To be clear, it seems to me if there was no civil war, this was an issue about why there is a need for reconciliation all of a sudden.
Willem van Eekelen: No, there is a need for reconciliation. There is no need for it being suddenly so urgent.
Q32 Mr Lewis: Why was it inappropriate as a one-off event, bearing in mind the context—we would not support the lack of gender diversity, et cetera—for people, leaders, to be making speeches as part of a reconciliation event or process?
Willem van Eekelen: One of the challenges within stabilisation and conflict programming is we do not really know what good looks like and what good design looks like, but we do know the things that do not work and we have some inklings of the things that may work. One of the things that may work is reconciliation, provided it is at grassroots level, very interactive and focused on those who are the primary victims.
Q33 Mr Lewis: Why do you say this was hastily convened? Was it because you were visiting?
Willem van Eekelen: No, it was part of that emergency 11-month time period of quick reconciliation initial effort of the multi-donor trust funds.
Q34 Mr Lewis: We are over a 12-month period, eight years on, going to embark on support for a multi-donor reconciliation process. Is that what you are saying?
Willem van Eekelen: Which makes sense in many places. In this particular place, the civil war had never been a war between FARC and the Government; it had been the paramilitary and FARC. That had ended some time before.
Melinda Simmons: I visited Colombia and I visited the reconciliation projects under the multi-donor trust fund. I have visited five of them. All of them had very clear evidence of community engagement. They were involved in tracing missing persons, identifying bodies via DNA and matching them up with relatives via counselling. We have visited women who had been raped and people who had been displaced, and all of those were community-managed and community-facilitated activities. They were quite extraordinary.
We did consult with the team over this particular event because that sounded, given my trip—Christian has also visited Colombia—so singular. The feedback that we got from our contacts at the Embassy was that that had been perceived as an event that they needed to big up as a launch because it was the first day of that project. It was part of Government wanting to get involved in a big new initiative, which I know that ICAI speaks of in our review—that without host Government involvement it is actually quite difficult to make progress, and this was an example of that—and it is just one of that range of projects.
I do not see it as being either representative of the UNDP effort in the country or representative of an argument that that portfolio does not have evidence of community engagement or community leadership.
Dr Dennys: There is a political moment around the peace process, and in ICAI’s previous review of the Conflict Pool, they say that the follow-on fund should be flexible, providing assistance in high-risk and politically sensitive areas. That is exactly what we have done in this instance: we have taken a political moment and pursued it. The fact that a community may not be in the most high-risk area or have had conflict most recently is more to do with the context of the visit rather than the broader project.
Chair: We are about 15 minutes behind. There is some interesting discussion happening. I do not want to stifle it but be mindful of the time if we can.
Q35 Mr Lewis: This is another example. In Jordan, ICAI found that a CSSF‑funded water programme had caused increased leakages elsewhere. Has this now been fixed?
Dr Dennys: Yes.
Mr Lewis: I assume you meant liquid leakages rather than other sorts of leakages.
Dr Dennys: Indeed.
Richard Gledhill: This problem says more about the local water authority than it does about CSSF. It was their failing. The bigger issue here, as far as the CSSF is concerned, is that until our visit the Embassy did not know about this problem. It had been causing frustration in the community for quite some time.
Q36 Mr Lewis: You were going to say something on the other issue very quickly.
Richard Gledhill: We commented very favourably about aspects of the work in Colombia, in terms of the flexibility, understanding the conflict dynamics and responding to change in the peace process. That is great. The failings that we are talking about here are about failings of design and failings of monitoring and not drawing on best practice. The UK is one of the funders behind the UN report, Pathways for Peace, and this is just the sort of issue that is dealt with in that report. We would like to see more drawing on best practice.
Q37 Mr Lewis: To be clear, on the leakage issue there had been no action taken until you identified it as a problem and the community made you aware that there was a problem. Is that fair?
Willem van Eekelen: Yes.
Q38 Mr Lewis: The final example is to do with the Malian armed forces where ICAI identified a number of very specific shortcomings. It is the same question: have those now been put right? Was this another example where it was only put right because ICAI identified these issues as being a problem?
Dr Dennys: The challenges around the provision of training and capacity building to the Malian armed forces have been there for some time, and it takes a long time to work with partners to work through those issues to ensure that there is a wider pool of Malian armed forces to train, and that all of the training is provided adequately, on time and to a high standard. That is not just about the UK’s contribution. The UK’s contribution very specifically in this context is around preventing sexual violence in conflict. There is also some non-ODA-able military advice provided as well. It is part of a package of, in this case, the European Union’s training mission to support the Malian armed forces. Yes, we are working through those issues.
Willem van Eekelen: I am pleased to hear that.
Q39 Mr Lewis: You identified during your visit that there was a problem.
Willem van Eekelen: Yes.
Q40 Henry Smith: This is to the National Security Secretariat. Obviously you are dealing with some conflict-affected areas where the organisations you are operating with maybe have a dubious human rights track record. As such, an overseas security and justice assessment should be done, but ICAI found in a number of cases that that was either missing or incomplete. Can you explain why that is and what is done to rectify that situation?
Melinda Simmons: Sure. Thank you. The first thing is that the ICAI report stated that they could not find any evidence that the projects were doing harm—that was very helpful feedback—and that the programmes similarly were not doing harm. There are several ways where we are doing activity and these risks are captured. One is in the conflict analysis and the conflict sensitivity work that the CSSF does, which was praised quite positively by the ICAI review, compared to the Conflict Pool. It was one of the main recommendations from the Conflict Pool that there should be conflict sensitivity injected into programme planning. We have just started piloting a conflict sensitivity marker, which will sharpen this further. That is the first: that it comes out in the analysis.
The next is in the risk registers, which the programmes have to put together for each of their programmes, which are reviewed regularly by directors in those boards in London and by ambassadors in their programme boards at post. Those risk registers will already pick up these issues.
The OSJA is an additional tool for security and justice programming. That speaks to making sure that the UK is operating within the law with respect to human rights. It is a fair point that ICAI makes that in some cases the OSJA may either have been not filled out particularly well or not managed, if you like, in a live way, which is part of what we discussed when Willem returned from the trip: that it needed to be used more as a live tool. We take that point. It speaks to some of the issues around our programme cycle documentation. I am sure we will come on to results, M&E and so on, but having put these systems in place and having training people in them, the next thing has to be to make sure that if we are doing things, we are using these templates to record them.
The annual reviews also capture the use of the OSJA as they do the use of the results framework. In fact, last year if you did not have a clear theory of change, a clear results framework and a completed OSJA, you got no score on your annual review. That affected the allocation for the following year, so there is a little bit of stick built into that already. We take the point that we need to come back round to team and make sure that OSJAs are used in a live way and are completed, obviously, consistently across the programmes. It is important that it is understood that that is one of several ways in which risk is captured and that that is an additional proactive tool.
Q41 Henry Smith: To ICAI, what percentage—I am not looking for an exact figure but generally—did you find that the assessments had been missing or were incomplete?
Richard Gledhill: While Willem tries to think of the percentage, can I comment on Melinda’s response? In addition to the issues we found around whether things had been completed or not, there is also the issue that where OSJAs had been completed, they always led to a decision to proceed, which suggests that these were just a box-ticking exercise: “We have done the OSJA; therefore, let’s crack on”, rather than being an effective control mechanism. Rather than saying we did not find evidence of harm, we would rather say we did not really know.
Dr Dennys: This speaks to some of the methodological limitations, because as we have said to ICAI previously, there are cases where the production of an OSJA has led to a project or a programme not proceeding. A team goes through the process of, “We have a strategy; we are going to design a programme to try to deliver this”, and then they start working through the ins and outs and the details of what that is going to mean, and then it does not happen because the OSJA is not going to get approved or it is too high-risk. It is not in one of the case studies that they have looked at, but that does not mean it does not happen.
Willem van Eekelen: What Christian says is true, and it did come across in one of our case studies as well: “We did not do an OSJA because we decided not to pursue this because the conflicts and safety analysis showed that this was too high of a risk. I also want to add to that that the CSSF employs conflict advisers who very clearly know what they are talking about almost all the time.
I also take Melinda’s point that the OSJA is the tip of an iceberg, but it is an important tip of that iceberg and it does need to be good enough. If it is not good enough but it does get approved, that is a problem. How often did that happen? I would say that seven times I found serious concerns.
Q42 Henry Smith: Out of how many?
Willem van Eekelen: There was one project in Pakistan that had 13 OSJAs that I did not even look at so I do not know how often or out of how many, but seven is a problem.
Q43 Henry Smith: I am a bit concerned to hear the expression “box-ticking” in response. Are they fit for purpose? To the National Security Secretariat, are you satisfied with DFID’s response to the missing assessments?
Dr Dennys: The OSJAs are owned by a team in the Foreign Office. They are not a CSSF product. They cover and can be used for activity that is not just programming, hence why there are lots of OSJAs in Pakistan that Willem has not looked at. We have already been working with the relevant Foreign Office team to update their guidance, and we will be providing additional training. Everybody is in agreement that it is important that the documentation is clear, that those risks are laid out appropriately and they are updated regularly. Again, I go back to the point that there is no evidence yet that the programmes have caused any harm.
Willem van Eekelen: The bigger problem, though, is not the OSJA. It is not the start of the project; it is the fact there are monitoring issues, as a consequence of which there is no real insight on what is happening with some of the projects, and there is no history of evaluations, so you do not know if there were any negative externalities. When Melinda said that we did not find any evidence of harm being done, that does not mean that no harm was being done; that means that there is no evidence that we could look at.
Q44 Stephen Twigg: I am going to bring this on to the issue of eligibility of CSSF’s work for classification as overseas development assistance. If I can start with a question to ICAI, in the review you state that nearly all of CSSF projects characterised as ODA are indeed ODA-eligible. Can you give us an example of one that you think was not, which was perhaps wrongly badged as ODA?
Richard Gledhill: It is fair to say that the examples we found where we felt that ODA eligibility was questionable were small in number and for relatively small amounts of expenditure and typically related to the fund’s first year of operation. Clearly, following ODA rules is important, so we do not want to belittle this point, but we did not see this as one of the major concerns. Willem, are we able to talk about the specific examples in public?
Willem van Eekelen: We cannot talk about all of them, but we can for a few of them. There was some intelligence work on criminal profiling in Colombia, which we felt was intelligence and not ODA-eligible. There was a media outlet in the Caucasus that used some of the money they received to portray a positive image of the Caucasus to western Europe, which is probably a valuable thing to do but it is not ODA-eligible. Again as Richard says, there are a few examples of minus sums of money.
Richard Gledhill: They are typically just parts of programmes.
Q45 Stephen Twigg: Richard, you said it was mostly at the beginning. Was the implication that it is less likely we would see that now? It was almost a teething issue with the earlier period?
Willem van Eekelen: That came across in interviews as well: “Yes, we would not do this project anymore”. They were small, minor and learning.
Q46 Stephen Twigg: Your remit as ICAI is to look at ODA. CSSF is a blend of some things that are ODA and some that are not. Would it help your work or would it have helped in this review particularly if you were able to look at all of CSSF and not just the ODA-eligible parts?
Richard Gledhill: Yes, there would be value in ICAI or someone else doing scrutiny of the totality. Always when looking at just a part you do not get a complete picture.
Q47 Stephen Twigg: Coming to the government witnesses, under the OECD DAC rules, spending cannot be counted as ODA unless it has a primary purpose around the economic welfare of the recipient countries. We have dealt with the issue of the relatively small number of cases that ICAI found should not have counted as ODA at all. I want to ask about a broader issue that our recent report on ODA highlighted, which is projects that technically count as ODA, but where we as a Committee struggle to see the focus on poverty reduction that we would expect. ICAI found that some CSSF projects categorised as ODA had limited prospects of success and had essentially been launched for diplomatic reasons. Can you comment on that and do you accept that those sorts of cases should not be counted as ODA?
Melinda Simmons: I will say a few things, then might turn to Christian for some of the detail. First, it is quite right that, in the first year of the CSSF—the first year of the settlement period—DFID had given advice but had not been as hands-on as they were from year two of the CSSF. We had all learned, and the cross-Whitehall ODA board had been established. We participate in that board, as well as Departments that have bilateral allocations, and DFID leans in very hard. In any case, we have to report all our ODA-eligible projects via DFID. DFID audits those and then they are provided to the DAC. It is a pretty forensic exercise. I am glad to hear that there are just a few small examples, but it is the case that, from the second year on and as we added to our training, people became more confident in understanding what that meant.
All of our guidance makes very clear the primary purpose of poverty reduction, but the piece I want to say on that before turning to Christian is that there is a range of activities the Government want to do under the banner of poverty reduction. The CSSF’s job is not to replicate what Departments do with their bilateral allocations—that includes DFID. It is also trying to blend, but not just blend ODA and non-ODA, but blend the range of ODA-able ways in which we want to tackle the causes of poverty. In the case of the Conflict, Stability and Security Fund, that is all about security and stabilisation, so it hits some SDGs and it does not hit others, where DFID ODA primarily or other Department ODA will be focusing itself. Those are the indicators that programmes are required to point to, so they need to show that they are operating within the terms of the rules and then they need to show which bit of poverty reduction activity they are targeting. You might want to say a bit more about how we do that, Christian.
Dr Dennys: At the macro level, we all recognise that conflict affects economic growth; that makes people poorer. It is estimated that, by 2030, 60% of the poor will be living in conflict-affected states. Therefore, this is a growing trend and issue. Against DFID’s fragile and conflict-affected states list, the CSSF spends 70% of its ODA in specific countries and then about 30% of it in multiple countries. That will be, for example, support to the UN Department of Political Affairs, helping them to deploy forward mediators to help address conflicts in West Africa, for example. We are comfortable that we are aligned with the name on the tin, which is the Conflict, Stability and Security Fund, because we are about addressing insecurity and conflict, which affects a significant number of poor people.
Q48 Stephen Twigg: Is there any comment back from ICAI to that response? You had raised concern around limited prospects of success and things being done for diplomatic reasons. Do you feel we have had a full enough response to that today?
Willem van Eekelen: It is a problem that I think is being addressed. We saw it mostly in Pakistan and less in other countries, and we had a conversation about that. I understood it had to do with the nature of the people making the decisions and the coaching they had or had not received. Correct me if I am wrong. If that is so, I would expect that in the post-one-year review we will find that this has been satisfactorily resolved.
Dr Dennys: ICAI recognises that it is worth trying and trying again in some contexts where there is conflict, in order to be able to pursue opportunities for peace. Not all of those will be successful. Therefore, you will try something initially as one narrative and then see where it goes.
Stephen Twigg: That is absolutely fair.
Q49 Mrs Latham: As you know, I have three questions but two of them have been covered by Lloyd, so I will only ask one. That means we should catch up on the time. ICAI, I wanted to specifically ask you: you found that CSSF suppliers face a disproportionate reporting regime. Are you guilty of micromanaging your suppliers?
Melinda Simmons: Is that to us?
Mrs Latham: No, it is to ICAI. They criticised the reporting burden.
Richard Gledhill: We were chatting about this the other day, as to why we had not raised a particular recommendation on this issue. The point is made most clearly in a small footnote to the report but, interestingly, it was picked up on Twitter and commented on by the NGO community. We feel that the reporting burden, particularly around activity-based costing for budgets and so on, is unduly onerous for many of the programmes. Certainly that was feedback we have had from the CSO community.
Willem van Eekelen: I can give a quick and very clear example. There are multi-reporting requirements for a consortium in Mali, where a programme takes place in the central north of the country, which is inaccessible and it is hard to get an internet connection. People there do not speak English. What you then have is a local organisation that writes a report at the start of the month that needs to go to their headquarters in the capital city. It then goes back and forth a couple of times. Every time, you need to drive somewhere to get an internet connection. It then goes to the consortium lead—again back and forth. It then needs to go to headquarters because it is a formal report to donor, so it needs to be authorised back and forth. On the 15th of the month it needs to be ready for submission to the CSSF.
You do this every month, so every first two weeks of the month there is a whole lot of tension and worry about your reporting requirements for that month. It means that, effectively half the time, you are bothered about that month’s exercise, narrative and financial, which is not conducive to good programming. Plus, as an implementer, you know that the CSSF will never actually have to come to visit you, so there is limited incentive to highlight the trouble that is there, potentially. Instead, you do reports that are broadly positive with small problems that you can resolve, because they will not see it and we are not sure we are going to get money beyond the first year.
Q50 Mrs Latham: Also, NGOs have come and said to us before that they get money from us and from other people. When they get it from multiple donors, one of the problems they seem to have is that each donor requires a report, but it is slightly different. They are doing a report for us, a report for B, C and D, and they are all slightly different, which adds a huge burden. Is there one way that we could co-ordinate better? We have to make sure that people in this country are happy that every penny spent is spent well, so we need reporting back, but do you think there could be a better way of doing it?
Richard Gledhill: I certainly understand the good intentions behind what is being done by CSSF, but there is onerous reporting, particularly for many of the smaller programmes. Willem highlights the vital importance of visiting programmes and seeing what is happening on the ground. You learn so much more by doing that than just looking at reports remotely. The UK is trying to show leadership in the international community to harmonise reporting, but it is harmonising at a higher standard of reporting. This is something that needs to be addressed on a case-by-case basis, depending on the scale of the programmes.
Melinda Simmons: The particular issue for the CSSF is that, because of the work it is trying to do, it is in some of the most dangerous countries. Therefore, going to see communities and projects is just not always possible. There are parts of the portfolio that are trialling third-party monitoring. They are doing it in East Africa and the Middle East. Opinion is split on whether obtaining verification of third-party monitoring is a good thing to do. That is something our M&E adviser is trying to work through, to make sure that those people are doing this in the right way. That is something we would want to encourage, because of course we do not want to make reporting onerous, particularly for community groups that do not have internet or resources.
At the same time, we always have to strike a balance between making sure we know whether the programme is having an effect, which is one of the core issues in the ICAI report, and making sure that posts have the accountability so that they can report back to London and, nonetheless, enabling organisations to be empowered to get on and do.
Dr Dennys: This is really about balance and whether we are too hard or not quite. You would have a slightly different line of questioning if you felt that we were not monitoring the spend. If we are being slightly too onerous on some of the small NGOs, we will take back that piece of feedback and continue to look at it, but we work in some genuinely high-risk countries. We use third-party monitoring where it is feasible and suitable, given the scale of the programming. The Mali example is a relatively small project within a medium-sized programme within the CSSF; it is not one of our biggest activities. This is about how we make sure we get that balance right and provide assurance without causing a detriment to delivery.
Q51 Mrs Latham: Is ICAI happy with the response from the Department?
Willem van Eekelen: Broadly we are, except that it is not just about the balance; it is also about the intelligent design of it.
Q52 Richard Burden: I would like to stay on monitoring, but come at it from a slightly different angle, looking at what it is you monitor and what conclusions you draw from it. Certainly what came out from the ICAI report is that you focused very heavily on monitoring short-term activities and outputs, but ICAI also said that the theories of change that underpin that were rather weak. In one case, it just added up to a list of bullet points. Given that, how do you know whether the short-term outputs that you monitor are delivering your intended long-term objectives?
Dr Dennys: Again, ICAI agrees that the important thing we have done at this initial stage is to build the system. We are midway through building the capability and we will get to a point when we are doing full-throated evaluations in due course, which I will come to in a moment. Monitoring activity happens on a quarterly basis. More importantly, in moving up from outputs to looking at the outcomes—so the bit in between what you do and what you are hoping to achieve—the annual reviews are being completed this year, so that is the third year of programmes. ICAI’s review happened before that. The annual review cycle concludes this month and all of our programmes will be given scores for whether they are delivering what they say they will and then what that adds up to in terms of outcomes.
That is building an evidence base of what the programmes are achieving, but in terms of evaluations, that also takes time in terms of looking back. You have to have delivered for long enough to then be able to evaluate. Some of our programmes are already starting evaluations of some of their activities, which includes some work in the Middle East and North Africa. In part, it takes time to build that full evidence base. ICAI is maybe concerned that, at the two-and-a-half-year point, we have not put in enough resource to do that. We have responded. We have said we will put in additional resource to make sure we hit that standard, and then we will see what those outcomes and evaluations say in due course.
Q53 Richard Burden: It is good to hear that the evidence base is building, because there are some criticisms in there about what is happening so far and assumptions being made. In Iraq, for example, you assume that the successful delivery of an infrastructure programme would build public trust in the Government. They could not see why you made that assumption. It is one thing to say that you have not yet built the evidence base, but why did you make the assumption that it was delivering?
Dr Dennys: That speaks to the fact that there is not a huge amount of evidence around what happens when a city on the scale of Mosul suffers the degree of devastation that it has and what is required to enable people to return. That multi-partner programme, run by UNDP, has 24 partners in it and has enabled 3.2 million people to return home, for which the UK is contributing. A part of our activity—not all of it—has picked up an element of infrastructure work. They are going to need things like schools in order to be able to live in the city that they used to call home. Whether that contributes to the consent and legitimacy of the state is a proposition that is still debated in the academic literature.
While it is true that assumptions have been made, the evidence base in academia is not in full consensus about what does and does not work. Therefore, we have participated in a multi-donor programme that has had a demonstrable effect. We accept that the individual component will need to be tested, and we will go back and test it. We will learn what we can from that activity and then we will apply that to other parts of our programme, elsewhere in the world.
Q54 Richard Burden: If I understand ICAI’s criticism correctly there, it was not saying that the programme itself was misguided. It was the assumptions that were made as a result of it. As you say, the academic analysis of that programme was that it was too early to say whether it was building any kind of trust in the Government either way, but the assumption of the CSSF monitoring was that it was. Was that premature?
Dr Dennys: Sure, but a visit in the midst of delivery, when the Government were still in the process of reasserting themselves in that context, is not the time to make a definitive statement as to whether that particular piece of infrastructure contributed to the legitimacy of the state. That piece of evidence will need to be drawn out in due course. I accept there is always going to be an assumption in some of the areas where we deliver because, as ICAI said in its previous report of the Conflict Pool, it expects the UK to be agile and responsive to those conflict environments. That puts us in positions where the evidence base in the real world is not as robust as it is on water and sanitation, for example.
Q55 Richard Burden: On external evaluation, ICAI looked at six country and regional programmes but, of those, only the Pakistan programme was subject to external evaluation. Does that not indicate a weakness?
Dr Dennys: This goes back to my point about the length of time. You need to build up the evidence base to then do the evaluation itself. There are other parts of the programme and portfolio, in Egypt for example, which are undertaking those evaluations. We are looking with teams at how to deliver them. Some regions within the CSSF have their own bespoke monitoring and evaluation contracts, and so are able to contract those in directly. That is what they are doing in the Middle East. Some of the other individual programmes, for example in Colombia and Mali, are not that big in and of themselves, so they cannot sustain their own bespoke third-party monitoring or evaluation from their own funding allocation. We are looking to do that from the centre and then to be able to provide support to do that.
Q56 Richard Burden: Going right back to where we started, Lloyd asked in relation to how far gender was central to the monitoring and evaluation that has gone on. You have emphasised that it is an important part of the monitoring that you do, but when ICAI looked at this, they said it was really quite difficult to find gender-disaggregated data in CSSF programmes. While everybody says this is central to what you are doing, getting the evidence out to make the point one way or the other was rather more challenging. How would you respond to that?
Melinda Simmons: Monitoring happens as part of the annual reviews and also, as I mentioned, as part of the monthly and quarterly look at both risk management and progress on the projects and the portfolio by regional boards, so it is captured. One of the issues we have been working through is how you can sufficiently aggregate our management information to provide macro evidence back to the boards. That is a work in progress, and it is a work in progress partly because of evolving the systems to do it, but also because, as I said when I opened, having nailed the leadership point and started the opening work—which is the audit of what we need to do to make sure gender is sufficiently mainstreamed—it is not consistent across the whole portfolio yet. Capability is one of the reasons why, so putting the right technical advice in the right place, but it is an objective that we have. I found it helpful that ICAI had highlighted this. It reflects the work that is underway to ensure that we can use the management information system we are building to get people to load and score their gender-related activity in the right way, so that we can aggregate that data. It is a work in progress.
Richard Gledhill: Could I comment overall on what has been said in the last five minutes on results management and M&E? These are some of the most important findings in this review and some of the most important recommendations. They underpin everything that the CSSF is doing. We recognise the steps that have been taken to improve results management and M&E via the fund. We would underline the urgency of progress in that area. We are encouraged by the acceptance of our two recommendations in this space, but we would also like to see that, where new funding is allocated, we are clear that there is appropriate M&E put in place to support that new funding, so that we do not have to play catch-up with new programmes.
Melinda Simmons: I cannot comment on the question of new funding. We are about to go into a spending review, so we are working with the settlement that we have. It is quite important to be clear that what we are talking about is not a fund that does not have the systems or the capability; this is embedding them in a real way throughout the programme cycle. There are programmes with result frameworks that are not adequate and there are programmes with OSJAs that are not being taken regularly taken for review by boards. There are parts of the world where gender is being mainstreamed brilliantly and other parts of the world that have yet to embed it in a meaningful way.
Our job is to make sure this all comes together consistently now and that we are able to challenge our programme teams, which can then work with their programme partners, comprehensively on the programme cycle. The ICAI report has been helpful in reflecting on work that is underway and gives a sense of impetus to it that we can take to our delivery partners and to our programme partners to help them to see it in a comprehensive way, but it is not catch-up. It is not something that we have not thought about, that we are not doing or that we are having to start afresh. It is something we have to put more time, more effort and more expertise into from our end. We have to make sure it is more consistent across the network, but it is not not being done in any part of the cycle.
Willem van Eekelen: Could I briefly reflect as well? On the question about infrastructure in Iraq, in the academic literature there is a fairly substantial piece of evidence that is reported on from Afghanistan, where the same type of programmes happened, so I think that this is one of the areas that is actually quite well evidenced.
The monitoring and evaluation plans look good. We had a conversation with the person in charge of monitoring and evaluation, and there are some good, solid plans for the short, medium and long term. If the CSSF implements this plan, it will be alright in a couple of years.
Q57 Richard Burden: Is that across the board?
Willem van Eekelen: Across the board, there is a good set of plans overall. The problem is that previous funds have also had plans that were not implemented. I now look forward to the post-one-year review, when we will see evidence that the plans are actually being implemented. If the plans are implemented, the CSSF will be well evidence-based in the near or somewhat longer future.
On the evaluations that are going to take place, normally I would hope that they are essential to the value for money of the overall portfolio. In the particular case of the CSSF that is not what I would propose. I would propose that, in a field of work where there is so little evidence about what works, it is important to evaluate the programmes that probably went really well, because you need to find out why so that you can contribute to the global lessons learned about what works in this field. The other thing to focus on when it comes to evaluations is programmes where there is a good risk of having done harm. Has that happened and, if so, what can we learn from that?
Q58 Chair: We are going to come to learning in a second. Thank you, Richard. While we are talking about learning, ICAI found that CSSF struggles to learn lessons, either from the research literature or from its own work. Its own contribution to the research literature is also poor, it said. What do you say in response to that?
Melinda Simmons: I will start on the question and Christian may want to comment. This is an area where I am less comfortable about ICAI’s recommendations, because it feels to me that ICAI may be thinking about the CSSF as a Department. The CSSF is a fund that is run by the joint funds unit, but all Departments are buying in and we are working with their own outreach, research and capabilities to do some of this macro learning. The Stabilisation Unit does quite a bit of it for us but, for example, DFID has a conflict research facility that it launched fairly recently and the organisations that are bidding into and using that are also delivery partners for the CSSF. It is about triangulating it. The way the CSSF is set up is not to be the leader of that, reaching out on the macro.
In terms of how we share this around the network, this is an area where, at least internally across Government, the CSSF has a really good reputation for learning and sharing lessons about the portfolio and using academic research, as well as our aggregated annual reviews. We have an annual lessons event that draws about 200 staff from across the network. The Stabilisation Unit aggregates the lessons from our annual reviews. We present thematically on it and by country. Teams from different countries working on different things will compare their approaches, so it is quite an exciting event for the chance that they have. It is their main chance in the year to be able to test their approaches on each other. That covers the complete range of themes: it covers security and justice, countering extremism, counterterrorism, migration and the range of ways in which the CSSF works.
Bearing in mind Christian’s point that we are still somewhat groping towards finding consistent evidence about what works, what we at least do, at a regional level and then globally for the fund, is make sure that, wherever there is an opportunity for learning through boards or these annual monitoring events, we are doing that. I take the point about reaching out proactively to stakeholders, which is something we could do more of collectively through Departments. That is something that we would like to go further on, particularly as there are other countries interested in the CSSF model. It feels, nonetheless, that the conclusion is a little harsh compared to the real investment that the funds and their teams make to ensure that lessons are picked up at macro and micro level.
Dr Dennys: To give you two examples of that, ICAI talks about the JACSs being a real good contribution, shaping both our programmes and wider HMG and international community efforts to tackle conflict in specific countries. Those are all evidenced by exhaustive reading lists. They are delivered under the auspices of the Stabilisation Unit. That is a clear product that is using quite a lot of that literature.
In terms of our own learning, as the team knows, one of our programme teams in East Africa has been looking at how you identify and influence as an outcome of a programme. That speaks to a lot of the questions you were asking us earlier and that tool has been piloted. We recognised some time ago—18 months or so—that this was an issue. It takes time to work up the evidence base: you have to go through some practice and work out what you are trying to measure. You then have to test it again, then you get to share and embed it. We are about to go into that phase at the event we are holding in July, when that will be shared with the wider network. The programmes this is relevant to will then need to grapple with how they understand, include and influence as part of the outcomes from their programme. We do it and, as I talked about earlier, individual teams hire in contractors or consultants to do individual bespoke pieces of work as required
Q59 Chair: Willem, were you too harsh?
Richard Gledhill: We were aware, as Melinda told us a few weeks ago, that she felt this was harsh. We certainly gave recognition to some of the good practices that we saw. We highlight those in the report. Our perspective, though, is that this is a very big fund spending a lot of taxpayers’ money and working in many challenging areas on challenging issues. It is generating a huge amount of learning. There is a need to share that more widely and to look beyond the concerns about security, document classification and so on, but there are also some processes in the engine room about sharing information on programmes. For example, relevant parts of the annual reviews are not always being shared with the implementers. That is basic good management of programmes, and I do not buy that it must be the fault of the department running the programmes. These are CSSF programmes and there should be good practices across the board.
Q60 Chair: Willem, you may be able to cover it on this one. I just want to come to the Government, then I will come back to you, Willem. The last criticism I want to come on to is that ICAI also found that the Joint Analyses of Conflict and Security were often out of date, while documentation in Colombia was particularly weak. What are you doing to resolve that?
Dr Dennys: The JACS as a methodology has gone through various iterations and there are continuous elements being added as a tool overall. The choices for undertaking an individual JACS are devolved to the posts, where they are relevant. The judgment from posts is that the structural factors identified in the JACS, as it had been produced in 2015, were sufficient for them to be able to programme against.
They recognised that they would need to update it. They also recognised that they were about to go through an election the following year. There are legitimate questions about value for money and resource use to do something just before a big political moment, particularly when you are programming in a politically sensitive manner or on conflict issues. As many of you may know, the election in Colombia had a particular flavour with regard to the perspective of the candidates on the peace process. It may lead you to a set of conclusions if you do that analysis too early and a set of programmatic actions that are not necessarily appropriate. We think they should be as up to date as teams believe they should be but, in this instance, we are not convinced that is the correct conclusion.
Q61 Chair: In your recent capability review of cross-government funds, including CSSF, you committed to developing public-facing versions of HMG’s strategies for priority countries, regions and themes. When is that going to be published?
Melinda Simmons: These are the fact sheets that we alluded to in the Government response. We have piloted two, and the pilot has been important because there is quite a lot of work that goes into producing them. Those two have gone round and we are soliciting feedback on them. I hope that we will be able to get the bulk of those done within this next reporting year.
Q62 Chair: Just turning to ICAI quickly—Willem, if you wanted to come back on any of the other bits, feel free—your review raised a number of interesting issues. You have touched on how, as we have been discussing this, not everything gave rise to a specific recommendation, like the over-focus on training, the specific programmes failings in Mali, Jordan, Colombia, et cetera. Why did you not include recommendations on those issues?
Richard Gledhill: There is a balance to be struck in these reviews to have a manageable number of recommendations that focus on the real priorities. When we come back to look at the follow-up of reviews, we are not just looking at the recommendations, but the underlying issues behind those recommendations. With hindsight, we might have included more on the CSO community. The strength of the feeling from the CSOs only became apparent to us after the report had been issued.
Q63 Chair: I am just wondering if, in future, you would consider addressing the totality of ICAI’s findings in responses, not just the ones that give specific recommendations. Is that something you would look at?
Melinda Simmons: Yes, we would. It is worth bearing in mind, particularly because this ICAI review technically only looked at about 12% of the overall CSSF, that we are thinking about this review in the context of the review you alluded to—the three-fund review done as part of the NSCR—and the Joint Committee on the National Security Strategy review. They all inform different parts of what we need to do and we are looking at all of them in the round. We would of course be looking at those reflections, as well as the formal recommendations.
Chair: Thank you very much. This has been a really good debate. Willem, Richard, Melinda and Christian, thank you so much for your time.