International Development Committee
Oral evidence: ICAI's Annual Report and follow-up review, HC 1731
Tuesday 11 December 2018
Ordered by the House of Commons to be published on 11 December 2018.
Members present: Stephen Twigg (Chair); Chris Law; Mr Ivan Lewis; Lloyd Russell-Moyle.
Questions 1 - 49
Witnesses
I: Dr Alison Evans, Chief Commissioner, Independent Commission for Aid Impact; Ekpe Attah, Head of Secretariat, Independent Commission for Aid Impact; Joelle Jenny, Joint Funds Unit Director, Foreign and Commonwealth Office; Daniel Davis, Prosperity Fund Head, Foreign and Commonwealth Office.
II: Dr Alison Evans, Chief Commissioner, Independent Commission for Aid Impact; Ekpe Attah, Head of Secretariat, Independent Commission for Aid Impact; Matthew Rycroft CBE, Permanent Secretary, Department for International Development; Juliet Chua, Director General for Finance and Corporate Performance, Department for International Development; Beverley Warmington, Director for Conflict, Humanitarian, Security and Migration, Department for International Development.
Witnesses: Dr Alison Evans, Ekpe Attah, Joelle Jenny and Daniel Davis.
Q1 Chair: Good morning, everyone, and welcome to this session where we take a look at ICAI’s annual report and follow-up review. I will welcome the first of two panels and take this opportunity to give my personal thanks to Alison Evans for her service as Chief Commissioner of ICAI. I know that you are back next week at the Sub-Committee, when my colleague Paul Scully will be in the Chair. Paul has sent his apologies for this morning’s session. We have really appreciated working with you, Alison, and we wish you the best of luck in your new role at the World Bank.
Let me kick off and ask this of our colleagues from the Foreign Office. Now that a larger amount of ODA is delivered through Departments other than DFID, ICAI argues that other Departments often lack robust systems for ensuring both impact and value for money. What do you say in response to that?
Joelle Jenny: Good morning, Chair, and thank you very much for inviting us and giving us the opportunity to talk about the Prosperity Fund. Let me from the outset say how much we value the opportunity to present the work, but also the recommendations that ICAI and the IDC have been making. As you know, the fund was set up to address inclusive growth opportunities in middle-income countries. It was set up to improve the business environment and make economies more open and more transparent, because we know this is how you can get an environment in which the poor have the best chances to access credit, access arbitration when they need it, and access the benefits of growth that will lift them out of poverty. It is fair to say that the fund is a fairly new instrument. We have been gradually putting in place the systems and processes that are required for adequate monitoring and evaluation of impact.
Q2 Chair: I am sorry to interrupt. We are going to ask some specific questions on the Prosperity Fund itself and I appreciate your role. In a sense, I am asking you to comment more broadly on behalf of the Foreign Office about ICAI’s observations on non-DFID ODA in general. We will get into detail on the Prosperity Fund a little later.
Joelle Jenny: As the head of the Joint Funds Unit, I can speak competently about the funds that are under my remit. A lot of work has been done in respect of these funds to ensure that there is thorough monitoring of the impact, and that the funds, particularly the Prosperity Fund, meet the ODA requirements. No project proceeds unless it meets all the criteria for ODA. We have invested a lot in monitoring, evaluation and learning so that the learning can feed back into the programming. We have robust assessments of human rights requirements and all the relevant requirements for compliance. The funds have established a ministerial-level committee, chaired by Minister Lidington. This was in response to the recommendation to ensure there was adequate top-level oversight of the implementations.
There is now a cross-Government governance system that enables all the business cases to be thoroughly reviewed. Each Government Department takes responsibility then for the implementation and quality assurance of the programmes. We have recruited external evaluators so that we have a robust independent assessment of what the funds do. We have developed key performance indicators in order that we have a clear yardstick against which we are assessing the impact, and we are building the capacity of colleagues at posts. We have invested in training and worked very closely with DFID to develop adequate guidance. DFID is obviously the Department that has the most expertise, so we are very grateful to DFID for its support in developing and training our network, through its expertise both in London and in post.
Q3 Chair: I will come to Alison in a moment. Let me just ask one more question. ICAI has raised concerns in a sense about two aspects: compliance with OECD rules on aid, the DAC rules, which it says is particularly challenging for Departments other than DFID; and then, more broadly, ensuring best practice to achieve the sustainable development goals. Can you tell us how the Foreign Office and the joint funds are seeking to fulfil those absolutely fundamental central requirements on DAC and SDGs?
Joelle Jenny: If I take the example of the Prosperity Fund, not one single project proposal will proceed unless it meets the OECD DAC requirements and definition of ODA. There is a first test. Before we look at any other aspect of impact, whether it meets the objectives of the funds or any other consideration, the project needs to meet the OECD DAC requirements. DFID sits on all the governance arrangements that oversee each of the funds. Again, we have a strict evaluation, monitoring and reporting cycle. We have also acted on the ICAI recommendation by increasing transparency, which is an essential element, to be sure we are open to the scrutiny of this Committee and of our NGO partners. We have now published two annual reports, programme descriptions, the financial data and the information on the procurement framework. We will publish, when they become available, the annual reviews. These are early days, as you know, because the programmes are beginning to deliver and we want to make sure we can be held accountable in that manner. We also have regular dialogue with NGOs so we can hear their feedback and take it on board as we design the programmes.
Q4 Chair: Alison, how do you rate that response?
Dr Evans: Since we are keeping it fairly broad at the moment, the first thing to say is that, certainly around our follow-up on the Prosperity Fund, yes, we recognise a number of the improvements that Jenny has set out. They are all hopefully moving in a good direction. More generally, we are keeping this under a very close eye. It is certainly the case that, as one moves beyond the reach of DFID, the experience with operationalising the rules around ODA eligibility is less deep. It is certainly the case that we look to the Department for International Development to play its role in ensuring that those Departments know how to achieve the critical thresholds. But it is also the case that those Departments have their own individual responsibilities through their own delegated accounting functions to do that job well. ICAI will keep a very firm and close eye on that.
Specifically in relation to prosperity, our particular worry at the beginning was that, yes, there were thresholds for achieving the primary purpose of spending under the Prosperity Fund, which would meet the ODA eligibility rules. We felt that actually was a very low bar. It is incumbent upon Departments to set themselves the kind of high bar that we have seen in the UK aid programme for many years now, and not to relax that in the interests of some broader commitment to national interest. They came back to us quite firmly on that and have done some good things. Maybe we can talk a bit about that in a moment. We will have to continue to look at this, Department by Department.
Q5 Chair: Our focus today in this first panel is on the Foreign Office and particularly the joint funds. Do you have any broader observations, Alison, on other Government Departments that spend ODA? In particular, are there any stars or any of particular concern?
Dr Evans: We have looked at other Departments. We have looked at the Department of Health in the context of our global health threats review. We have spent quite a lot of time with BEIS in the context of GCRF and now Newton. Are there stars? Possibly not, no, but they are all now actively thinking, working and making sure they can defend their ODA spending.
We have some challenges in the context of BEIS, particularly because of the need to balance its criteria around achieving research excellence, for example, and the basic rules about whether something is ODA-able, whether it contributes to poverty reduction and boosts the welfare of developing countries. That is a balancing act, whether you are talking about the GCRF or Newton, that they have to undertake. Again, we are keeping a close eye on that.
We have not come across major transgressions yet, but that does not mean we should not continue to be very vigilant. To finish, as the rules around ODA potentially begin to morph themselves, with the current debates happening in the OECD, it adds even more urgency to our need to keep a focus on this.
Chair: We are going to dig into a bit more depth on the Prosperity Fund.
Q6 Lloyd Russell-Moyle: Following the ICAI review of the Prosperity Fund, the Government began developing a portfolio of indicators for the fund, which had not been there before. The fund had already endorsed 19 business cases. ICAI therefore argues that the new indicators are being retrofitted around what you have already approved. How can this be justified? Are these indicators really worth the paper they are written on?
Joelle Jenny: I hear the concern, but it is important to recognise that the fund is at a really early stage. The business cases that have been adopted by Ministers are at a very high level. The key performance indicators that have been developed will help us monitor impact at the level of the portfolio so that we can really track the implementation and the learning that comes with it. It is important to recognise that the Prosperity Fund is an innovative instrument. It really aims to help and support middle-income countries to improve the business environment, create conditions that diminish opportunities for corruption and rent seeking, make access to financing much more possible and address challenges around rapid urbanisation, climate and detrimental practices that drive fast economic growth. All of this requires innovative approaches.
In the design of the fund’s interventions, we set the high-level key performance indicators, but then each and every intervention has to have very detailed, objective theories of change, performance indicators and milestones so we can really monitor what is happening on the ground. Again, that is why we have invested in external evaluators who can help us make sure we are as robust and thorough as we can be, in both the design and the implementation.
This should enable adaptive programming. We need to be able to see what works. We need to be prepared to accept that some of the interventions may not work because they are difficult or novel. We need to recognise that immediately, so we can make the adjustment that may be required and get the learning. It is also important to remember that the fund is relatively modest. We are talking this year about less than 1% of overall ODA, but it is enabling all Government Departments to work together, with the best expertise the UK has to bring to bear, to build a rules-based international order that supports trade and inclusive growth, in a way that benefits the global international order.
Daniel Davis: If I may add a few points, the key performance indicators that you asked about very much reflect the 19 business cases that you note have been endorsed. They were concept notes at the time, which were endorsed by Ministers. There was an iterative process of finalising those key performance indicators on the basis of the ideas that had come from the portfolio and the priorities set by Ministers, underpinned by the evidence. The key portfolio indicators were agreed internally in about March this year. They were only published for the first time in our annual report this year, but there has been quite a lot of time for us to ensure that the business cases are aligned to those key performance indicators. They very much reflect what programmes wanted to do initially, and then the programmes were at an early enough stage to ensure they were aligned with those key performance indicators. All programmes are aligned with those key performance indicators in some way.
Q7 Lloyd Russell-Moyle: Was it still the case that the programmes were agreed first and then the indicators were developed afterwards?
Daniel Davis: No, that is not the case.
Q8 Lloyd Russell-Moyle: Give me the timeline, because it was a bit confusing there. In March, you said, the indicators framework was developed. When were those 19 business cases approved?
Daniel Davis: The 19 business cases have been approved over a period of two years.
Q9 Lloyd Russell-Moyle: Before March or after March?
Daniel Davis: When we met last year, we only had two business cases that were approved. For all the other business cases, we have had an endorsement of the ideas by Ministers, and then they have needed to go through a departmental process for final approval.
Q10 Lloyd Russell-Moyle: Yes, I get that, but you have added extra dates in this timeline now so I am just trying to work it out. March 2018 was the date the initial indicators were bolted down. How many business cases were agreed after that date?
Daniel Davis: Only two business cases were agreed before that date.
Q11 Lloyd Russell-Moyle: Only two were agreed before March 2018[1].
Daniel Davis: Yes, only two were finalised and signed off by the Department.
Q12 Lloyd Russell-Moyle: The others were all being worked on before that, but were still an open book and you could have rejected them.
Daniel Davis: Yes, indeed.
Q13 Lloyd Russell-Moyle: We will come back for a comment on that. You mentioned in your comment just there about the global economic order and needing to support that. That was the phrase that you used. Following ICAI’s review of the Prosperity Fund, you introduced additional guidance on gender equality and the ODA requirements, but ICAI would like to see a more explicit weighting of primary purpose—i.e. the development goals and poverty alleviation—over the secondary benefit of the UK interest. Your comments just a second ago seemed to indicate that the primary interest was this global economic order and not poverty eradication and development goals. What do you say in response to ICAI’s criticism and your own presentation to us, just then, of what the Prosperity Fund is?
Joelle Jenny: We all recognise and accept—DFID, the World Bank—that aid alone will not be sufficient to lift everyone out of poverty. All the research proves that investment in infrastructure, better business environments, financial services and access to finances, growth, anti-corruption, are clearly correlated with poverty reduction. The overall recognition of those factors is also captured in the sustainable development goals. The fund works to three of those sustainable development goals, and puts particular emphasis on gender and the inclusion of women as a really strong one. That is a good example of where we welcome the scrutiny of ICAI and the recommendation that enables us to take action on them and refine our guidance on this. That was a very good example.
That being said, as I explained earlier, no single business case or concept note will proceed if it does not meet the ODA definition set by the OECD DAC. If you will, there is a 100% weighting for the criteria of meeting the requirements set by the OECD DAC on eligibility of ODA, which in itself encapsulates the fact that the primary purpose is inclusive growth. Once that test has been successfully passed and the interdepartmental governance is satisfied with it—and that includes DFID, which has the best expertise in assessing that—the projects are assessed against the other objectives, including their secondary benefits.
Q14 Lloyd Russell-Moyle: Would you say that all the projects on the Prosperity Fund fulfil that ambition under the International Development Act of poverty reduction?
Joelle Jenny: Absolutely.
Q15 Lloyd Russell-Moyle: They all have that. When you presented it earlier on—maybe this is a question to leave hanging there—why was it presented as trying to maintain or further the global economic order? That was the phrase you used. Surely poverty eradication sometimes needs the disruption of a global economic order that is not working for people. It is not always about maintaining and building. Sometimes it is about changing. You did not seem to acknowledge that.
Joelle Jenny: What I mean by the international order in relation to business and growth is an international order that enables more open, better regulated and less corrupt economies, as I referred to at the beginning when I was describing the purpose of the fund. That is because, if we do not support more open, less corrupt and better regulated economies, those who suffer most are the poorest. It is the poorest who are robbed of the opportunities to set up businesses, to access small financing and to seek redress if their properties have been damaged. It is the poorest who suffer if there is unregulated growth that has huge environmental impacts. It is the poorest who suffer in most of those environments, particularly through rent-seeking practices. What I mean by supporting the international trade and business order is enabling those open, more transparent and fairer economies, because we know this is a requirement without which we simply will not succeed in delivering the SDGs. It is not the answer in and of itself. It is not going to achieve that alone. That is why what DFID does is absolutely critical in addressing other aspects of poverty.
Q16 Lloyd Russell-Moyle: That co‑ordination between Departments, as you mentioned earlier on and just then, is really important. After ICAI’s review of the Prosperity Fund, responsibility for monitoring, research, evaluation and learning was transferred from the FCO to the Cabinet Office. My understanding is that that resulted in delays. Is that correct?
Joelle Jenny: I do not recognise this.
Dr Evans: The contracts have to go to the Cabinet Office to be signed; it does not have responsibility.
Q17 Lloyd Russell-Moyle: The contracts all have to go via the Cabinet Office. Explain the process. That would help me and probably others. Just explain the process of what is co‑ordinated and by whom, very briefly.
Chair: Can you be a bit briefer? We are going to run out of time, unfortunately.
Joelle Jenny: The Joint Funds Unit was established to facilitate the interdepartmental work around the funds. The Joint Funds Unit is part of the Cabinet Office. We are there as a secretariat to bring different Government Departments together, to facilitate the work of the board and to support the work of Minister Lidington, who is the senior responsible officer and oversees both of the funds. The senior responsible officer is actually John Mahon at DIT, but Minister Lidington provides the overall review of the funds and their impact. Within that, once the boards have agreed on a programme, that programme sits on the baseline of the Department that will deliver it. For example, DFID has responsibility for the green growth fund. The monitoring and evaluation contract sits with us at the Joint Funds Unit because it cuts across all the programmes. It is not limited to one programme on one departmental baseline. The monitoring, evaluation and learning contract is a facility available to all programmes, all Government Departments, to help with the design, monitoring and learning around each of the projects.
Q18 Lloyd Russell-Moyle: I do not quite understand why—if the primary purpose of the Prosperity Fund is ODA, if it is all meeting the idea of poverty alleviation and if, as you said earlier on, the experts leading in that field are DFID—the joint unit does not sit under DFID, under the responsibility of the Minister who everyone in the public would suggest has the expertise and should be the person responsible, rather than a Minister who, quite frankly, most of the public would not associate in their minds with these kinds of funds.
Joelle Jenny: The funds fill a small niche, complementing the work that DFID primarily does in relation to poverty but that other Government Departments are also doing. The funds act as a catalyst. There is a willingness to take some innovative design approaches.
Q19 Lloyd Russell-Moyle: Would that be impossible under DFID? Could it not be innovative and co‑ordinating? DFID is able to co-ordinate the VNR, for example, on the SDGs across Government Departments. Why is it not able to co-ordinate other things across Government Departments?
Joelle Jenny: A choice was made at the time of the establishment.
Lloyd Russell-Moyle: It was probably not your choice.
Joelle Jenny: Those funds enable the Government to also support national security objectives in a very consistent manner that is complementary to what the Departments do. They provide a connection between the overall objective that DFID is serving and the broader objectives that the Government are pursuing, as represented by the National Security Secretariat.
Q20 Chris Law: The most important thing to the public and to us is transparency and accountability for every penny spent under ODA. One thing that came up is that, despite an earlier ICAI recommendation, the way the data is presented on the IATI registry remains confusing. I have had a look at it and I do not understand it either. I wanted to ask why more Prosperity Fund information is not on the development tracker website. Is it going to be on there, and how soon?
Joelle Jenny: As I said, we are on a trajectory whereby there is a lot more information available than there was. We have worked with DFID to seek its guidance on transparency. We have worked with the IATI to seek its guidance on how to publish the information. We have made the information that I listed earlier, including 13 business cases, all the financial information, all the annual reports and so forth, available both through gov.uk and through dev-tracker. We are constantly looking for suggestions on how we can further improve. Again, these are early days in the performance and delivery of the fund. We have seven programmes currently in delivery, out of the 26. There is plenty of scope still to see how the data is coming across and to gradually make sure that the data is meeting the requirements of “publish what you pay”.
Daniel Davis: It is a very good question. At the moment, there is more information on gov.uk than there is on dev-tracker.
Q21 Chris Law: Why is that, though?
Daniel Davis: There is an issue. The way other Government Departments work is that they send the data to IATI and then IATI sends the data to dev-tracker. There is an issue about the timing and sequencing of that, and the tagging of the data on dev-tracker. You can find the data on dev‑tracker, but we need to get the tagging system right so all the Prosperity Fund information is in the right place. You will actually find all the business cases and the quarterly financial information on gov.uk, and an increasing amount of it is on dev-tracker.
Q22 Chris Law: I suppose the question for ICAI is whether you are satisfied with the responses you have just heard.
Dr Evans: I really want to come back to Lloyd’s question too, if I may.
Chair: You may.
Dr Evans: Let me answer that specifically. When we first started working with the Prosperity Fund on this review, there was virtually nothing in the public domain. Let us be clear; it was just a placeholder on gov.uk. They have come some distance, but being IATI compliant is the beginning, not the end of transparency. Accessibility—the ability to manipulate the data and really to interrogate it—is where we need to be, and there is a distance to go. The most important thing now is not to lose momentum or get caught up in the technicalities—it’s a software problem—but to have a completely full-frontal commitment to public accessibility to this information. That is where we want them to be and that is where they still have to get to.
Q23 Chris Law: Who should be pushing that momentum? Should it be the Government? Should it be another Department? Is it you?
Dr Evans: It is clear that the Joint Funds Unit itself has a commitment to doing that. There has to be a commitment the whole way down the delivery chain too, right to post at country level, where the same principles of transparency and openness have to apply. It has to go to the top of Government, too. This has to be a principle for these funds that exist, as it were, outside the transparency architecture that DFID has established.
Q24 Chris Law: What about the timeline? When should this be at the point where we can all go into the development tracker and open it?
Dr Evans: The aid strategy commits by 2020 to be at least IATI compliant. They are well on the way to that. But, by 2020, there should be no Department that is not up with DFID, which is more than IATI compliant.
Q25 Chair: Alison, can I tempt you to respond to Lloyd’s point, particularly on this question of the architecture within Government and DFID not being in the driving seat?
Dr Evans: That is obviously a policy question, which in a way is above my pay grade too.
Chair: I am tempting you, as it is your final month.
Dr Evans: The reasons for that money being where it is have less to do with the comparative advantage of Departments, potentially, and more to do with the need to build cross-Government engagement with the spending of ODA. To the point you raised, Lloyd, about the Prosperity Fund and its commitment to poverty reduction, there is quite a lot of paper commitment now, but we would want to see three things being pursued vigorously all the time. The case for spending ODA in middle-income countries needs to be constantly remade and restated. Even if you are working within middle-income countries, are you targeting your assistance to those areas where poor people reside? Spending Prosperity Fund money in São Paulo is less obviously pro-poor than sending it to the poorer states of Brazil.
Secondly, are you focusing that money on sectors where poor people earn a living? Again, this question has to be asked constantly. Finally, are you working in a way that involves and engages with people who are poor and vulnerable? I am not entirely sure that is the kind of mindset that is working through the Prosperity Fund right now, but it clearly needs to be. I will leave it there.
Chair: Thanks, Alison. That chimes very well with the conclusions we reached when we conducted our inquiry into non-DFID ODA. Thank you very much. Can I thank our witnesses from the Foreign Office for joining us here today? Please feel free to stay in the gallery to listen to the second panel or to leave. Thank you very much indeed.
Witnesses: Dr Alison Evans, Ekpe Attah, Matthew Rycroft CBE, Juliet Chua and Beverley Warmington.
Q26 Chair: Welcome back. It seems like only last week you were with us last. We will try to race through 10 questions in 45 minutes. Let me kick off with violence against women and girls. In last year’s follow-up, ICAI noted that its recommendation to scale up the mainstreaming of violence against women and girls initiatives across programming had achieved little traction at a country level. This is a crucially important priority area. Do you accept, Permanent Secretary, ICAI’s point on that and, if so, what is being done to address it?
Matthew Rycroft: Thank you very much, Mr Chairman. Thank you to the Committee and particularly, on her final appearance here, to Alison, for everything she has done in strengthening ICAI and ensuring it is a very powerful instrument for helping us improve the quality and value for money of our ODA spending. I thought that the follow-up review on violence against women and girls was a typically helpful and useful one from ICAI. Since the report, we have stepped up our investigation of what works in this area. We have a £25 million flagship research programme called What Works, which is designed to create the evidence that we will use for future programming, through research.
Q27 Chair: On that, ICAI specifically asked that you use the evidence from What Works to improve future policies. Is what you have just said a commitment to so do?
Matthew Rycroft: Absolutely.
Beverley Warmington: Good morning. I am Beverley Warmington, director for conflict, security and humanitarian at DFID. I just wanted to quickly talk about that What Works programme. We had 15 interventions in 12 countries, and we are getting some quite interesting stuff coming out of it. For example, in Afghanistan, just through play in schools, we have seen a 50% reduction in peer-to-peer violence with girls.
Chair: Wow.
Beverley Warmington: But it is not only that; it is taking it back into the home, and we are seeing reduced violence in the home as well. We are seeing some quite interesting stuff come out. It is the same in Pakistan.
Q28 Chair: That is as a consequence of what particular intervention?
Beverley Warmington: Play in schools. It is the same in Pakistan with play in schools: 25% less peer-to-peer for boys, and 50% less for girls. We are getting some interesting stuff coming out of this. We are getting a repository of what works. We have called it What Works, and it does what it says on the tin. It is a great piece of research. Now that we have some of this evidence, we are looking at how we can use it in our country offices. The team in London is going to hold this repository of evidence, and explain to country offices and help country offices with this. We are getting the tools in London to help country offices mainstream this. The central London team will hold the excellence and expertise, and act as a hub. This hub will then give the experience to the country offices and help them to do it. That is how we are trying to mainstream this particular issue across some of our country offices.
Q29 Chair: Thank you very much indeed. Alison, what is your response?
Dr Evans: This is an area where we have broadly given a very positive assessment to DFID. We think its violence against women and girls work is genuinely a bit of a market leader. But we were concerned that, following our very positive review, we felt the commitments around scaling out and mainstreaming broadly across sectors within the Departments had come to a bit of a halt, hence our follow-up comment. We can see, though, that the Department is still minded to do this and wants to use its What Works initiative, which is a remarkable initiative, and we have seen some really positive stuff coming out of it, to inform that. All I would say is this. Do not hang around until it is perfect. Get moving. Work with what you have and build that experience over time, so you create a real community of good practice across the Department. This is an area where undoubtedly DFID is a global leader.
Q30 Chair: On a related issue, a major concern this year has been sexual exploitation and abuse in the aid sector. Can I ask Alison or Ekpe to respond on this? I know that ICAI has included this issue in a number of recent reviews. Can you tell us more about how, in practice, you expect sexual exploitation and abuse to be reflected in the reviews ICAI undertakes going forward?
Dr Evans: We have spent a lot of time thinking about this internally. Ekpe can maybe give an answer on what we as ICAI, as an entity, are doing. In the context of reviews, there are two things. We have adopted what we call a risk-based approach, in that we make sure we give particular prominence to this issue in reviews where we see it to be highly pertinent and relevant, and gradually fold it into our protocols for the evaluation questions that we ask.
We have, at our last board meeting, come round to the understanding that we also need a specific protocol for when safeguarding issues have not necessarily been centre stage in our approach, but when we come across something, or something is raised with us, as part of undertaking a review. We need an escalation process for very quickly red-flagging that and making sure DFID or others, including you, are alerted to that fact. Maybe you can talk a bit about what we are doing internally.
Ekpe Attah: We held a workshop over the summer with our contracted service provider, to make sure our processes were up to date and reflected best practice. We have now published our updated policies and procedures around reporting, and we informed the Committee of that at the time. They are reviewed and we look at them every month with our board and the service provider. We had no instances reported in the last year, but we are very mindful of the issue and reporting generally. As Alison said, we are going to publish, in the interests of transparency, our own protocol around how we deal with the SEA issue in our reviews. That was probably the missing piece. We are quite transparent about what we do internally, as an organisation, and with our service provider, but we can probably be more transparent about the second piece.
Q31 Mr Lewis: Good morning. This is quite a serious question to Matthew—perhaps one of the most serious questions, in terms of the work ICAI does. ICAI basically says that the figures you produce on the people you reach and the people you support are, in many ways, meaningless and misleading, and bear little relation to the actual difference DFID makes. This is directly connected to results and outcomes. There are a number of concerns. One is that that practice seems to be widespread. We are going to be focusing on maternal health next week. We have a very tangible example of that. DFID still refuses to publish tangible outcomes of its country programmes. That seems very strange given DFID’s country-led approach, which we all support and value.
Finally, in the context of DFID Somalia, for example, ICAI has found that the suggested figures for reach and people supported bear very little relation to the actual programmes DFID is operating and running in Somalia. These are some pretty serious suggestions of failures within the DFID system. Obviously, you are relatively new, Matthew, but you are head of the organisation, so I wonder what your response to ICAI’s view is and what you intend to do about it.
Matthew Rycroft: That is a very important set of issues. Before I answer, can I pick up two previous points? First, on sexual exploitation and abuse, I am very proud of the work the Department has led, not just within the British Government but probably across the whole of the development sector, to tackle safeguarding and make sure there is zero tolerance for all sexual exploitation, sexual abuse and indeed sexual harassment. Through the two summits we held in 2018, we have led the way on that and will continue to do so. Secondly, on violence against women and girls, the Secretary of State announced within the last month a new £50 million commitment to end female genital mutilation, which is the largest single-donor commitment ever on that crucial issue.
On results, we hang on every word of ICAI and there are some very important lessons for us. ICAI accepts that we do very well at the top level, the global level, thinking about the totality of our programmes. We do very well at the lowest level of detail, which is on each individual programme, but there is a missing piece in the middle. When we look at a portfolio of programmes that, in a horizontal way across the world, seek to tackle a global issue, or when we look vertically at all the programmes within a country, we have much further to go. We have taken that criticism on board and we are working very actively now to come up with a methodology that will meet our requirements, ICAI’s requirements and your requirements. In the course of the year ahead, we will seek to fill that gap.
I do not think that ICAI has gone as far as to say that this is a failure. It has encouraged us, quite rightly, to do more, particularly in difficult environments. Somalia is the example you mentioned, and there is some very important ICAI advice on our programming there. I discussed it in person with the DFID Somalia team last week, sadly not in Somalia but in Kenya, where some of them are based some of the time. The combination of the drivers of conflict and, indeed, poverty more broadly in Somalia is extremely complex. This is not an excuse for failing to put results at the heart of everything we do; I am just seeking to explain that it is very difficult in an environment like Somalia to be absolutely sure about what works and what does not, but we are committed to continuing to improve in that area.
Q32 Mr Lewis: We probably, as a Committee, would need some sense of timeframe, as to when you are going to come back with a definitive response to the overall concern about the way results are presented. As I say, I think words like “meaningless” and “misleading”, obviously not across DFID’s portfolio but in some circumstances, are worrying. I want to make the point that it is not just a criticism at country level; it is also a criticism about global reporting. ICAI’s concerns are not just country by country. It says that the way DFID describes the people it reaches and supports—its results, essentially—is pretty flawed at a global level as well as at a country level.
Juliet Chua: This is a really important conversation and we are thinking about it very hard at the moment. Our results framework has sought to be very strong at programme level. We are using data that, necessarily, at times is evolving. Data systems in developing countries can be evolving, but we are transparent about the methodologies we use. We constantly come back to test that and use the code of statistical practice to test it. The more global question you are asking is about our overall framework and how we think about results. As Matthew has said, we have taken on board the criticism about that missing middle question and we have been going through an internal process of reviewing how we think about results.
In the context of thinking about the next spending review and our overall strategic direction programme, we are developing methodologies around our country diagnostics, which will essentially require countries to see the way in which the portfolio of programmes has an impact across the board, and to test that. It is worth being clear about how results can be used for a number of different reasons. They are decision-making tools; they are there to drive practice, but they are also there to communicate and give accountability. There are things we seek to do that do not lend themselves as easily to straightforward, traditional counting. We are trying to get at the broader outcomes. We want to draw out that line of sight up to the SDGs and through that broader, longer-term development impact.
Q33 Mr Lewis: The point I would make is that sustainable long-term change is difficult to record. That is what DFID should pride itself on. The fact that politicians and Ministers may have different imperatives does not justify feeling the need to rush to provide information that is not accurate and may be misleading. I think we need some sense of when you are going to come back to the Committee—maybe not today, but perhaps write to the Chair—with responses to these concerns.
Matthew Rycroft: We are absolutely committed to providing results that are accurate and reliable. We are looking at this piece in the middle between the global and specific levels. We will come back at the latest in the annual review for this year.
Q34 Chair: Alison, do you want to respond?
Dr Evans: I do not want to prejudge a conversation that we will, I am sure, have next week about this. However, it is certainly the case that, particularly in a number of our impact reviews, we have pointed to what we regard as real problems with results calculations, partly because of their reliance on what we call reach data, which was a criticism that also existed in phase one of ICAI—this emphasis on reach. The results framework of 2011 to 2015 was a great piece of public communication, but it held within it also some genuine challenges for the Department in how it could represent its work across such a wide range of geographies and sectors, in a way that was also respectful of the long-term challenges of development. It simply could not meet all those requirements. In the end, we found that, in a way, many of the results that were reported against that framework were derived through algorithm, essentially. That inevitably sets up questions about what assumptions it is based on and whether it is the right fit for this particular problem. The Department is certainly looking at that. That is really valuable, but Ivan is absolutely right: this is an area we have come back to repeatedly through phase two.
Chair: As you rightly remind us, we will return to this next week in the sub-Committee on maternal health, but you also rightly remind us that this is not a new issue. It is a longstanding issue, going right back to the first phase of ICAI. There is a very real concern about this in the Committee; we want figures that are robust and accurate. If there is any sense that the figures are inaccurate or exaggerated, it undermines the true value of the work that the Department is doing.
Q35 Chris Law: ICAI’s annual report expresses concern that UK aid sometimes consists of “a scattered portfolio of programmes”. It quotes your governance work in Nepal and Uganda, which it reviewed recently, as one example of this. What steps are you taking to become far more strategic?
Matthew Rycroft: Let me kick off, and then hand over to Juliet. We have a top-down and a bottom-up approach. The top-down approach begins with the commitments in the manifesto of the governing party, plus the sustainable development goals and everything in our single departmental plan. The bottom-up bit of it starts with each programme in each country. Then we seek to make sure they meet in the middle, which is partly what we were talking about in relation to results at that middle level. Rest assured that we have a system of ensuring strategic cohesion across, for instance, our governance programme. We do that through people who are looking horizontally across the world at our governance programmes.
Juliet Chua: To come in specifically on the governance point, the ICAI report referred to the governance position paper that we have now put out, which essentially gives guidance to country programme officers as they are thinking about governance and how to drive greater coherence. It is important to come back to that results work, because essentially that provides the vehicle for ensuring that we see the whole portfolio of our programmes, not just the individual programmes, and test the additionality of how programmes sit together against each other. That, for us, is a really central drive across the Department as we go into the next spending review.
Q36 Chris Law: All six of ICAI’s scored reviews this year include a question on learning. In four of those areas, ICAI awarded an amber-red score for this. It is good to learn, obviously. It is vital to improving effectiveness and impact. What are you doing to improve your performance in this area and that of other Government Departments?
Matthew Rycroft: This question goes to the heart of the culture, actually of any organisation, but, for these purposes, of the Department for International Development. When I came in new in January, I was very struck that the culture was of a strong sense of learning with the new organisation. One of the bedrocks of expertise on which the Department’s culture is built is this real sense of wanting to learn what works in one part of the world or on one programme, and seeking to spread it to another. I was quite surprised that this issue of learning has featured in so many of the ICAI reports. I have had a very helpful discussion with Alison and her colleagues on the commission about that.
I would agree that it is a long-term effort to improve continuously the ability of the organisation to learn from what works. Sometimes, as in any matrix organisation, the bureaucracy can get in the way. There are people who look at the world horizontally and there are others who look at it vertically. We need to make sure that everyone is able to access the evidence of what is working and apply it to their own areas. We are seeking to make it easier for each individual in the organisation to do that. I want to underline this sense that it is a very driven Department that has a high level of expertise and a reliance on that expertise. There is no absence of will to learn from every ICAI report and, indeed, other constructive feedback.
Juliet Chua: We really welcome ICAI holding our feet to the fire on this. You have recognised the degree of commitment there is as an organisation to it. To Matthew’s point about culture, I would add systems and processes. To be able to transmit information across a decentralised complex organisation that is trying to address some of the most difficult challenges, you need the right systems and processes that support that. We have a senior group that sponsors this across the organisation, chaired by our chief economist and the director of policy. We have invested in the way in which we record decisions and transmit information across the organisation. We have a knowledge for development helpdesk. Our technical advisers provide a very strong cadre for trying to transmit what works. It is one of those things you have to keep coming back to, keep pushing and driving harder. That is absolutely where the commitment of the organisation is.
Q37 Chris Law: This is a more specific question. ICAI found that your efforts to include marginalised groups sometimes faced opposition from reciprocal communities. What are you doing to overcome this? That is obviously a fairly practical challenge.
Matthew Rycroft: This is on seeking inclusion of marginalised groups.
Chris Law: Yes, and facing opposition from both communities and Governments.
Matthew Rycroft: The culture of the organisation and, indeed, the whole sector of development is, on the whole, to work with the grain of what works within a country. That usually means working with the authorities of the country. That can be very difficult in certain parts of the world, including some of the poorest parts of the world, where, frankly, the biggest obstacle to us helping them meet the sustainable development goals is the Government of the country. There are exceptions to the rule but, on the whole, we work with the authorities. Sometimes, those authorities are least open to persuasion from a donor like the UK on issues like marginalisation of minority groups or others who are oppressed. That does not stop us; it just makes it harder, sometimes, to have a very direct, immediate impact. I recognise that this is a long-term issue, but we have some really compelling examples of how we have worked successfully even to shift the societal culture in some parts of the world, to ensure, for instance, that girls can get to school or that people with disabilities are not excluded from everyday life and so on.
Beverley Warmington: There are a number of areas in which we do this. We work hard in our programming on inclusion. Everyone has to say, when developing a programme, who would benefit and therefore who would be excluded. We look at it at the programme level. At the country level, we are looking at country diagnostics, which are much broader than our previous growth diagnostics, which include looking at marginalised groups. We are giving countries tools now to do this diagnostic of who would be left behind. Part of our “leaving no one behind” work is very much looking at inclusion at the country portfolio level but also at the programme level. We are asking, at that level, to assess the impact of each programme on marginalised groups.
For example, in our girls’ education work, we are looking at the marginalised, and we now have some centrally managed programmes in education. The second round of those look not just at girls who are in education and finding it difficult to access, but at girls who are outside mainstream education and how to get them in.
Q38 Chris Law: I wondered what ICAI’s thoughts were.
Dr Evans: Do I have a chance to respond on a few of those? There are so many. On marginalisation, we were very pleased to see, as part of the value-for-money review, that DFID had made a genuine commitment to including equity as the fourth E in its value-for-money approach. That is now going to take some time to work through its system, but it means that it is one of the critical components around which value for money has to be assessed. That is a really important development. Yes, we have absolutely seen, through the diagnostics and the revamped diagnostics, a desire to now be more focused around distributional impacts, as we call them in our inclusive growth review. That is, again, positive.
There is something, in a way, that relates again to this missing middle. Why is it the case that country programmes are not spending the whole time trying to understand, across the span of programming, how issues relating to marginalisation are being addressed and how you can build momentum across a portfolio of programmes to crack these deep-seated problems that keep people left behind? It is the blockages and drivers of exclusion that DFID really should be focusing on—not symptoms, but causes. Again, that needs to come together at the country programming level. That is something to watch, going forward.
On learning, we are very positive about the amount of investment that DFID puts into building evidence—formal evidence, if you like—particularly around What Works. It is a major contributor to the research body on international development. We definitely see some cases where that knowledge is translated down into programming through the business case process and so forth. We are constantly looking for evidence beyond the paper that learning is part and parcel of everyday activity; it is not an afterthought, it is not something you only do on a Monday, and it is not something you only do in the face of positive examples. It has to happen in the face of things that have not worked and where there have been mistakes. That is the cultural piece. That is the permission to say, “We are not getting this right. Can we really now spend a bit of time trying to work out how to get it right?”
There is one thing that struck me, and I have been in and around this business a long time. In quite a few country offices that we have visited of late, there are not many hours of the week or the month that DFID staff are out of the office in the field. They may be down the road in the Ministry of Finance; they may be down the road in the Ministry of Health, but they are not out in the field, with mud on their boots, as much as I would expect and would want to be if I was in their positions. That needs to change. That is where so much of the learning happens. The ability to feed that back into the organisation really is valuable.
Q39 Lloyd Russell-Moyle: In its follow-up of the amber-red review on exit and transition, ICAI expressed concern that too little progress had been made, with most of the actions still in planning stage. An example that ICAI gave was the need to communicate more clearly the relationships with countries such as China, where the bilateral programme has now ended, and India, where financial aid has ended. The Government partly accepted these recommendations but ICAI still had some concerns. Can you summarise the current status of UK aid relationships, particularly with India and China—the direction of travel we are going in, how quickly and how far along that road we are?
Matthew Rycroft: Yes. The most important thing to say is that, in every single country in the world, however poor it is now, our overall aim would be to transition away from aid towards a more normal, balanced, mature partnership between two countries. That is what we are seeking to do in every part of the world. Some countries where there has been a long tradition of DFID and British Government involvement are relatively close to that point; others are many, many years away. China and India are two examples of countries that do not receive any financial aid from the Department, and quite rightly. They do not need our aid to lift the remaining people in those countries out of poverty.
What we do have with those countries, though, is a development partnership, working with them to ensure that, as they spend their own development assistance in other countries, they do so in a way that maximises the development benefit. In relation to China’s belt and road initiative, for instance, one significant geostrategic issue for the Department in the coming years will be the extent to which we seek to influence that, working alongside the Chinese, or whether we step aside and let them get on with it without any form of influence. That is an open question and one that will be worth watching.
Q40 Lloyd Russell-Moyle: Alison and ICAI, are you happy with that kind of response?
Dr Evans: We understand that this is a challenging set of issues for the whole of UK aid. Gaining clarity is partly dependent on setting out these clear strategic directions both for the Department and, to some extent, for its relationship with other Departments. Our principal concerns here are around transparency, including transparency with the public, being very clear about the development offer in relation to these two countries, and what the transition to a different kind of partnership looks like, but having that underpinned by a set of principles that are also very transparent, which would apply to other countries when they come to a similar stage.
It was the absence of a common set of principles, even though the specifics vary country by country, that we were particularly worried about. We will come back again and look at this, because we know the situation has been somewhat delayed but is now focused on as part of the strategic direction process that Matthew has put in place in the Department. However, wherever we go, we trip up over this question of what ODA should be doing in China or what it is doing in India right now. We are looking at the Newton fund right now, and we have similar questions. It is very complicated. There is a very high bar to be achieved in ensuring that what is done with ODA money in those contexts is absolutely in line with the International Development Act and the ODA requirements.
Q41 Lloyd Russell-Moyle: Do you think DFID has that bar right, at the moment, or is it still working to a moveable line?
Dr Evans: DFID knows absolutely where that bar is. I am less convinced that other Departments do.
Matthew Rycroft: The Prime Minister in her Cape Town speech talked about putting development at the heart of our international agenda. That is a very positive place for development to be, from DFID’s perspective. Of course, development is not just about the aid budget. It is also about talking about development in the broader sense and putting other tools at the British Government’s disposal to the benefit of these shared cross‑Government objectives of reducing poverty, ending extreme poverty and, indeed, implementing all the sustainable development goals.
Q42 Mr Lewis: This is a question to DFID. Despite earlier ICAI recommendations in relation to DFID Somalia on inclusion and human rights, ICAI’s feeling is that DFID has not really responded positively to the proposal to focus on inclusion and human rights. Do you have any response as to why that is the case?
Beverley Warmington: The question was about whether we are doing enough to include marginalised groups and then doing enough on human rights activity. We take that really seriously. The first thing I want to say is that we agree with ICAI: human rights and inclusion are really important. The UK’s work in Somalia is designed to improve human rights and the rule of law. We do not tolerate human rights violations, and we take all possible steps to respect and protect human rights. I just wanted to say that at the outset. We take it very seriously in our programming. As I have said, at the level of programming, we consider the risks and opportunities for marginalised groups, and we are expected to be quite explicit in the programming about that. That includes women, people with disabilities and vulnerable communities. There is a body of practice—of work—in the programming and what we do.
I wanted to mention a couple of our programmes, if I might, just to explain a little bit. There is work that directly responds to this challenge of ensuring inclusion. We have something called the Somalia Stability Fund, which has a real emphasis on bringing women in at district level to work with women’s engagement. That has had some really good successes. Then there is something called social norms and participation, which looks at women’s participation and social empowerment in opening up space for political conversation. There is directly addressing women’s engagement and then there is changing the dynamics in country through engagement. We are working on both these types of programmes to tackle that. We are looking at this quite seriously.
We have a really big humanitarian programme in Somalia. It is a very big programme. Our next phase of monitoring for that is doing some really interesting stuff on beneficiary feedback. We are hoping to get some interesting new work on beneficiary feedback in that humanitarian programme, so we can see how the people we are working with are benefiting and get real‑time feedback on how we are doing. That is going to be electronic feedback. We are hoping to move on that.
We do take it seriously. We are doing work on it. We think they maybe slightly underestimated the work we are doing, but it is a good thing to keep us going on and something we take very seriously in Somalia.
Q43 Mr Lewis: On a similar line, there is a question of oversight and political support to private contractors. Again, ICAI has raised this specifically in the context of Somalia. You have tried to explain that you feel you are responding to the concerns, but ICAI does not feel you are. It might be useful if ICAI at this point could explain why, in terms of the follow-up reports, you still do not feel the message has been received and responded to.
Dr Evans: Part of that, to be fair to DFID, relates to the 1HMG approach in Somalia. That point was also focused on the need to underpin that whole response with an inclusion and human rights lens. While we see DFID, in its own programming, taking up this particular mantle—and it certainly was working on issues of inclusion, but this has become much more formalised and much clearer—we want to see that underpinning the thinking and approach to programming across the whole of 1HMG. Of course, that is CSSF money as well, which is coming into Somalia. Particularly on the point about human rights, it is something that we would want to see as underpinning the whole of the response. We are not sure that is the case yet.
Q44 Chris Law: In ICAI’s inclusive growth review, it recommended—this is quite a long quote, so I will do it verbatim—that “DFID should provide more guidance on how to build a portfolio that balances investments in long-term structural change and job creation with programming to increase incomes for the poor in existing livelihood areas, taking into consideration the time required for economic transformation in each country context”. While DFID partially accepted this recommendation, it has not yet identified any concrete actions to address it. Why not, and when is it going to happen or begin?
Matthew Rycroft: We totally agree, by the way, with the bit of the ICAI report you have just read out, Chris. We are totally aligned with ICAI on the importance of inclusion in economic development. The bit that we are currently doing is updating our country development diagnostic tool that will look particularly at governance and inclusion. The bit that we did not think was necessary was to create some generic, potentially global, guidance on this set of issues, simply because the challenges in this area above all others are so country specific. We are seeking to empower our country officers to get this right in relation to their country. We did not think we needed additional global guidance in order for them to do that.
Dr Evans: On this point about guidance, we have been very careful not to overplay our hand in suggesting guidance. Where we do think it is relevant, it is because there are shared understandings. While there is a lot of country specificity, there are some questions you have to ask everywhere. That is really what we want to see enshrined. The slight danger is that if you simply give all that responsibility to a country office, it will reinvent those questions every single time it goes about it. There could be a template to say, “Your answers will be different, but these are the questions you need to be asking everywhere.”
Matthew Rycroft: Fair enough; that is a very good point. Our answer is probably that, in the country diagnostic tool, that sort of template of questions is there.
Beverley Warmington: We are providing a suite of tools to the country office to analyse who is benefiting and who should be benefiting. I guess it is just language. We think a suite of tools is the best way for a country to do that analysis, and it is better to take those kinds of decisions at country level, depending on the nature of the development trajectory the country is on. We do provide that. Whether you call it guidance or a suite of tools, it is there to help the country office make those decisions.
Matthew Rycroft: This point is very much linked to other big points we have been discussing: first, about results; and secondly about the role of empowered country offices. We will pursue it, as we tackle this big issue about the missing middle on the results agenda.
Q45 Chris Law: As part of this wider “leave no one behind” commitment, DFID has also begun to disaggregate results data by age, sex, disability status and geography. ICAI argues that this is not sufficient for monitoring distribution, impacts or obligations to ensure programmes do not cause unintended harm. Do you agree?
Matthew Rycroft: We absolutely agree with the imperative of having disaggregated data. We have a data programme, which is seeking to provide that, making sure that everyone counts and is counted, so we are able to drill down into that data and apply it to whichever potentially excluded group we wish. That is much harder to do than it sounds because of the scale of the requirements, but we are absolutely committed to doing that. We have four “leave no one behind” trailblazer countries, in the form of Rwanda, Zimbabwe, Bangladesh and Nepal, that are looking to explore and share what works in terms of mainstreaming the “leave no one behind” thinking in how they design and deliver their programmes.
Q46 Chris Law: ICAI has also stated that “there has been no new guidance or other concrete action to ensure that inclusion is addressed”. Given the scale of the exclusion of women, young people and marginalised groups, how can this be justified?
Beverley Warmington: We talked about that a little bit. The new country development diagnostics that each of our country programmes is now undertaking have all this additional guidance in them. Country offices are now expected to have a look at who is benefiting, which groups are being excluded, the broader governance issues and fragility, with help from the central London teams—and in Abercrombie House, not just in London—that have expertise in any of those areas. We are offering that kind of guidance, but countries are doing that diagnostic at the moment. We are looking across the piece at all of that, so we should get a pretty good comprehensive view of the development journey the country is on, where we can add value and where the best piece of DFID work can add value. We are doing that at the moment. It is a much broader diagnostic than we have done before and it has taken us some time to pull it together. Countries are undertaking it at the moment.
Dr Evans: We are looking forward to seeing the results of this broadened diagnostic. It is the direction DFID has chosen to go in, in relation to these issues of marginalisation and inclusion. Let us now wait and see how that unfolds. We very much hope that DFID, as an actor in the data space, continues to keep the pressure on for countries themselves to collect ever-better data on issues of disadvantage, because trying to do that from the centre is very hard. You need a country programme to bring that back.
Q47 Chair: The final questions are for Alison. You are about to hand over to your successor, who is sitting behind you. Welcome, Tamsyn. Your successor has been recruited and one of her first tasks is to be part of the process of recruiting the other two commissioners. Based on your experience, can you say what qualities we should be looking for in the other two commissioners? In particular, how important are investigation skills in terms of those two commissioners?
Dr Evans: Investigation skills are very important, but there are a whole number of disciplines that, in a sense, support investigation skills. It is not really for me to then dictate exactly the professional background of future commissioners, but in my tenure I have benefited hugely from having commissioners who bring very different skillsets and experiences of international development to the table. Being able to have a board, therefore, of multiple talents is a real strength of ICAI. But it is really the ability to inquire, to know what good inquiry looks like and to build the relationships in such a way that you are able to really get to the nub of the issues and the problems.
There is one thing I would say about the role of scrutiny generally. We have pushed very hard in this commission the message that scrutiny is a means to an end. It is a means to better accountability, but it is also a means to doing better. To do that, you need commissioners who can be objective, impartial and capable of investigation but who are, crucially, willing to be engaged and who are there to listen as well as to tell. That is really important.
Q48 Chair: Alison, during your time you have seen a number of changes. One very specific change that was discussed a lot at the time was giving the Government a bit more time to respond to your reviews. Has that helped with the quality and punctuality of responses?
Dr Evans: It has helped with timeliness; we have not had any late departmental responses. It is still a mixed bag. We have been pleased in the case of some responses, but we have also seen slightly variable responses that introduce new language, such as “partially implemented”. We are not really sure what that is about. The heavy use of “partially accept” to us seems like a slightly grudging rejection. What I would like to pass to Tamsyn is probably another conversation at the beginning of her tenure to ask, “Can we agree what the purpose of this exercise is and how we can max out on its effectiveness?”, recognising that in the end it is the follow-up that really gives us the teeth. The responses set us off for that follow-up. It is still a bit variable and a bit of a work in progress. I would like to see Tamsyn spending a bit of time with the Department working that out.
Q49 Chair: Is red/amber/green a good way of doing it, or should they look at something different?
Dr Evans: I thought you were promising not to raise scoring.
Chair: Did we promise that? Sorry, I think it is only me who is obsessed with it.
Dr Evans: We have agreed that it is a blunt instrument. We have agreed that it is quite difficult to operationalise given the complexity of quite a lot of the topics we focus on. Scoring—yes, but traffic lights may need some attention.
Chair: That is a great note to finish on. Can I say a massive thank you to our second panel? We are really grateful to you for your evidence today.