International Development Sub-Committee on the Work of the Independent Commission for Aid Impact
Oral evidence: ICAI's review on DFID's approach to disability in development, HC 1495.
Wednesday 5 September 2018
Ordered by the House of Commons to be published on 5 September 2018.
Members present: Paul Scully (Chair); Richard Burden; Lloyd Russell-Moyle; Stephen Twigg.
Questions 1 - 43
Witnesses
I: Dr Alison Evans, Lead Commissioner, ICAI; Paul Spray, Team Leader, ICAI; Penny Innes, Head of Disability Inclusion Team, DFID; Richard Clarke, Director‑General for Policy, Research and Humanitarian, DFID.
Examination of witnesses
Witnesses: Dr Alison Evans, Paul Spray, Penny Innes and Richard Clarke.
Q1 Chair: Thank you very much, as ever. We have the ICAI’s review on DFID’s approach to disability in development. As ever, it is good to have the two teams alongside each other, to have what I hope will be an interesting discussion. I will kick off, and if you want to say anything as a brief introduction when you are first called, do feel free. First, I want to look at the methodology of ICAI. Alison, your methodology for the review was largely desk-based and did not include country visits. Why not?
Dr Evans: I will come to that question directly, but first I will take you up on your offer and say a few opening remarks to set the scene. We undertook this review to really understand and assess DFID’s progress in delivering on its commitment to disability inclusion. As we say in the report, the best estimates are that something like one in six adults in the developing world has a disability. There is a lot more to do in improving the data there, so that is potentially an underestimate, but it is obviously a very widespread concern and issue in development.
We decided on undertaking a rapid review, which was largely desk-based, because we recognised that a lot was changing in DFID and wanted to ensure that we were fully up to speed with that. I am going to ask Paul in a moment to elaborate on how we undertook the review and why we did not do country studies.
In the report, we acknowledge and give credit to DFID for its positive international leadership in the area of disability inclusion. We note that it played a very important role in securing the place of disability within the sustainable development goals framework. We also recognise that from 2016 onwards, there has been a stated policy commitment in the Department to put disability at the heart of everything that the Department does. We want to acknowledge the great energy that the Department brought to its partnership with the Government of Kenya and the International Disability Alliance to host the Global Disability Summit in July.
Reflecting on this Committee’s 2015 report that called for a step up in ambition on issues of disability in the Department, it is reasonable to say that the Department has responded well to the requirement to be more ambitious in this space. Our review also focuses very much on the hard implementation end of that commitment, and importantly we found that DFID has been quite slow to start delivering on its own stated commitment to systematically and consistently take a disability focus in all its areas of programming and all its work.
We find that programming is currently a bit too scattered and too modest, in a sense, to achieve the transformative results that DFID is seeking. There are promising areas, and I hope we have given those some prominence in the review, but there is not enough connecting those promising areas. They are still very much islands of success, and relatively modest and small in scale. We do not think there is enough cross-departmental learning going on to really take those isolated successes and turn them into a transformative agenda. The Department also needs to look internally at its own staffing and skills mix.
Notwithstanding the incredible energy and drive that the Department and its partners brought to the Global Disability Summit, and the positive announcements made there, we rate that there is still a considerable distance for the Department to go in really addressing the gap between its ambition and its delivery. That is what this review principally focuses on. I am just going to ask Paul to elaborate on methodology and how we got to that conclusion.
Paul Spray: We chose to do it in two phases. That is partly because we were trying to keep the burden on DFID fairly light at a busy moment for it leading up to the summit, but also because, from the point of view of a rapid review, we wanted to establish early on what the issues were in phase 1, and get into them in phase 2. Our main objective was to have a wide range of voices and viewpoints. We talked to 99 people in semi-structured interviews, about half of them DFID staff, and then to a number of people from the United Nations, the World Bank, CSOs and other bilaterals. We were trying to get a range.
We were particularly pleased to be able to talk with staff in DFID who had a disability themselves. We were helped in our own work, in that one of the members of our team was not only a disability expert, but self-identified with a disability. We did a number of desk reviews, both of research programmes—that is a really important here—and of delivery programmes. They key point there is that because DFID’s commitment was so late, if you like—2016—none of these programmes had got very far. We were never going to get impact figures out of those programmes.
Dr Evans: That is one of the reasons why going to country would not necessarily have shed considerable light on the programmes themselves. We wanted to make sure that we were able to do a rapid review in order to meet the policy window around the Global Disability Summit.
Q2 Chair: You do not think you lost out in any way by not going to country.
Dr Evans: We would always prefer that, and we always feel that we can add value. In this case, we had to make a fairly difficult judgment about spending more time on this review, and possibly finding modest additional information—we could probably have found a lot of evidence that things are not happening, because the programming is still very scattered and very new in many cases—over speed and being able to be involved in an ongoing dialogue with the Department about its ambitions in this area. We chose the latter, but it is fair to say that going to countries and being at the sharp end always adds value.
Q3 Lloyd Russell-Moyle: ICAI found that although 22% of DFID programmes address disability, only six of the 1,161 addressed disability as a principal objective. We have already heard about it being scattered. Do you think that is sufficient?
Richard Clarke: Good morning, and thank you very much for the opportunity to come along. I might just say a couple of words before responding to that question, if that is okay. I am Richard Clarke. I am the director‑general for policy, research and humanitarian at DFID, and I oversee this area of business. I wanted, first of all, to thank ICAI and indeed the Committee for their interest in this area. We have followed very closely ICAI’s work in this area, and found the report itself incredibly helpful.
We were very pleased that the report came out prior to the summit. We have been able to use its findings and recommendations to influence the way we are taking forward the work in following it up. There is really very little, if not nothing, that we would disagree with in the assessment we have just heard. Our priority now is very much translating the huge momentum and the very practical set of commitments that the Government of Kenya, the IDA and we were able to secure at the Global Disability Summit into a set of measurable, real outcomes on the ground.
To respond to your question, Mr Russell-Moyle, no, that is not yet where we would want to be. I might ask Penny to say a little about some of the detail in a moment, but we are currently taking forward work as part of planning for a new, revised disability framework that we will publish by the end of this calendar year. That will include two important aspects in response to your question. The first is making sure that every country office has an action plan for mainstreaming disability into its programming. I would expect that to increase. Secondly, we announced at the summit major elements of new programming to help country offices and central programmes respond to these challenges. Perhaps Penny will be able to say more.
Penny Innes: I am Penny Innes, head of DFID’s disability inclusion team. We in DFID take a twin-track approach to disability inclusion. That means we do some specific disability-inclusive programming, but we also want to mainstream across all of DFID’s other programming. The six programmes that you mentioned are the specific ones, and we announced at the Global Disability Summit two new programmes about disability. One is the Disability Inclusive Development Programme, which will support people with disabilities in employment, livelihoods and health. That is a large innovation, research and evidence programme. The other one is—through DFID’s civil society funding—Aid Connect, a window for disability where there are two consortia leading work on supporting people into employment. Those are two examples of specific programming.
We also want to influence across DFID to make sure that we have more disability-inclusive programming, working with country offices. They are already doing more. The summit had a big impact on raising awareness across DFID and galvanising new thinking and new programming. For example, in Bangladesh they have been working with disabled people’s organisations to influence their programming and have three new economic empowerment-related programmes, which will support people with disabilities.
Q4 Lloyd Russell-Moyle: After the disability summit, do you have an idea of where that number of six might grow to?
Penny Innes: We do not yet have a target we can tell you now. We are working on a disability framework.
Q5 Lloyd Russell-Moyle: That will be in the framework.
Penny Innes: Yes, and that will have targets.
Q6 Lloyd Russell-Moyle: Alison and Paul, do you think that the announcements and the outcomes from the Global Disability Summit are enough to address those concerns about specific disability programming?
Dr Evans: The announcements, as I said in my opening remarks, are really positive and set a really strong steer from the Department that UK aid is serious in this area. But there are real practicalities and crunchy issues about really making disability a key focus in every area of DFID’s work. I note that in the Secretary of State’s letter back to you, Stephen, following your letter to her around the global summit, she restated this commitment to make disability central to all business units across the Department. Those high-level announcements were great; they set a framework and a level of ambition. But there is a huge amount to be done to translate that, almost on a day-to-day basis, across the Department.
Doing programming in this area requires specialist skills. At this point, let us hope that begins to close the gap. We have some concerns that the gap is maintained by the fact that there are still a number of things the Department needs to do to create a really positive, enabling environment for disability inclusion, including specialist skills and recruiting more staff with disabilities. Let us hope, but we do not know yet.
Q7 Lloyd Russell-Moyle: We wait with bated breath for the further report later on this year. Richard, ICAI’s review points out that there was no requirement for DFID programmes to include disability-related matters. Penny has just mentioned the mainstreaming principle. Nor is there a requirement to consider the potential consequences of failing to include disability in measures. You have made the claim that you have mainstreamed disability across DFID’s work. Given that the programmes are not required to do that, can that claim genuinely stand up?
Richard Clarke: I would say we are mainstreaming the work.
Lloyd Russell-Moyle: But it is not yet mainstreamed.
Richard Clarke: I would see the Global Disability Summit as an end of a phase and a start of a second phase. We have had a long period of build-up. The team grew in preparation for the summit; galvanised activity around the world with our partners, multilateral organisations, donor countries and other countries around the world; secured a number of extremely important commitments; and started a process of, as the Secretary of State put it, absolutely putting disability at the heart of everything DFID does.
In some ways, the disability summit was almost a moment of stepping back and reflecting on that work, agreeing that set of commitments and really putting them front and centre. I started in DFID the week after the summit. I came for the day, and I have been working in DFID since then. You can really feel the sense in which DFID staff talk about the summit as being a major achievement. I have heard them talking about disability; they raised it with me during that period.
The next phase is about, as Alison says, translating that into practical delivery across the Department. Until we have the framework finalised—we are consulting with disabled people’s organisations about that at the moment—and until we have got to the point where the action plans are in place out in country offices, we will not have completed this work, clearly. In some ways, the disability summit was the start of the next very important phase of work.
Q8 Lloyd Russell-Moyle: Is there a commitment to ensure that every programme either includes disability or has done that analysis of explicitly why it would not?
Richard Clarke: Yes.
Penny Innes: Through the framework, we are proposing to bring some standards that all business units have to meet in areas like their own culture, staffing, programming, how they disaggregate their data, how they work with national Governments and, crucially, how they engage with disabled people’s organisations. That is really critical in helping us to do better on this. There will be a set of standards that business units have to meet.
Q9 Lloyd Russell-Moyle: Alison and Paul, is it realistic to expect DFID to include disability in every single one of its programmes, or should there be exceptions sometimes?
Dr Evans: We worked from the basis that DFID has already committed to what it called a systematic and consistent inclusion of issues to do with disability, which dates back to 2014 in the first disability framework. The fact that they are still working on this and actually have a long way to go needs to be understood in the context of a slightly longer timeframe. It is not a very long time, but it is longer, and of course these issues have been around for ever.
I will ask Paul to comment specifically on mainstreaming, but what we focused on was also something we heard from some of our stakeholders working with disability organisations or organisations with a strong focus on disability. They felt the burden of proof needs to be, “Why are you not including?” rather than, “Why are you including? How do you include?” It seemed to us to be a very good rule of thumb, which would mean in some cases it is not entirely appropriate and would never be in the main event. “Why are you not including?” seems to us to be the burden of proof that programmes need to achieve.
Paul Spray: In that context, the mandatory requirements for business cases and for the annual review of DFID programmes are at the moment pretty weak. By comparison, the Australians ask a couple of really interesting questions of all their programmes. They ask, “Have you tried to identify the barriers to including people with disabilities?” Secondly, “Have you tried to involve disabled people’s organisations?” Those seem to us to be questions that have rather more bite than the current ones. They are the sort of thing that one hopes might be in the disability framework. If I might say something further on the mainstreaming, the crucial thing is to have targets and timelines for mainstreaming. That was not in the old disability framework.
Dr Evans: Nor were resourcing and funding, and maybe we can have a conversation about that, too.
Q10 Lloyd Russell-Moyle: In terms of resource, Alison, you found some instances where concerns about cost per beneficiary, or the overall number of beneficiaries, had led to decisions not to address disability in the programmes, which goes to this burden discussion. Are there circumstances where you thought this was justified in the cases that you found?
Dr Evans: As a general point, this is a really tricky area. The value‑for‑money equation, particularly when handled by outsourced fund managers and others, leads to decisions around programming that tend to drive towards least cost. We have identified this in a number of reviews in different ways. In this area, of course, that poses real challenges, not least because very often you are dealing with a set of barriers that have to be addressed, which are costly to address, but also communities that are quite hard to reach, often very marginalised and sometimes small in number. You do not necessary get large beneficiary numbers.
You are dealing with groups that are sub-groups of sub-groups, very often. That tends to mess with the equations around value for money or what looks like value for money. We felt they needed to step back, take a look at how to deploy that equity and value-for-money principle in a really robust and consistent way, and communicate that to fund managers. I do not know if we saw examples, Paul, where we felt it was justified.
Paul Spray: It depends on the question that is being asked. If the issue is how people with disabilities can be included if the Secretary of State has said that they are to be at the heart of everything we do, the real value for money question is how most effectively they can be included.
Q11 Lloyd Russell-Moyle: The Secretary of State has also said, of course, that every pound must not be able to be better spent anywhere else.
Dr Evans: There is a tension there.
Q12 Lloyd Russell-Moyle: Yes. What do you say, Richard and Penny, about that balance?
Richard Clarke: I might just say a couple of words and then ask Penny to come in as well. I do not think this is an either/or situation. The way Paul described it seems to me absolutely right. If we are to be serious about the commitment that we will leave no one behind, we absolutely have to mainstream into our programmes ensuring that we can reach people with disabilities. We must absolutely do that in a way that gains maximum bang for buck. That is not to my mind at all a choice between spending money in one place and spending it in another.
Penny Innes: We need to get better at measuring impact. It does cost more, quite often, as Alison said, to reach the hardest to reach and people with disabilities, but the impact can be so much greater, and we do not necessarily demonstrate that as well as we could. Just to be clear, all business case submissions have to set out how the programme has supported people with disabilities. There is a process in place, and that is quality assured by our quality assurance unit. There are some mechanisms in place for new programmes.
The other things we are looking at are procurement processes, working with procurement and commercial departments on how to make our processes stronger, for example in the terms and conditions and supplier code of conduct. We have already put into the guidance on terms of reference that there should be questions about how people with disabilities are going to be supported.
Dr Evans: Could I just make a comment on that? One thing we heard stated in numerous ways, but pointing to a similar point, is that as a commissioning Department, which DFID largely is these days, it is those whom DFID works with and through who have to get this message. At the moment, that is not necessarily translating in the way that DFID perhaps intended when it set up these business cases.
Q13 Stephen Twigg: Just to develop further some of Lloyd’s questions, focusing on the quality in particular of disability programming, ICAI states in your review, ”For economic empowerment, we judge that DFID’s current approach is too small to be effective at scale.” They go on to say that it is a “small proportion of DFID’s investment in economic development”, and, “It is of course entirely appropriate to pilot approaches, but we would have expected a DFID programme to plan to take successful pilots to scale”. Penny, you have already referred to an example from Bangladesh that sounds encouraging. Can you give us other examples of how you are addressing that particular criticism?
Penny Innes: Yes, I can. For example, in Burma, the Livelihoods and Food Security Trust Fund has mainstreamed disability and is rolling out a monthly disability grant. That is more on the social protection side, which is really critical for underpinning economic empowerment. Also, at the summit we announced three new programmes. The two that I have referred to already are the centrally funded Disability Inclusive Development Programme and Aid Connect.
There is also a new strategic partnership with the Government of Kenya and leading private sector businesses, to support each other to offer more support for people with disabilities. We also recognise that we need to develop a more strategic approach on economic development, and we will be doing that for the disability framework. Also, we have been working with the World Bank, and it made some good commitments at the disability summit on social protection, so that 75% of its programmes would be disability inclusive by 2025. There was a raft of other encouraging commitments made at the summit by national Governments, which we will be working with them to make sure they deliver.
Q14 Stephen Twigg: Can you say a bit more about the partnership with Kenya and the scale of that? What sort of scale will that have?
Penny Innes: It is starting off with a few leading companies, including M‑Kopa, KCB Bank and Safaricom. It is starting off with a few UK and Kenyan businesses. They will have a strategic partnership and will meet regularly to share knowledge, learning and how they are putting it into practice. We hope it will scale up and grow within Kenya, and perhaps move to other countries as well. That kind of approach has been successful in other areas like our youth work.
Q15 Stephen Twigg: In the review of the eight delivery programmes that ICAI looked at, it pointed out that only two tackled specifically the issues of stigma and discrimination. In your response, you have not committed to scaling up funding in this area. Is there a reason for that?
Penny Innes: We think that addressing stigma and discrimination is incredibly important, and that is why it was one of the four themes of the Global Disability Summit. It is not necessarily about new programming. When we were preparing for the summit, we were able to dig a bit deeper into our understanding of stigma and discrimination. There was an external expert working group supporting us, and it identified four different areas and outcomes for stigma and discrimination. For some of those, it is relevant to do some programming, and for some of them it is not.
For example, one of them was new national legislation, policies, frameworks and strategies. A lot of those have come up as commitments for the summit, and we will be supporting Governments to put them in place, but that does not necessarily mean new programming. Similarly, data was an important area to really highlight the barriers that people with disabilities face. At the summit, we signed up to the Inclusive Data Charter and announced a new action plan. Again, it is through our current programming that we would generate more disaggregated data.
The area where new programming might be appropriate is addressing harmful behaviours and attitudes against disability. We already have some programmes in place, but we did not necessary feel it was right to create an incentive across all programmes, because we do not yet really understand the drivers behind those negative attitudes. There is some learning from other areas like HIV and violence against women and girls, but for disability the drivers can be more complex or rooted in religious and traditional beliefs. The areas are different and we want to understand better before we roll out more widely.
Q16 Stephen Twigg: We know from our own experience in this country and in other countries that legislation is necessary but not sufficient to address issues around stigma and discrimination. One of ICAI’s findings was that DFID is reluctant to commit to media and communications campaigns, even though they are often very influential in reducing stigma. You gave the example of HIV; that is a very good example of where those campaigns have been effective here and elsewhere in the past. Do you accept that you are reluctant? If you do, why are you reluctant?
Penny Innes: We are doing some of that. The new Disability Inclusive Development Programme is going to have some elements of that and will aim to reach 16 million people with messages. In Ghana, DFID Ghana is already doing some work in the field of mental health, and the same for Bangladesh, where the Better Health programme is doing some behaviour change communications. We are doing some, but the reason is that we want to understand the drivers better, because there is a risk of doing harm if we do not do it in the right way.
Q17 Stephen Twigg: Alison, what do you think about that?
Dr Evans: We accept, and can see in statements and evidence internally, that DFID is committed to the idea of this. It goes back to the disability framework of 2014. I am less convinced that the pace of change is really adequate. This is an area that is potentially quite uncomfortable for people to have to confront. Genuinely, particularly for people in power but not exclusively, this requires you to rethink your understanding of the world, your implicit biases and many things that lead to stigma and discrimination around people with disabilities.
I genuinely think that there is evidence from other areas that could be usefully drawn on to move on this more quickly. Penny is right that in some settings one has to understand local context, but I genuinely believe, and we felt quite strongly, that there is probably sufficient information out there, whether it is around violence, harmful practices or HIV, to provide DFID with the evidence to move forward in a more forthright manner.
It is about embracing this in its totality, which is both working internally in DFID and working in whatever way is feasible through programming. That does not always mean a component looking at stigma and discrimination, but it means having it as part of the mindset driving programming, which is one reason we think change at country level would be very helpful. It would allow the conversations to happen about what is limiting change and what DFID can do to contribute, in trying to unleash positive change for people living with disabilities. We do not really see eye to eye on this one with the Department, and we are a bit concerned that some of the external reaction to DFID’s work in the past on social norms work has forced it to pull back a little on creative interventions in this area.
Q18 Stephen Twigg: I was going to ask this before you said it. In a sense, you are saying that there is risk aversion because of, for example, the Ethiopian work on gender.
Dr Evans: Yes, I think there has been some of that.
Q19 Stephen Twigg: Are there any comments on that from DFID?
Richard Clarke: I feel strongly that issues around tackling stigma and discrimination were a recurring theme that was taken very seriously at the summit, both in terms of things the Secretary of State herself said and people who were speaking from the stage and in the plenary sessions. It was deliberately and very consciously, as Penny says, one of the absolutely fundamental strands of the summit, and it will be one of the fundamental strands of the work that we take forward as part of the framework.
ICAI is absolutely right to challenge us and push us on pace, and we will take that seriously in terms of how quickly we can do this. It is also absolutely right to say that, as Penny has said, this needs to be rooted in what works locally. I do not think we are at all uncomfortable with the challenge, but we do want to take it very seriously, bearing in mind, as Penny says, that if you do not do it quite right it sometimes does not work well. That is driving our thinking in this area.
Q20 Stephen Twigg: There is a distinction between, “Will it work?”, and, “Will it open us up to criticism in the media because it could be misrepresented or misunderstood?” Will you have a strategy to avert the latter?
Richard Clarke: We are focusing on what will work as we think about how to do this.
Q21 Stephen Twigg: You will have a strategy for where someone comes along criticising the programme.
Penny Innes: The answer to that is having good outcomes and good measurement of outcomes so you can demonstrate what you have achieved.
Q22 Stephen Twigg: ICAI in the programmes that it reviewed only reviewed one that had a specific focus on issues relating to learning disabilities, or intellectual disabilities, as they are described in the review. What more is DFID going to do, both on addressing issues around learning disabilities and more broadly on mental health, as part of this work?
Penny Innes: We definitely need to do more in this area. We want to do more, and we want to do it in a way that builds our understanding of how best to help people with intellectual disabilities. We have been working quite closely with Inclusion International, which speaks on behalf of people with intellectual disabilities, and we were delighted that two people co-hosted the summit and we were able to support them through that. That felt like quite a big moment that has not really been seen before. We should be doing more in our programming as well. We have some examples: our central catalyst programme is supporting people in Bangladesh, for example, and supporting Inclusion International in countries like Burma, Cambodia and India. We are doing pockets, but we need to do more and share understanding of how best to do it.
Dr Evans: I want to talk about some specifics here. The genuine concern here is that, in an agenda that is about dealing with a whole range of disabilities, often intellectual disabilities somehow fall off the grid. They are often not as visible and, where they are, people have less in the toolbox to deal with them in these contexts. We felt very strongly that this was an area where DFID could really show considerable leadership, and we were a bit disappointed to see that the education strategy, which is relatively recent, does not really mention intellectual disabilities. It seems like a missed opportunity to do that mainstreaming, and we would like to see that addressed. Paul, do you have any specifics on this agenda?
Paul Spray: One interesting issue for people with intellectual disabilities is employment, because it boosts their status and enables other people to understand them better. We mention in the report an interesting initiative of the city of Seattle, although it is not a developing country. They have managed to redefine a whole number of jobs in their departments so that they are suitable for people with intellectual disabilities. They have managed to get 100 people with intellectual disabilities into such jobs. That is a public sector initiative, but clearly the private sector could do the same.
Stephen Twigg: The term “intellectual disabilities” is not one I am very familiar with. Is it synonymous with learning disabilities?
Paul Spray: Yes.
Q23 Chair: Out of 10 programmes ICAI studied, only two made significant use of disabled people’s organisations—NGOs run by people with disabilities—as suppliers or delivery partners. Do you think you should be making greater use of DPOs?
Penny Innes: We definitely should do so. One of the things that country offices have been doing more through preparing for the summit is engaging with disabled people’s organisations in country, and working with them to influence Governments to come up with commitments, but also engaging with them on their programming. We need to move to the next step, where they are more actively supported to deliver or to work with DFID. Some country offices, like DFID Bangladesh, as I mentioned, are already doing that, but we could do more. It is also fair to note that a lot of these organisations are really low in capacity and are small. We have a role to support them to develop their capacity.
Q24 Chair: ICAI also found that DFID does not require its suppliers to employ people with disabilities or to follow universal design principles. Why not?
Penny Innes: That will be addressed through the changes I mentioned in procurement.
Chair: It is just a lag.
Penny Innes: Yes.
Q25 Chair: Alison, do you have any views on that?
Dr Evans: It is partly that the messaging just has not been filtering down in the same way. Internally within the Department, there have been commitments to these things, which have not been properly translated, particularly through some of the fund management arrangements. We definitely heard frequently that disabled people’s organisations that were looking to be part of, for example, tenders for work or contracts found themselves simply not able to make the grade in terms of showing financial breadth or capability.
There is a need here to think very carefully about how those processes potentially exclude. At the same time, Penny is right that we have to be really careful. It is also possible to get this wrong and for really important DPOs to find themselves overwhelmed by the need to fulfil requirements of a major commissioning Department. This was a comment made not specifically about DFID but about a whole number of funding bodies. A representative of a DPO in one of our sessions said there was a vicious cycle: “We are not seen as capable because we do not deliver programmes. We do not deliver programmes because we are not seen as capable”. They need to break that vicious cycle.
Q26 Richard Burden: First of all, apologies for my late arrival. If I go over ground you have already covered, I must apologise for that. I would like to just get a bit more of a sense of the section of the ICAI review and DFID’s response that deals with working with partner Governments. I suppose this first question is mainly to you, Richard and Penny. DFID provides support to advocacy organisations in the disability arena, but it does not generally engage in direct advocacy itself with partner Governments. Why is that?
Penny Innes: We have done that a lot through preparations for the global summit. There was a lot of effort in country, by country offices, to work with other partners in country and engage with national Governments. That generated some really great commitments. For example, in Uganda, the country office worked with the Ministry of Gender, Labour and Social Development. They had an initial meeting back in March and then built up to biweekly meetings. The Ministry set up some working groups for each of the themes that were led by different Ministries. DFID supported that process and also supported a satellite summit in early July. That is just one example of the activity that was going on in the run-up to the summit to influence Governments.
We now need to follow up on that, to continue the momentum that was happening in country and to make sure that is carrying on, to hold national Governments to account. The situation is that lots of countries have signed up to and ratified the Convention on the Rights of Persons with Disabilities. They have stated an intention to do more, but they are not necessarily implementing very quickly. That was the aim of the summit: to push that forward and make faster progress. That is what we need to do now to follow up.
Q27 Richard Burden: What would ICAI’s response be?
Paul Spray: We saw a number of good examples where DFID country offices had engaged and brought together the Government and DPOs. For example, in the case of Rwanda, there was a social protection programme where the idea was labour-intensive public works. That programme existed, and it was the DFID office that brought in a disabled people’s organisation to talk to the Rwandan Government Department responsible for that, and got it to agree that people with disabilities could benefit from this if there was the right kind of work offered. The programme was changed and suitable work was offered. That kind of intervention, through conversations, is certainly possible. The summit was a great springboard, but this brings us back to the need to see not just action plans for all the country offices, but also some analysis behind it of the sort that we call theories of change.
Q28 Richard Burden: Could you perhaps say a little more about what you think DFID should do when its priorities perhaps run contrary to the priorities of partner Governments and national Governments? An example might be that DFID clearly has a requirement for universal design in schools. Penny was saying that, in terms of your own procurement, you are trying to further those sorts of objectives, but they may not align with the priorities of national Governments. What do you think DFID should do where there are cases like that?
Dr Evans: The first thing I would say is that this is not new territory for DFID. It works all the time in situations where it has policy positions that are not entirely shared by national Governments. It has quite a long track record of finding very effective ways to build an evidence base and a constituency of support, often working with civil society organisations and other donor partners, to try, over a period of time, to impress on partner Governments the value and importance of moving in a particular direction. This is territory in which DFID is very well experienced over very many years, and I would have thought it could absolutely throw its weight behind this, entirely recognising that partners have to be taken on a journey, but working in a systematic way to try to get that kind of policy change.
The summit was, as Paul said, a great springboard for getting a number of partner Governments to say, “Actually, this is the moment we are going to move on this.” There were some great announcements made, for sure. They can be used as peer support to countries that are not quite there yet. There can be opportunities to learn across different contexts where this is working. DFID is good at this; this is what it does well. It really needs to deploy all its experience to do this systematically and consistently with partner Governments.
Richard Clarke: I agree with all of that, and we absolutely will. That is our view.
Q29 Chair: ICAI recommended that DFID country offices draft theories of change for their disability work, but you have stated in your response that you do not plan to make this mandatory. Why would that be?
Richard Clarke: I might say something on this, if that is all right, and then ask Penny to come in as well. This was an area in which we partially accepted the recommendation. It is important to say at the outset that we absolutely agree with the intention behind this recommendation, and both Penny and I have already made several references this morning to the fact that it is very important to make sure country offices, and indeed DFID colleagues in the UK who are planning programmes that are centrally managed, are thinking in a way that is culturally in context and appropriate in terms of what will work.
The reason we have suggested this way forward and taken the approach we have—and we will absolutely continue talking to ICAI about how we do this as we build up towards the publication of the revised framework by the end of this year—is that we feel it is really important to start with the theory of change centrally, to make sure we are giving as much of a boost as possible to country offices as they think about how to mainstream this.
Here, we are balancing pace and wanting to crack on with delivery with making sure that country offices think about what works for them locally. If the recommendation and the aspiration is, as it absolutely is, to make sure that things are done in a way that works locally, we completely agree. But we felt it was more appropriate in the first instance to start with the theory of change that was managed by Penny’s team, including the new expertise that we have brought in to help to do that in preparation for the summit, which will continue with us afterwards.
Penny Innes: We want to incentivise action, and it might be that country offices choose to develop or adapt a central theory of change to their own circumstances. We need to recognise that there is not a lot of evidence. There are big gaps in quite a lot of the theory of change, and we did not want to halt countries taking action while they spent a lot of time analysing based on not much evidence. It is a balance, but we also want to make sure that countries are progressing in areas like looking at their programming and how they disaggregate their data and work, and engaging with disabled people’s organisations.
Q30 Chair: Alison, have you anything to say in response to that?
Dr Evans: Yes. We do not quite meet eye to eye on this one. They have a central theory of change now, but it is just not visible to most within the Department. That needs to be made very visible and very central. We feel strongly that we need to move beyond inspiring country offices to do this, to essentially making it really clear to them that this is absolutely a central priority for the Department.
DFID does not generally like us saying things should be mandatory. In this case, we have made it very clear that we thought the theory of change process should be mandatory, precisely because it pools together local evidence and an understanding of local context, and it gives space to engage directly with disabled people’s organisations at a country level to agree what the big barriers are and how DFID can make a difference.
We felt that was the best possible way of moving quickly on this agenda, but also building a common understanding between DFID, its key partners and DPOs in country. That is why we suggest it needs to be mandatory. The danger of an action plan at the country level is that it is all DFID saying what it will do; it is not DFID asking, “What are DPOs telling us about what the key potential for change is, and how can we contribute to that?”
Q31 Chair: Alison, DFID’s humanitarian reform policy states that people with disabilities are among the “most marginalised and vulnerable in times of crisis”. You criticise that characterisation in the report, arguing that it portrays people with disabilities as passive. Do you recognise the validity of DFID’s assertion, namely that people with disabilities are indeed the most marginalised?
Dr Evans: Yes, absolutely, but we felt that the way it was articulated in the humanitarian reform policy was perhaps more inclined to treat disabled people—as, I have to say, recipients of humanitarian support are at times—as passive beneficiaries, as opposed to active agents in their own journey. We felt that the language was not quite right there, and was not as forward looking as it should be. We do not disagree with the fact that the hardest to reach in humanitarian settings often include a high percentage of people with disability.
Q32 Chair: Richard, have you anything to add?
Richard Clarke: Only to say that the people with disabilities I heard from and in some cases met at the summit were very clear that they were active recipients and very active advocates on these kinds of issues. They would be, and indeed were, the first to say how important it is for humanitarian donors and support organisations, including national Governments, to think really seriously about how we provide support in a way that recognises their unique needs.
Q33 Stephen Twigg: ICAI found that the Department’s central disability team did not currently include any staff with prior disability inclusion expertise. You responded quite positively in addressing this issue. Can you update us on what you are doing to ensure there is that expertise within the team?
Richard Clarke: Sure. I will say a bit about that. We grew the team, as you know and as the report outlines, quite significantly in preparation for the summit. Some of that was always going to be a temporary uplift of people who had specific summit-related roles. The team is now at a settled level of 10 full-time equivalents, which I am clear is a suitable size. It is, if I may say so, a very strong team. They know their stuff.
I am delighted to say we have also actively recruited individuals who have detailed prior experience of working in precisely this kind of area. That is important for a couple of reasons, really. First, it is very important in terms of making sure that we are designing our work and delivering the framework in a way that is able to draw on that expertise. Also, those individuals will form the core of an operating model role for that team where they are assisting colleagues in DFID, in the UK and overseas, to think about their programming. This is both a delivery hub and a hub that can be used by the rest of the Department. This is an area where we have made some real progress.
Dr Evans: We are really pleased that there is change in this respect, and particularly look forward to coming back to this issue in follow-up, and seeing how well that increase in expertise has consolidated and is starting to have an impact. We see that there is a real urgency here. It is going to be very important to be able to move out of an increasingly strong central team to having real expertise across the Department. That is what we really want to see developing from now on, but credit is due to the Department for seeing this as important and beginning to make some change. We will come back to this in follow-up.
Q34 Chair: In an anonymous survey within DFID, 12% of UK-based staff and 4% of locally hired staff said they had a disability, yet the proportion of staff who had notified the Department was far lower. Does DFID have a cultural problem with disability?
Richard Clarke: As a Department, we are absolutely committed to ensuring that individuals from a full range of backgrounds can thrive, flourish and build a career in the Department, both in the UK and in our country offices overseas. We have made good progress in the last few years. We measure our declaration rate—people who are prepared to say whether they have a disability—in the summer each year, and we have seen a rise from 39% in June 2017 to 50% this year. That is absolutely not enough. The Permanent Secretary, my board member colleagues and I would want to see that as close to 100% as we can manage. It is a slightly different way of measuring it, but I think I am right in saying that the average across Government Departments is about 70% at the moment. There is definitely some more progress for us to make in terms of declaration.
We have also seen a rise in the number of staff who describe themselves as having a disability, from 6.4% to 9%. Again, we want to make more progress in that area. My sense is that DFID has a very good and inclusive culture. It is very important not to take that for granted. We have very active staff networks. We have had some really good recent blogs on our very well-used intranet where people have talked about their experience of working in DFID. We are working closely with colleagues in the Foreign Office who are responsible for our overseas estate to make sure that our buildings and the way in which people work overseas are accessible. We have a way to go, but we are making progress, and it is absolutely central to the Department’s vision.
Q35 Chair: In the baseline diversity and inclusion questionnaire conducted in 2017, some staff told DFID that they did not feel comfortable discussing their disability with their line manager. Does that concern you?
Richard Clarke: Absolutely. We are publishing a revised people plan this year. We have slightly changed our performance management approach recently, and it puts a much stronger focus on regular communication between line managers and the people who work for them. We will look to put that at the heart of what we do. That situation absolutely would not be appropriate.
Q36 Chair: In response to the review, you claim to have improved the confidence of your staff to tell you about disabilities, as you have just been talking about. How have you done that? What have you done specifically?
Penny Innes: One of the issues is about trust and raising awareness for people to talk about disability. We have had two staff events recently, one organised by the listening network, which was about men and mental health. We had some senior managers who were very open about the struggles they have had with their mental health. That was supported by the Permanent Secretary, and was a step forward in raising issues and getting them out in the open. The other event was organised by the disability and empowerment network shortly before the summit. It was called “Outside In”. It was about saying that, in the work we are doing in our policy and programming on disability, we should be looking at our own practices as well. That was also well attended; the Secretary of State and the Permanent Secretary came along. There is progress in showing leadership to talk about these issues.
Q37 Chair: In an internal DFID questionnaire, 41% of DFID staff thought DFID was doing enough on disability, but that fell to just 13% among staff who had disabilities themselves. What does that tell you? What have you learned from that?
Richard Clarke: It tells us that our staff, as I would expect, are very focused on making sure that what we do in this area we take seriously internally—that we put our money where our mouth is when it comes to making a reality of this. I would strongly hope that, in the coming months and years, that figure will improve as we roll out some of the things we have mentioned as part of the people plan. It is clearly an unsatisfactory number. But, as I say, there are a number of areas, particularly the increase in self-reporting, that we see as cause for encouragement.
Q38 Chair: Alison, do you have any views on what you have just heard?
Dr Evans: I might get Paul just to give you a couple of examples from the conversations we had with some DFID networks inside the Department. The listening network is an impressive network. One of the features of it that really works is that it is peer to peer. That has been incredibly helpful inside the Department. Where staff are reliant on their line manager, for example to help them with reasonable accommodations around assisted technologies or other things they may need to work, there is a reticence. That is the thing that needs to be looked at carefully about people self-reporting and asking for things.
I want to draw your attention to the locally hired staff figure in the data we presented there. As far as we know, the percentage of staff appointed in country who have a disability is somewhere between 1.4% and 4% depending on the methodology for getting that data. That is very, very low. DFID has a very large number of staff appointed in country in its country office network. There is clearly a job of work to do there to think about whether they are doing enough. Paul, can you say something about conversations we had inside DFID?
Paul Spray: There were a number of conversations where we cannot be specific, because they were in confidence, but where people expressed concern about being able to talk to their line manager, in terms of what it might do for their image, their chances of promotion and so forth.
Q39 Richard Burden: I think I am right in thinking that DFID has only two research staff working on disability, with each of them spending about 10% of their time on disability issues. When we are looking at programming, ICAI reviewed 10 delivery programmes, but it seemed only two of those included any research elements or designated learning elements. Is that really sufficient?
Penny Innes: There are probably more people than that actually doing work on research related to disability inclusion. For example, we have a mental health research programme called PRIME, and there are some people supporting that. There is also the new research element of the Disability Inclusive Development Programme, which is £7 million-worth specifically on research. That does not sound enough, and it does not sound quite right compared to the different activities that are going on. We are also funding some research on putting the Washington Group questions—a good set of questions that help us find out the prevalence of disability—into practice in humanitarian situations. There are people leading on those areas as well. There is more we can do and more research.
In advance of the summit, we commissioned an evidence mapping exercise. That looked at different interventions and outcomes, and it assessed the extent of evidence out there. That showed what we already knew, really: there is a lack of evidence. The evidence that was found was, on the whole, at intervention level, addressing specific impairments and supporting people, rather than looking at institutional barriers that people with disabilities face. We need to move into addressing those kinds of evidence gaps, and that is what our new research part of the Disability Inclusive Development Programme will do.
Q40 Richard Burden: From ICAI’s point of view, do you reckon that DFID is staffed up enough on the research side?
Dr Evans: We are very pleased to see the new DID—Disability Inclusive Development—research programme of around £30 million over six years. It is a really good initiative. But we are somewhat bemused that there is not more at this point, given that this is an agenda that the Department has obviously been engaged in for a very long time and really started to focus on from 2014 or 2015. Paul can probably give some specifics on this.
Paul Spray: The crucial point is the mainstreaming of disability into research programmes in general. Mainstreaming is clearly something that DFID wants to do, so it has a wide range of research programmes, and it is the mainstreaming in that will make a real difference. They did have a very good programme—it was evaluated well—that finished in 2014, which was a cross-cutting disability research programme that did exactly that: it took forward DFID’s existing research programmes and asked them to add on bits and pieces about disability. Obviously, it is not good enough to add on some things once you have started, but, if disability is to be mainstreamed into research, research staff in general need to understand about disability.
Q41 Richard Burden: The last area of questions from me is on evaluation. DFID plans to evaluate the new Disability Inclusive Development Programme using randomised controlled trials. As far as ICAI is concerned, you questioned the appropriateness of that. I accept that RCTs have their faults, but are they not at least more rigorous than other forms of evaluation?
Dr Evans: They absolutely have a place. We are very clear to acknowledge that they have a place. It is certainly our expectation that there would be a broader suite of evaluation tools, which would also take on board more qualitative dimensions of how and why things work as opposed to just, “Do they work?” We are looking to see DFID moving into that space.
We have had some interesting conversations within the community of practice on a more global basis around whether RCTs have particular ethical issues for people with disabilities. One would expect DFID to be able to navigate those very carefully. We do not want to see an overreliance on a single methodology that might be to the exclusion of some others that would get more quickly to the answer of how and why things work. Our objection was twofold: not to reify RCTs as the only way forward, but also to make sure you are considering the ethical dimensions in a very transparent and open way.
Q42 Richard Burden: Could you perhaps say a little more about what you would like to see, particularly on the qualitative evaluations? What do you think DFID should actually be doing there?
Dr Evans: We talked quite a lot about the evidence gaps, the missing evidence and how there is not a strong body of work that says what works in this area. That is correct. We would also want to see not just what has produced the outcomes, but how they have been produced, and then how they can be transferred and replicated elsewhere. RCTs, on their own, can limit your ability to do that.
Qualitative evaluation or mixed-method evaluation allows you to ask why something has worked as well as how, and then to transfer the learning more quickly elsewhere. As I said, it is a mixed toolbag that we would be looking at. DFID puts a lot of stock in its RCT evidence, and some of it is incredibly powerful. But, in this case, we feel that there is also a strong argument for understanding the mechanics of how to do programming well in areas where we simply do not have good evidence at this point.
Q43 Richard Burden: Do you have any comments in response?
Penny Innes: I agree with everything Alison said, really. You need a range of different things. We will be doing some RCTs, but we will have to do them very carefully.
Richard Clarke: We definitely do not see RCTs as a panacea in this area, but I can absolutely assure you that the way in which we are thinking about applying them is entirely consistent with doing it in an ethical way.
Chair: That brings an end to our questions. Thank you very much. This is a very welcome report into something that has been a priority for the Department and the last two Secretaries of State. Thank you very much for the interesting discussion.