International Development Committee

Oral evidence: Pre-Appointment Hearing with Candidate for Chief Commissioner of ICAI, HC 685
Tuesday 9 December 2014

Ordered by the House of Commons to be published on 9 December 2014.

Written evidence from witnesses:

–       Independent Commission for Aid Impact

Members present: Rt Hon Sir Malcolm Bruce (Chair); Fiona Bruce; Pauline Latham; Jeremy Lefroy; Mr Michael McCann; Fiona O’Donnell; Chris White

Questions 1-43

Witnesses: Dr Alison Evans, the Government’s proposed candidate for Chief Commissioner of the Independent Commission for Aid Impact, gave evidence 

Q1   Chair: Welcome to our Committee, Dr Evans.  As you know, this is a confirmation hearing by the Committee for the Secretary of State’s appointment to the job of Chief Commissioner for the Independent Commission on Aid Impact.  We are going to take about an hour, if that is all right, because we also have to agree afterwards what we are going to do.  The basic question is: why did you apply for the job?  Why do you want it?

Dr Evans: If there is one feature of my entire career in international development, it has been a commitment to development effectiveness.  There have been many episodes when I have worked explicitly on trying to understand the context in which aid contributes to development and when and why it works, and to be involved in a body dedicated to trying to achieve high standards in development effectiveness is a fantastic opportunity. 

The UK is a very prominent player in international development.  Therefore, ICAI is a unique opportunity to do that in a context where aid and development are changing.  To have an independent scrutiny body like ICAI not only directly scrutinising the way the UK Government use aid—what works and so forth—but also being able to report to a Committee of this nature is a fabulous notion.  It is a unique opportunity. 

Within the aid architecture, ICAI has a unique place.  There are not many bodies quite like it.  Therefore, both the commitment to development effectiveness and the fact it is a unique body definitely attract me to the role.

Q2   Chair: It is a part-time job; we are talking about five and a half days a month, maybe a little more.  If it is five and a half days a month, that is about £36,000 a year, give or take if there are extra days.  I presume you do not see this as your only and full-time job.  Would you be running other jobs alongside it, and would this be your top priority?  How do you ensure that it is?

Dr Evans: It certainly would be the top priority, but it could not be my only job; I need to be able to earn a living in other ways too.  I would need to be very careful about the kinds of other consultancy roles and so forth I took on so that they did not conflict in any way with this particular role—in terms of not only substance, but time.  But this would be absolutely my principal concern.  For any Chief Commissioner, this job would be very demanding.  As a part-time role it is very demanding, but I understood that when I applied for it.

Q3   Chair: Had it been a full-time job with a full-time salary, would that have been more attractive to you, or would it have closed down other opportunities?

Dr Evans: I would still have been very interested at that time.

Q4   Mr McCann: Dr Evans, you still have positions with Oxford Policy Management and BBC Media Action, and up until last year you were involved with the ODI.  They all take or bid for funding from DFID.  What measures will you take to ensure you can address conflicts of interest that may arise?

Dr Evans: Managing conflicts of interest is of paramount importance, and I take it extremely seriously.  My general understanding is that the way to do this is to have a very clear framework of rules about identifying potential areas of conflict and good safeguarding measures in place to be able to handle them on a case-by-case basis.  That should apply certainly to everyone involved in ICAI.  I would be looking to ensure that a very robust framework was in place to manage all conflicts of interest, including my own.

As to my particular areas of private interest, I hope I have been very open about what they are.  In relation to Oxford Policy Management, I am a non‑executive director and have been on the board for two years.  This is a private limited company and it does bid for contracts from DFID.  Based on discussions I have had with the board of OPM, we have decided that, if I was to be confirmed in this role, I would step down from my function as a non-executive director.  That is appropriate, not least because of perception. 

When I applied for this role I had no particular worry that I could not firewall my engagement in OPM very carefully.  I do not do any work for OPM; my role is purely in governance, but I feel that perception matters greatly in the treatment of conflict of interest.  Therefore, I have agreed with the board chair that, if I was confirmed in this position, I would step down from my role as non-executive director, not least because as a private limited company it pays me a small amount to be on the board.  Therefore, I think that would not be appropriate in the circumstances.

As to the other two interests you mention, I have now been out of the ODI for two and a half years; it will be three years by May.  I have no material engagement, other than being asked to sit on one or two advisory panels now and then.  I have one live advisory group on which I sit.  I would be happy to let go of that if I were to be confirmed in this post.  They are very modest roles.  I have no involvement other than chairing the occasional meeting.

As far as BBC Media Action is concerned, this is the international charitable development arm of the BBC.  It has a programme partly funded by an accountable grant from DFID.  I am a trustee and vice-chair of that board.  I have a specific responsibility on that board for research and learning.  I am not involved in direct oversight of the global grant as they call it, which is the DFID grant.  My engagement is almost exclusively of a governance nature; and as a public good entity my role as a trustee is to ensure that Media Action is contributing to the public benefit. 

I feel it is very possible for me to manage any potential conflict of interest.  It works in a very particular sector on very particular issues.  I can certainly firewall my engagement on that board.  For example, I would recuse myself from any review in which ICAI was involved that had anything to do with that particular sector.  However, I would be open to a discussion, if this Committee felt it was of concern, as to whether I could continue that role.  I would be open to discussion on any of those interests.

Q5   Mr McCann: That is a very thorough answer.  You have already answered a supplementary in terms of maintaining clear firewalls.  As Members of Parliament, even those who have been in this place only since 2010, we bear scars on our backs from conflicts of interest and perceived conflicts of interest.  We have to think very carefully in our daily business about how we deal with those matters. 

Let me give you a hard example.  IDC was looking at the programme we run in Burma and we were there on a formal visit, and we saw the work being carried out by BBC Media Action.  If you were doing an investigation into elements of that programme that brought you into direct contact with BBC Media Action, how would you firewall that to protect yourself from any conflict of interest, or perceived conflict of interest?

Dr Evans: The crucial point is that once I knew we were engaging in a programme of that nature, as Chief Commissioner I would step out of any engagement in that particular review.  It would then be the role of other commissioners to champion that particular review and ensure adequate measures were in place to assure the quality and veracity of that review.  I would need to ensure as Chief Commissioner that I did not have any involvement in the communication of the recommendations or findings of that review.

That would be a very limited case as far as BBC Media Action is concerned.  The reason I am stating that if I was confirmed I would step down from my board role with OPM is that there might be more instances where that would need to be the case, which would make my position as Chief Commissioner somewhat untenable.  I think that needs to be addressed.

However, for my part all these engagements are purely at the level of governance.  I am not involved in the execution or delivery of any of these programmes on behalf of any of these organisations, so the conflict of interest is simply that I have some commitment to those organisations but am not actively involved in any of the programmes.

Q6   Mr McCann: The obvious comment arising from that is that, if you had to recuse yourself from more of the job, you would have difficulty doing the work, but I appreciate the point you make.  It is a small part of it.  Dr Evans, do you plan to continue with independent consultancy, or return to it, after ICAI?

Dr Evans: I have to do some because I need to be able to supplement my income from the role of Chief Commissioner, if I was to be confirmed.  For example, I do a lot of work outside the UK.  For example, I work with the Danish Government; I am currently working with the German Government; and I work quite a lot with the World Bank.  Therefore, there are opportunities for me to engage in consultancies that are some distance away from the work I might be doing in ICAI, and I would want to do a limited number of those to supplement the income I would get from taking up this role.

Q7   Mr McCann: In your answers you have covered thoroughly the issues I was going to raise.  Do you think there should be any future restrictions after your time with ICAI is over in terms of what work you could carry out, or do you think that is unnecessary?

Dr Evans: It is an interesting point.  When you have been part of organisations, many would say they have a limit on the amount of time you can spend being contracted to do such work.  When I left the World Bank there was a limit on the amount of time I could spend being rehired as a consultant, if you like.  I think that is quite good practice, but in this case as ICAI commissioner it would be unseemly to depart from the role and then somehow to be on the books of DFID doing work for that Department.  There could be a grace period when it would be regarded as not appropriate to be employed by the organisation you were primarily scrutinising for the period while you were Chief Commissioner.

Q8   Fiona O’Donnell: Following up Michael McCann’s question, it is not just about future employment with DFID but with other organisations.  If you are pursuing employment towards the end of your term of office, that could be seen to be influencing the way you are doing your job.  Do you think there should be a restriction in terms of other employment you can take—for, say, a period of six months after your term of office comes to an end?

Dr Evans: The purpose of ICAI is fundamentally to scrutinise the way in which the UK Government are allocating, deploying, evaluating and learning from its aid programme, and in that sense to hold Government to account through this Committee for the way it is deploying its aid programme.  Its primary purpose is not directly to hold partners of UK aid to account.  How far would you stretch that?  There is a degree of recognition that, DFID being the major focus of ICAI’s scrutiny, it would be inappropriate to take up consultancy or employment for a period, but how far outside of that is it?  Is it any other bilateral agency that has had anything to do with the UK?  Is it any contractor that has had anything to do with UK aid?  Potentially, that would be: how long is a piece of string? 

For someone like me, who has been in the sector for a very long time, it is hard not to have quite a lot of knowledge of, dealings with and experience of organisations in the sector, so the red line would be taking up any kind of employment or consultancy with the Department, or related Departments.  Beyond that, it is a little harder to be able to know where to stop.

Q9   Fiona O’Donnell: It is often difficult to know where perception ends, as the Home Secretary recently experienced in appointing people to lead inquiries.  You answered Michael very fully on the links between OPM, BBC Action Group, ODI and DFID.  Since April 2013, have you been, or are you currently, involved as an independent consultant with any organisation that takes or bids for funds from DFID beyond these three?

Dr Evans: Yes, but they do not bid for funds.  In the last two months I have agreed to join the board of Social Finance, an organisation that works predominantly in the UK.  Some of you will know of it as being involved in the development of a concept called the social impact bond.  That was involved in a very interesting pilot in Peterborough around trying to crowd in different sources of funding to solve tough social problems, in that case prison recidivism. 

Social Finance is an organisation with international aspirations.  It would like to grow some of its work internationally through the development of ideas, such as the development impact bond.  It is a non-profit organisation.  I agreed in the last few months to sit on its board as a trustee, but it does not have direct funding from DFID.  It is engaged in a pilot with DFID on devising a development impact bond, which precedes my time on the board.  I would be very open to a conversation, if this Committee felt it important, to address whether that is an interest I can maintain.  It seems to me that all these things are very manageable.  As to OPM, after very close conversations with the board, that was the one from which I felt it most appropriate to step away.

Q10   Fiona O’Donnell: In terms of your assessing and making that judgment, what are the criteria you would apply to decide when a conflict of interest has arisen, or that line has been crossed?

Dr Evans: Materiality is probably the biggest one; that is where there are strong material overlaps between the organisation and DFID, both financial and to some extent reputational.  I feel that I apply that on an everyday basis in the work I do, where you constantly want to manage potential conflicts of interest, because I sit in a number of situations where that is very important.

Q11   Fiona O’Donnell: In 2011 in a ConservativeHome article, you wrote: “The knowledge and passion of the [then] Secretary of State”—the right hon. Andrew Mitchell MP—“to try to ensure an effective and value-for-money UK development enterprise is not in doubt, even amongst the experts.”  Would you assign the same description to the current Secretary of State?

Dr Evans: This Government have shown enormous commitment to the effectiveness of UK aid.

Fiona O’Donnell: But the current Secretary of State?

Dr Evans: The current Secretary of State is very much part of the same culture that has been established around the Department in particular.  I would see this Secretary of State as being a continuation of that.  It is certainly the case that Andrew Mitchell, when Secretary of State, put in place a number of measures that took UK aid into a different place in terms of attention to effectiveness and value for money.

Q12   Fiona O’Donnell: In addition to ConservativeHome, have you written for websites hosted by other political parties?

Dr Evans: No.

Fiona O’Donnell: Only ConservativeHome.

Dr Evans: I did not write for them, as I understand it.  I was interviewed when I was making a presentation at a party conference.  I was not commissioned to write it; I was interviewed.

Q13   Fiona O’Donnell: You are clearly well known to DFID, which appointed you to review its multilateral aid review and so must approve of you.  Were you not seen as a favoured insider?

Dr Evans: I very much do not see myself as an insider.  All the contexts in which I have engaged with DFID, particularly in the past 10 years, have been very much in my capacity as an independent and somebody who is very plain speaking, and is considered as somebody who speaks their mind.  That is something it has valued, and it was very much part of my role as an external reviewer of the multilateral aid review.

My contact with DFID is very minimal; it is perhaps perceived to be greater than it is.  I have very limited contact, other than through these very specific channels.  I am not somebody who has developed close working relationships with senior DFID officials, but it is certainly the case that I have been around quite a long time and therefore am fairly well known.  I would very much challenge the view and perception that I am in any sense somebody who is regarded as being either very close to DFID—or, even worse, an insider.

Q14   Fiona O’Donnell: Going back to the piece that appears in ConservativeHome, have you attended other party political conferences?

Dr Evans: Yes.  In my role a director of the ODI, I attended Labour, Liberal Democrat and Conservative conferences.

Q15   Chris White: How do you see this Committee’s role in ICAI’s work?

Dr Evans: This Committee is the central piece in the jigsaw.  ICAI is reporting to you as a Committee; it is seeking to inform you of the findings and recommendations of reviews; and I hope also to build a relationship where ICAI can be of value to your work as a Committee in holding the Department to account.  I see that it has both a formal and potentially informal element.

Q16   Chris White: How do you envisage yourself working with this Committee?

Dr Evans: In both forms.  In a formal sense, we would be looking to provide you with reports that are credible, accessible and of value to you in being able to scrutinise the Department and other Departments that handle UK ODA.  I would like to see, wherever there are opportunities, us discussing what should be in the ICAI work programme; I would like to think there are opportunities for more informal engagement around particular topics that ICAI is involved in reviewing, or would like to review, or information that has come up in the process of ICAI’s work. 

You as a Committee have a programme of inquiries.  It would seem to me to be a very important part of the role to look at the extent to which ICAI can be dovetailing some of its work to support those inquiries, or to do specific reviews that are a pillar or plank of one of those inquiries.  I would see myself as Chief Commissioner as wanting to lead on how that relationship builds and to deploy other commissioners to be able to do some of the detailed work.

Q17   Chris White: Do you have any ambitions for the relationship between ICAI and this Committee?  Do you see it changing?

Dr Evans: It sounds to me from what I understand that this relationship has been building over four years, and is now in a stronger place than perhaps it was at the beginning, which is no surprise given this was new to everybody.  I would very much want to build on the spirit of what I gather has been established.

If there is one area open for discussion that I would like to explore, it is the ability to dovetail in an appropriate and useful way ICAI’s work programme with that of the Committee to ensure there is a sense of building and sequencing as much as possible, without losing the need for ICAI to pursue its own agenda and without the Committee feeling it needs to compromise around the work of ICAI.  It has to be a relationship of trust and accountability, and one in which you feel you can challenge ICAI too on the work it is doing.

Q18   Chris White: Would you be happy to agree ICAI’s work plan with the Committee?

Dr Evans: Yes; I think that is entirely appropriate.

Q19   Pauline Latham: How would you distinguish the work of ICAI from DFID’s own evaluation programme and that of the NAO?

Dr Evans: This is a very important point.  Looking constantly to product-differentiate, if you like, is one of the key challenges for ICAI.  The National Audit Office has a primary concern with value for money.  It does fewer reports but it does them often quite in depth, and it is reporting function is to the Public Accounts Committee.  It certainly does not have anything of the volume of ICAI, but it is largely an assurance-based approach.  It is important that ICAI seeks to avoid falling into a similar purely assurance-based approach.  There are times when that is appropriate.  I am thinking of the merits of the TMSA investigation that was handled by ICAI last year in terms of taking the slightly more assurance-based approach, but it seems to me that ICAI’s value lies more in addressing some of the strategic developmental questions that the Department is tackling, not purely narrow value-for-money audits.  That is a key differentiation.

As to DFID’s own evaluations, which are very well funded, significant and built into programming, the role of ICAI is by no means to try to replicate that work.  It cannot.  Its budget is miniscule by comparison, and it does not really have space to do what you would call full-blown evaluation.  The key is that ICAI can utilise and draw on that evaluation base that DFID now has to raise questions about the nature of the findings of those evaluations and their veracity; how DFID is learning from them; how much that programme of evaluations is contributing to better effectiveness within DFID; but also to draw out some of the primary data in that evaluation base as raw material for ICAI reviews.  There is a huge opportunity to do that.  That evaluation programme needs time to come to fruition, but that is a way in which ICAI can build on and differentiate itself from that body of work.

Q20   Pauline Latham: You are happy that there would be no overlap.

Dr Evans: We would have to guard very carefully against that to ensure ICAI was not trying to do the job of evaluation, but equally that DFID evaluations were not trying to second guess what ICAI was looking to scrutinise.  We would need to be very attentive to that.

Q21   Fiona Bruce: On value for money, in the debates both in this House and among the public, Michael Moore’s Bill is an issue that people are very concerned about.  Many people disagree with the 0.7% full stop; others say there is huge waste in the way we direct our budgets. Are you saying that in terms of your reports value for money would not be the priority?  I believe the public would like to see it as a priority.  How would you prioritise value for money?  How would you probe it in the ICAI reports to ensure it was a priority?

Dr Evans: Value for money is central, but there is the way in which it is interpreted in the context of an NAO audit and the way in which it is seen in the context of an ICAI review, which is within the full range of engagement that DFID might have in a particular sector, country and engagement.  Value for money is the central piece, but it is important not to see this purely through the lens of following the money but all the instruments DFID and Departments deploy to try to address any particular development issue. 

It is also about putting value for money in the context of the other things that are also valued, including learning and the ability to learn from mistakes as well as success.  There are a number of features but, as to the central role of ICAI, I have no difference of view about its central function in scrutinising value for money for UK aid.  It is a matter of how that is done: whether it is done very narrowly or within the context of the broader framework of understanding what we are trying to achieve in development.  It is that to which ICAI could contribute very helpfully.

Q22   Jeremy Lefroy: Could you describe how you would see the relationship between ICAI and DFID?

Dr Evans: Fundamentally, it is a relationship born out of challenge in a sense.  ICAI needs to be seen to be a credible but challenging partner for DFID.  The phrase I often use is: accountability without learning is not entirely meaningful; and learning without accountability is pretty toothless.  You need to handle that accountability and learning function very carefully together.  One of the good developments I have seen in ICAI over the past couple of years is that there is more attention to the accountability and learning piece.  The relationship with DFID has to be about those two things. 

Through this Committee, it has to be about holding DFID to account, and then requiring the appropriate response from management about how it will deal with findings, but it also has to be an opportunity for these reviews to contribute to productive learning, which means the relationship has to be one of constructive challenge.  It is about building a climate in which constructive challenge can take place and DFID sees ICAI as credible and, therefore, to be listened to, and in which ICAI is absolutely holding DFID to account for the things it said it would do.

Q23   Jeremy Lefroy: We have heard talk about that in evidence given to this Committee before by Richard Calvert, saying that DFID wants to work in a way that allows it to learn as much as possible from ICAI.  Perhaps there was an implication from that and other comments that an informal relationship that allowed that to happen might be suitable.  Would you regard that as appropriate?  If not, how would you make sure that both the accountability and learning were conducted in a relatively formal context?

Dr Evans: It is important that this be kept formal.  One of the ways in which one does that is by ensuring that attention to learning is a very big piece of ICAI reviews.  My experience in evaluation, which goes back a very long way, is that there is always a tendency to treat learning as an afterthought and then somehow wonder why it is never picked up.  It is important that ICAI reviews, in going about the work of scrutinising aid, also attend to the lessons learnt from the very beginning quite literally—that is one way in a formal context where DFID is going to be exposed to that—and then to do formal set pieces where we can communicate ICAI reviews to DFID staff and management in such a way that they are encouraged to take on board the substance of the review, and for us to be able to identify for DFID any key issues of learning that might have come out of ICAI reviews.

I would be uncomfortable with the idea that there should be a more granular informal relationship, because the perception could start heading in the wrong direction, and it would be better to keep this as formal but as open and constructive as possible.

Q24   Jeremy Lefroy: You referred earlier to the need to do both very detailed examination and more general examination of the effectiveness of aid impact, which is absolutely right, but how do you think you will be able to do this?  Clearly, the public and we want to see very detailed examination to ensure that fraud, incompetence or ineffectiveness is rooted out, or exposed if it is there. 

On the other hand, we appreciate that sometimes there are much bigger issues at stake and you have various Departments working together to help a country effect very substantial change.  There is always the risk that the latter will be used as a smokescreen or cover for getting away with ineffectiveness in aid, because, as you say, where a nation is in conflict you have to spend lots of money doing something because of the special circumstances.  How do you think you will be able, as a relatively small team with a small budget compared with DFID’s, to do both, which are very important, and to be pretty definite rather than vague in your conclusions, as sometimes we have seen?

Dr Evans: There is a lot in there.  I completely agree that it is important not to take the eye off particularly the deep operational issues that could expose some form of inefficiency or ineffectiveness that raises bigger questions about the way in which UK aid is being used.  It is really important to have within a portfolio of reviews a number of those deep dives.  The idea then would be to blend those on a portfolio basis with reviews that move slightly higher up the value chain to bigger strategic relevance but help to flesh out the bigger picture.  Often, we are dealing with big issues of materiality because of very big allocations of funds, so getting a balance between those deep dives and the bigger picture reviews helps to get to grips with the way in which DFID is making decisions about big chunks of its budget, where it is headed and who it is working with.

One way to build the learning is to treat the work programme on a rolling basis and see reviews building with one another under certain themes.  You see the work programme building over years rather than everything being purely on an annual basis.  You can then stretch the value added of individual reviews because they connect to a bigger theme.  Then you have space in your portfolio for these deep dives on particular issues where you are looking to hold feet to the fire, whether it is on control and risk, the handling of fraud issues or whatever.  You would want the blend. 

There is value in trying to think about the programme on a much more rolling basis so you build learning across a theme such as how DFID is approaching girls’ education in different contexts and risk environments. You build that over time, so you do not see everything as having to happen in one year.  Then you extract the learning over a period of time, together with the more operationally focused reviews.  That is the only way you could stretch the work programme to do more than one thing because, as you say, it is a very circumscribed budget.

Q25   Jeremy Lefroy: Looking at your CV, your experience is very much on the broad policy issues, as far as I can see.  What experience do you have of doing really deep-dive investigations of aid programmes or development programmes from the perspective of value for money, or investigating whether fraud is going on?

Dr Evans: It is a fair point that my experience has been very much at the policy and strategy level, but when I was working in independent evaluation at the World Bank one of the classes of products involved was project audits, in which we had to do deep-dive analysis of bank projects.  They were much more akin to a classic NAO-style audit than perhaps a more developmentally oriented audit of the other kinds of products that IEG now produces.  I would also be looking to ensure that the team of commissioners had real skills in that area too so we were able to complement one another.

It is important that all those bases are covered in the selection of the whole group of commissioners.  If I were to be confirmed as Chief Commissioner, I have experience but I know my limits, and I would be wanting to make sure that my commissioner team also had strong, close audit-style experience to bring to the table.

Q26   Mr McCann: Rumour has it that ICAI is shortly to be evicted from the Cabinet Office and moved to DFID in Whitehall.  Given that potential, how would you stop or temper ingrates like me standing up at DFID questions and saying, “How can the Secretary of State claim ICAI is independent when it is in your back pocket in your office in Whitehall?”

Dr Evans: In all honesty, I think it is a rather bad idea.  I am hoping that that might be just a temporary measure and some other physical location for ICAI can be found.  It is not all about geography and physical location.  Independence comes from many sources.  There are many ways in which ICAI preserves its independence, not least through its staffing and the way it goes about its methodology, and there is behavioural independence too.  All that is terribly important, so physical location does not trump everything.  However, perception really rules the day.  My strong preference—I have already indicated this in the process—would be that ICAI does not have a permanent home inside 22 Whitehall.

Q27   Chris White: Do you think that 65 days a year is adequate for this role?

Dr Evans: It is very tight, and I did look at it very hard.  I confess I was quite surprised it was only 65 days, and 65 days for the commissioners is tight too.  I understand very well the desire to keep the whole body on a short leash, to manage budget very closely and tightly and to some extent to get a high-performing commissioner team without necessarily having a standing and potentially stodgy full-time team. 

However, we should look very hard at whether that time commitment is adequate to the level of expectation that exists around ICAI. I would very much like to have a conversation, if I were to be confirmed in the role, about the 65 days.  Whether or not it should a full-time role is slightly different, but there is a bit of an issue.  One can do it in that time; one would absolutely throw everything at it, but if you want to get the best and highest possible standard of scrutiny it is asking a huge amount of the commissioner team to do it on that kind of budget.

Q28   Chris White: How many days do you think would be needed to get the best quality?

Dr Evans: Have I thought about it?  Yes, a little bit.  I would say it is closer to 90 days.

Q29   Chris White: Thank you for being so frank.  When would you envisage starting work to in a shadow capacity?

Dr Evans: As I am not confirmed, I have only a superficial understanding at this moment, but I understand there is an attempt to manage the transition as carefully as possible, and that by March there would be a team in place.  Then it would be a question of working in such a way that we could execute the best possible handover with maximum attention to continuity while not getting in the way of the incumbent team, because it has work to do through May-June.  It will require skill on all our parts to make sure that happens, but I would hope that by March the shadow team was all in place, and, as I understand it, the timetable is on for that at the moment.

Q30   Chris White: In its annual report ICAI said it was concerned that the end of the commissioner and contractor contracts in May 2015 presented a significant continuity risk.  How do you propose to manage that risk and ensure continuity of reporting?

Dr Evans: When you have pretty much everything turning over it certainly raises challenges.  The good news is that the secretariat is the constant.  That will be absolutely essential in the handover and management of continuity—more so, now that they are going to have added capacity to handle some of the management pieces previously contracted out.  That will improve their positioning in that continuity. 

A lot of the handover work will be making sure we have a work programme that is ready to go as close as possible to the end of the incumbent team’s work programme; it will be about making sure we continue very much with the spirit of everything ICAI has done so far and try to evolve it rather than suddenly change it. It will be about making sure that we have done our homework on all the required contacts within DFID and this Committee and that we know the schedules and are able to slot straight in. 

There is a lot of procedural stuff we can do.  It will then be very much about keeping the spirit that has been established in ICAI going with the new team fully briefed on what that should look like.  Other than that, all we can do is make sure that communication lines are open and we are working to the best of our abilities to pick up where this team will be leaving off.

Q31   Chair: On Friday, Michael Moore’s Bill completed its passage in the Commons and it now goes to the Lords.  In the process of doing that the 0.7% went through but the proposal for an independent international development office was dropped.  That was part of a deal, partly, presumably, because the argument was, “Well, we have ICAI”, and we had discussions about that.  What do you think of the basics of Michael Moore’s Bill?

Dr Evans: This has been a long time coming.  In my time as director of ODI I argued against the enshrinement of 0.7%, so it is very interesting to see it now in place.  I felt that at the time it was not the best place to apply political capital, but it is happening and that is the will of Parliament.

As I understand it, the reference in the Bill to independent evaluation and the need for the Secretary of State to put in place a programme of strong independent evaluation is very much in line with what is taking place.  It seems to me to add even more weight to the importance of ICAI.  If this is going to be a permanent feature of the UK expenditure scene, ICAI’s role is even more important in scrutinising how well those resources are utilised on behalf of the public good.

Q32   Chair: Which I think is why that clause was in the Bill in the first place, because if you are going to spend it, you need good value for money.  Your background, as you have said, is mostly in policy, and that is specifically the area from which ICAI is excluded.  Andrew Mitchell said on a number of occasions that he was looking at the model a bit more like the PAC and National Audit Office.  This Committee would like that, because the National Audit Office is a creature of Parliament, not a Ministry, which is part of our frustration.  But it does not do policy; it is looking at retrospective spending.  ICAI is a little better than that; it is concerned with retrospective and current spending.  Do you agree that that is the right approach, and that ICAI should not be commenting or reporting on policy?  Given your background, how would you restrain yourself?

Dr Evans: Indeed.  It is understood to be part of the role, but I am not entirely sure I fully agree, to be honest.  There are some areas of policy and forward strategy on which a body like ICAI, with its unique position, could offer quite interesting challenge and insight.  The future directions of a Department that has such an enormous budget, and responsibility to spend it well, are as much the concern of the public as its current and past directions.

Q33   Chair: You would not be surprised to hear that this Committee thinks that is partly our province.

Dr Evans: Certainly, but to what extent can ICAI be of use to you in that respect as an organisation that is looking to inform this Committee, hopefully to feed in recommendations that are of value?  To what extent could it be helpful to you in some of those more forward-looking agendas?  However, ICAI has a lot on its plate anyway, so it would need to be highly selective and specific to need.

Q34   Chair: That raises a couple of issues.  Because ICAI has been focused I suspect that explains how it exposed the problem in TradeMark Southern Africa, because that is absolutely its kind of function.  How would you ensure that kind of rigour was retained?  Indeed, as the previous commissioner was an audit professional and you are not, how are you going to ensure that that professionalism is maintained under your watch?

Dr Evans: This has to be a standing feature of the ICAI work programme; it is non-negotiable by any Chief Commissioner to question the critical importance of having in one cycle of any programme these kinds of deep-dive, audit-style reviews.  I feel very strongly that that space has to exist.  My understanding was that the triennial review of ICAI was clear that a balance between that and some of the more thematic reviews was possibly the place where the work programme could deliver most value.

Q35   Chair: You are probably aware that some ICAI commissioners have been a bit concerned that the Department is trying to compromise its independence in some ways and there is a desire to be more collaborative. The reports are published simultaneously to the Secretary of State and to me as Chair of the Committee, but there have been indications in the past that DFID would quite like to see them first and have some influence over them.  Indeed, the Secretary of State has indicated that she sees a more collaborative approach.  How is that compatible with maintaining independence and operating in the sort of way, even if it is not the way it is structured, that the PAC and NAO operate?

Dr Evans: “Collaboration” is not a word I would use.

Chair: You can raise that with the Secretary of State.

Dr Evans: I see a relationship based on constructive challenge, and that challenge will come in a number of forms, but we all agree that without the learning that goes along with that to some extent we are under-powering the value of ICAI.  Therefore, regarding the straightforward question of whether reports should go equally to the Secretary of State and the Committee at the same time, that seems to me entirely appropriate.  ICAI reports to you.

Should there be a climate in which DFID has an involvement in the kinds of issues that are taken up by ICAI?  It is one of the stakeholders and it absolutely should.  Should it then be able to dictate in any sense how ICAI reviews come out on any issue?  Absolutely not.  It would be entirely inappropriate to have a huge amount of informal collaboration, but we need to consider carefully how to get this learning across, because you want the Department to be doing the learning.  As to whether or not after an ICAI review it would be appropriate for a commissioner or commissioners to have a public event inside DFID to communicate the findings and to have a debate after publication, maybe that is not such a bad idea, if it allows for some take-up of what ICAI is producing.

Chair: I do not think we have a problem with that.  The Committee takes evidence, as it is doing tomorrow, from ICAI on its reports.  That is an interesting point.

Q36   Fiona Bruce: What do you see as ICAI’s greatest successes?

Dr Evans: One of its great successes is being able to produce a portfolio of reports that is highly accessible, readable, is built around consistency of method, is of consistently high quality and in such large volume in such a short period of time.  ICAI had a very tough act in trying to establish itself at very short order and position itself as a very different kind of scrutiny body, and it did enormously well. 

My feeling is that it has got into its stride in the past 18 months to two years, because some of the more thematic reports are beginning to address bigger picture issues and there is also the success of a few of the very deep-dive reports, which have been very fertile.  However, accessibility and consistency of the reports themselves is a huge success.  I have some areas where perhaps on method we could strengthen it, but nonetheless there is a consistency there.

Q37   Fiona Bruce: That is very interesting.  My next question is: what would you see as ICAI’s three priority areas for improvement—not in terms of the procedures you talked to my colleague Chris White about, but in terms of the way it tackles reports and what it wants to achieve through them?  What would be your top three priorities for ICAI continuing this journey of improvement, which we all agree we want?

Dr Evans: The phrase I have in mind is that it is crucial for ICAI to be able to address the question: is the UK doing the right things by its aid programme, and is it doing them right?  Is it making the right strategic and operational choices, and is it then executing according to value for money?  I see ICAI needing to be even more assertive about the need to focus on those two elements and being very clear that those are two big drivers behind its work programme, accountability and learning functions.

There is a case for addressing the issue of how evidence is gathered in the context of its own methods and how it addresses the question of how DFID uses evidence.  It seems to me there could be a more muscular approach to address DFID as an evidence-based and learning organisation, but also evidence in the context of its own methodology.  It needs to be absolutely credible on its own methods.  The methodology has gone a long way and it has been improved, but there are still some question marks about how evidence is gathered, utilised and how credible it is.  I would see that as another priority.

The third priority for me would be working with what I call the interfaces between ICAI and its key stakeholders much more effectively.  We have talked a bit about perhaps increasing the frequency of informal interactions with this Committee, if it so desires and it is regarded as useful, and being able to work within the wider development community to communicate ICAI’s findings and build a constituency of interest in this particular body of work, and recognise its value and engage with it more forcefully.  That is a key interface that perhaps also needs a little more work. I would see those three things as priorities.

Q38   Fiona Bruce: Can I probe the more muscular approach to the use of evidence, using the example of the criticism in ICAI’s report on DFID’s anti‑corruption work?  Criticism was made regarding evidence and the use of statistics.  How would you improve that use of evidence?  How would you describe your more muscular approach?

Dr Evans: One of the things we have to be careful about is not critiquing on the basis of paucity or lack of evidence, or the poor use of it by the Department, and then to try to fill gaps in that evidence through methodologies that do not fully stand up to scrutiny themselves.  It is important that ICAI is consistent on this matter. 

I would prefer that in its reviews ICAI spent a lot of its time questioning, challenging and scrutinising the evidence base that DFID is using to make its decisions, manage risk and make operational choices rather than try to go off and generate primary data for itself.  This is a difficult area, because one of the ambitions of ICAI’s work is to get as close to the beneficiary as possible.  That is a very strong and meritorious ambition, but there is a danger in going too far down that road and coming up with forms of evidence gathered under methodologies that do not stand up to scrutiny, which then allow for a challenge to the credibility of ICAI reports.  I would probably want the commissioner team to look at that very closely to see if we can tighten up on that, because it is a potential weakness.

Q39   Fiona Bruce: Looking at the private sector report that ICAI did on the Department’s work, which was Red-Amber, it illustrates that there is quite a steep learning curve still to be travelled by DFID in this area.  Could you say how you would ensure ICAI continued to probe that, and what you see as the next steps for DFID’s private sector work?

Dr Evans: At this point I would not like to comment directly on the substance of what I think should happen in the private sector work itself, but with a report of that nature, which has such fundamental observations to make on where DFID is at in relation to that portfolio of work, you would want to follow that up very regularly, possibly with shorter form reviews that look at specific aspects of the agenda and follow up the management response, but you would want to continue to grow.  That is a good example of where you want to look at a theme over a number of years and then probe it with a number of different kinds of reviews to build a body of scrutiny evidence, if you like, that holds DFID’s feet to the fire on its performance in that domain.

Q40   Mr McCann: As a comment on the last point, you might want to look in detail at announcements by Ministers and how programmes then fit in.  That was one of my concerns about the private sector report, having witnessed the programme in its early days when we visited India.  One of the concerns of the present ICAI commissioners has been DFID’s programme management.  Do you have any views on how that is developing?

Dr Evans: I am not that close to DFID these days, so I am not sighted on what is going on, but I understand this to be a very important feature of a number of recent ICAI reviews, and there has been a programme within DFID to try to address programme management, if you like, in the context of a Department that is incredibly focused on the delivery of results.  There are a whole number of features of that that ICAI should be looking at quite carefully: not only the programme management cycle but also how that operates in different risk environments; the kind of control and risk environment around that; how that is then incentivising staff in particular directions; and how that is affecting overall delivery on the ground.  This is a very fertile area for operational reviews by ICAI over a period of time to build up a picture of how programme management is changing and improving and is line with the aspirations of the Department to be completely results oriented.

Q41   Mr McCann: Do you have any view at this time as to whether there is too much emphasis on design rather than implementation?  Perhaps I can throw into the mix the whole question of the logframe process DFID uses, which some of us might have concerns about in that it favours large programmes to get money out of the door rather than smaller and potentially more effective ones.

Dr Evans: I know a number of bilateral agencies through my career.  I also know quite a few multilaterals, having worked in at least one of them.  It is a culture they share, which is a lot of emphasis on what one might call quality at entry, which is the front end, where a lot of the actual capital goes in.  Then things begin to look a little less well organised, well supervised, at the implementation end, particularly as you get through working through partners, intermediaries and so forth.  This is a problem that is shared across the system, if you like.  Certainly in the World Bank, my experience was this was an issue that had to be seriously addressed and is constantly under focus, both from the executive board but also from the Independent Evaluation Group.  So, undoubtedly DFID needs to address this issue. 

My sense as an outsider is that over the last number of years a lot of effort has gone into establishing a very high quality design end to the work that DFID does, including volumes of evidence that have to be prepared and put together to be able to build the business case.  In many ways, that is entirely appropriate, given the volume of money we are talking about and the often very difficult environments in which that money needs to be spent, with the focus on fragile states and so forth.  But it would not be of surprise to me if that then was not matched by quite the same level of attention to detail at the implementation end, or that the implementation instruments have not kept pace with some of the thinking that is going on in the front end around design.  It would not surprise me, because I have seen it elsewhere.  However, I would not want to pre-judge that.  I would want to spend a bit of time looking at that, because I do not know the ins and outs of what has happened in the last few years. 

Q42   Jeremy Lefroy: DFID has a fairly substantial number of large centrally managed programmes, even in countries where we have bilateral contact.  One of the questions that we have raised is how much knowledge locally there is of these centrally managed programmes.  What is your view of these programmes?  How would you seek to assess them by comparison with perhaps more focused bilateral programmes?

Dr Evans: It is a very complex sort of internal architecture, where you have these quite significant bilateral programmes under the management of country programmes.  Then you have these more centrally located programmes, which are often pursuing a whole number of aims and objectives, quite often at country level, and it is how to get a handle on it.  I suspect that one of the challenges there is simply having a management information system that can keep abreast of all of these things.  You have these horizontal instruments and then these big vertical instruments, and I suspect that is an issue; it is in many bilateral agencies. 

This is a really interesting challenge for DFID: how to essentially get the maximum out of the combination of centrally managed programmes and bilateral programmes, and whether centrally managed programmes are put through the same level of internal scrutiny and have the same level of requirement to build the business cases and the results offers around them, or whether they are somehow treated by a different set of rules.  That really needs to be looked at quite carefully.  My understanding is there are quite large amounts of money behind some of these centrally managed programmes.  It is an area where just simply on the rules of materiality you will want to spend some time as ICAI trying to get to grips with how they function. 

It is certainly my experience from working at country level with a whole number of different aid actors that there is often quite a different incentive set facing staff when they are dealing with their own bilateral programme, versus something that is coming, as it were, according to headquarters’ rules or objectives, and they can get treated quite differently in the field.  But, as I say, I have no particular basis to say that is happening, but it is an area that would be very ripe for a close look, including probably a good look at where trust funds are deployed, which is a weak link of many bilateral agencies. 

Q43   Chair: Thank you.  Just one transitional point: the Secretary of State has said that none of the commissioners should be reappointed, although one of them did apply for this role.  Do you have a view about that?  Would you welcome the possibility of any of them being reappointed?  Given that you would be the new girl in that context, would you prefer to have a new slate?  Do you have a view about it? 

Dr Evans: I am not sure it is my job to comment on what has been decided already.  But in future years it would not be a bad idea to have commissioners contracted on slightly different rolling contracts, so they did not all go at the same time.  That probably would be a good way of building institutional memory and continuity over time so that we did not have this all change, which is going to be for everybody a heavy lift.  One could guard against that by thinking it through.

Chair: And a parliamentary change as well—a new Committee and so on.

Dr Evans: Exactly.

Chair: Do any colleagues have any further questions?  If not, can I say thank you very much, Dr Evans, for coming in and answering our questions?  We are grateful. 

              Oral evidence: ICAI Pre-Appointment Hearing, HC 741                            4