Examination of witnesses
Paul Streets, Gen Maitland Hudson and Dan Corry.
Q51 The Chairman: Good afternoon and welcome. Before we begin, I must tell you that there is a possibility of votes this afternoon. If you hear the bell ringing, we will adjourn the Committee for 10 minutes.
I am Jill Pitkeathley, the Chair of this Committee. I know many of you, so it is very nice to welcome you here. This is the fifth formal evidence session for the inquiry and I must point out to you that it is, as you will know, televised and broadcast. A verbatim transcript will be taken of your evidence, which will be put on the parliamentary website. A few days after this evidence session, you will be sent a copy of the transcript to check for accuracy, and we would be very grateful if you could advise us of any corrections as quickly as you possibly can. If, after this session, you want to clarify or amplify any points made during your evidence or have any additional points to make, you are welcome to submit supplementary written evidence to us. I should also point out that we have a lot of questions to ask you, so fairly succinct answers would be welcome because we have a second session starting at 5.00 pm this afternoon. Perhaps you would you like to introduce yourselves and then we will begin with our questions.
Paul Streets: I am Paul Streets. I am the Chief Executive of the Lloyds Bank Foundation for England and Wales. To put that into context, we are independent of the bank, I have 12 trustees, we are chaired by a Member of this House, Baroness Rennie Fritchie, with eight independent members and four from the bank. As to the inquiry subject matter, we are focused on small to medium-sized charities with a turnover of between £25,000 and £1 million, although I have been chief exec of a large charity too, so I can make some of those comparisons.
Gen Maitland Hudson: I am Gen Maitland Hudson. I am the Head of Impact and Evaluation at the Power to Change Trust, which is an endowed foundation that funds community businesses. I have a longstanding interest in impact and evaluation and have worked with many small and medium-sized organisations, looking at their impact.
Dan Corry: I am Dan Corry. I am the Chief Executive of New Philanthropy Capital. We act as a consultancy and a think tank dedicated to improving the impact of the charitable sector.
The Chairman: Do not necessarily feel that you all have to answer every question. If somebody else has said what you want to say, you do not need to come in on that one. I will start with a general question: is it important for charities to evaluate and measure the impact of their work and how does that relate to the wider issue of accountability? If you think it is important, and I guess that you do, could you combine in your answer the best way for charities to go about this measurement and evaluation?
Dan Corry: In a sense, the purpose of charity is to do good. Everybody, when they are doing their sponsored run, being a trustee or volunteering, is trying to do some kind of social good. The question always is: are you doing good and could you do a bit more good if you perhaps organised yourselves a bit differently, allocated your resources a bit differently and so forth? It is a very important question for charities to be thinking about their impact. What we find is that there are two aspects to this. One is being clear as a charity what it is you are trying to achieve, what a good year would look like and how you think what you are doing helps you achieve that. In the jargon that has grown up, it is not a very nice phrase, but the “theory of change” is the idea of understanding what you are trying to achieve, your ultimate outcome for a set of people or a cause and how what you do contributes to that. Then, once you have got that, you can start to put some concept of measurement around it and the question is: are you managing to achieve what you hope to achieve and is it getting better this year than last year, so that you can go on and do better?
The whole concept, at least that you are trying to achieve something, is very important in the charity sector. It has not always been deep in the charity sector; there has been a feeling that just the act of doing or giving is what charity is about, as opposed to whether it achieves anything. I think most people these days would accept that achieving something and seeing whether you could achieve even more next year for your cause is a good thing to do and, if you believe in that, you do need to get into some of these concepts of impact.
The Chairman: Thank you.
Gen Maitland Hudson: I would agree with that. I think it is a central ethical concern to ask the question: is what we are doing making an appreciable difference to the people we want to help? That is really all that evaluation and impact measurement is; it is trying to systematically and reliably answer that question.
On the broader point you were making about accountability, it is a core part of accountability; it is difficult to know how you would be accountable without being able to answer those questions. The best way to do it is a more difficult question to answer succinctly. There are alternatives and it depends on the size of the organisation and the kind of work you do. I have particular opinions about this, as I am sure Dan does as well, so we will probably get to some of those answers in some of your questions.
Paul Streets: I will speak, if I may, my Lord Chairman, to the third question, which is: what is the best way? I speak particularly for the organisations we reach, which, just to remind you, are small to medium-sized local charities dealing with complex disadvantage. For the purposes of this inquiry and the area we are looking at, I would say that we have distorted the way charities ought to be measuring impact. Actually, what they should do, and what the best organisations that we fund do, is start with the beneficiary, the user, because that should be their primary accountability. My fear is that commissioning has created an environment where that has become distorted and they look up before they look out at those they should be serving, so I would say the primary accountability should be that. The best organisations we fund are very effective at working with those they try to serve and understanding what it is they want.
The Chairman: So measuring the effect directly on the beneficiaries?
Paul Streets: Indeed[1].
The Chairman: Thank you very much. We will move on to our next question, from Lord Chadlington.
Q52 Lord Chadlington: What do you think the obstacles and risks are associated with evaluating impact, particularly if we are talking about small and medium-sized charities where resources are limited, and how can the requirement to demonstrate impact be reconciled with those limited resources?
Dan Corry: It can be hard for a small or medium-sized charity to measure anything and one should not underestimate that. For instance, if you are working with young people to try and get them into jobs, measuring how many you have worked with is quite difficult for some, let alone the more challenging thing, which is how many of them have got jobs who would not have got jobs without you—so your net impact. These things are not straightforward and get even trickier if you are measuring something less hard than job outcomes—their well-being or sense of purpose, for instance—so it is difficult and you have to be proportionate. If you are a small or medium-sized charity, you are not going to be doing complicated randomised control trials or anything like that, so you have to see what is sensible. You might be doing some sensible before-and-after interviews and you might be collecting all sorts of data.
You asked how charities can be helped to do this. There are some things the Government can do. For instance, we have worked quite hard with the Ministry of Justice to create the Justice Data Lab, which is, essentially, trying to help small charities with the problem of following people over time. In this example, a lot of charities work with prisoners and they hope that what they do in working with them leads to less reoffending, but the typical charity does not know whether that is true. They do not even know whether, two years later, the people they have worked with have reoffended or not, and there is government administrative data that would allow you to answer that question. The Ministry of Justice has done a great job in creating this Justice Data Lab so that you can say to them, “We have worked with 100 people. Can you tell us whether they have reoffended or not later on” and they compare it to a control group which they can create. That is an example where the Government are sitting on tonnes of data which would make it much easier for small and medium-sized charities, let alone big ones and private and public sector organisations, to understand their impact, so there are things that the Government and others can do to help in this. That is quite high-level evaluation because it has control groups. Where you have not got that kind of data, you have to think quite hard about what you can do that is sensible. My view is that anything you do is better than just telling a few nice stories.
The Chairman: Paul, do you want to come in?
Paul Streets: You talk particularly about small charities and, in looking at the juxtaposition of large and small, having run a large charity, I think one of the great advantages is that many have a diversified income base, so they can choose how they wish to measure, and there are few obstacles because they have plenty of resources. With small, local charities, the real issue is that they do not have enough resources to invest in evaluation, and we have a contract culture which has driven them towards performing around a contract before the focus on beneficiaries far too often. My fear is that what we have is a numbers game where people will look for numbers that satisfy a commissioner, rather than measuring the views of the beneficiary, and that is certainly what we would like to fund.
To answer your question on how can this be reconciled, I believe that funders, whether they are government or indeed funders like ourselves, ought to be supporting that evaluation[2]. For us, as a funder—and we are not unusual in doing this—we would support an organisation directly in its efforts to evaluate. For example, a very simple measure, as Dan has mentioned, would be the Outcomes Star, which is beneficiary-focused. It can create a conversation between the client and the charity and it can be aggregated up rather than reduced down, which is the concern we would have with the commissioning-based approach. We think that funders like us should be funding those sorts of evaluations. Indeed, we fund, for example, PQASSO, which is a quality assurance system targeted specifically at small, local charities that enables them to think through the systems they have in place to measure what really should matter, which is the outcome for the beneficiary on the front line.
Lord Chadlington: There is of course a follow-on issue, which is that very often a new funder who might come into your small charity, and I am involved in a number of them, will want some pretty hard numbers on how successful you have been to date before they say, “Yes, we will come in and fund this as well”. Sometimes, I am afraid, they are just warm stories about the qualitative effects in a local community where there has been some big difference made, but you cannot quantify what you have done. Can you see a way through that strange dilemma of the local role that so many charities can play?
Gen Maitland Hudson: You can collect systematic qualitative data and systematic feedback. There are ways of improving on those measures, so it is not necessarily all about quantifying or finding hard measures or even using things like Data Lab. It is potentially about improving on some of those softer measures and supporting small charities to be more systematic, and there are different ways of doing that. There is a question about the different demands that are then placed on charities by commissioners, and probably what most small and medium-sized organisations running social programmes need is a trusted way of saying, “This is our impact. This should work for everyone. Here are our measures. You can look. It is transparent and reliable and we are collecting in this way, which is recognised by the sector as being reputable”. We do not have that yet at the moment.
Lord Chadlington: That is exactly right.
Gen Maitland Hudson: But we could get there; it is possible.
Dan Corry: We work with funders and charities and the frustrating thing is to find a charity that finds that each funder wants a different set of metrics on a different timescale, which is completely mad. We often say that, as a funder, you should ask the charity how it assesses its impact and, if you think they are doing it well, then you should use whatever they are doing and, if you do not think they are doing it very well, it may not be an organisation you want to support at all. But, do not ask them for bespoke measurements and then—even worse, as a lot of funders do—collect all this data and do nothing with it.
The Chairman: Thank you very much.
Q53 Lord Foulkes of Cumnock: I have to declare that I am now the Chair of Age Scotland, so I am one of these trustees that I am going to ask a question about. Do you think trustees, unpaid volunteers as we are, have the right skills to enable us to assess impact?
Dan Corry: There are lots of issues about trustees, everyone being volunteers and the difficult job that it is, as we have seen with some rather famous failures recently of charity governance. The issue about impact is not so much that the trustees do not have the skills; it is their incentives—I am a trustee of a couple of things as well—to worry less about the mission of the charity and its impact, which is why you got involved with it in the first place, and, get pushed much more to finance and financial sustainability, the legal requirements and so forth. Some nudges which make trustees think more about impact would be powerful.
We have proposed that, along with your annual report to the Charity Commission, each year you should have to submit something about your impact. If you are a small charity, it might be two paragraphs about how you think you have done this year and how it will go next year. If you are an enormous charity, it would be something more robust. That would make trustees think about the impact the organisation is having, they would have to discuss it at trustee meetings and I think that would be a very useful change.
Lord Foulkes of Cumnock: You said in your evidence that there is increasing difficulty in recruiting trustees. Why do you think this is? Is it because they are frightened?
Dan Corry: Probably, the Kids Company thing had more impact than it should on people feeling risk-averse, “Do I want to be the trustee of a charity that goes down the plughole?” Paul has talked about commissioning. A lot of charities, by no means all, are involved in payment by results contracts, social impact bonds and all sorts of stuff, which is complicated, so people run away from that. The other side is that people have very busy lives and it has got difficult with people retiring later with more family responsibilities and so forth, so there are lot of things pushing against it. Again, rather controversially, we think for some charities, if they can afford it and they want to pay some of their trustees if it means they will get a more diverse set of trustees with the right skills and that helps them turn philanthropic resources into more social outcomes, they should be able to do that.
Lord Foulkes of Cumnock: That raises other problems.
Dan Corry: It does.
Lord Foulkes of Cumnock: New Philanthropy Capital said that they want trustees to play a bigger role in impact assessment.
Dan Corry: What I do not worry about is that you have to be an economist to play a role in promoting impact. I am an economist, but do you need to be an economist to understand all this impact stuff? Not really, although there are some technical things where it might help. For me, it is more a cultural thing—that the trustees should be there and the number one question for them is: is this charity delivering on what it said it is trying to do, helping the people; is it doing the best it could and could we do a better job next year? If you get that mentality in, a lot then falls into place with impact. Technical skills and so forth you can bring in from somewhere else, but you do not need to get fancy on impact measurement for most charities.
Lord Foulkes of Cumnock: Next time, I will have to wear my glasses because it is Paul I should have asked about the difficulty in recruiting trustees.
Paul Streets: Let us compare those two charities again. For large charities, there is no problem with recruiting trustees and it is very easy to get high-quality trustees. For small charities, it is harder because there is no glamour, there is no cachet associated with being a trustee of a small, local charity in the way that there might be prestige with Macmillan or Diabetes UK or even Age UK, so it is harder to recruit.
What we find is that some of the trustees engaged with local organisations are far more effective, I would argue, than those engaged with larger organisations. If we look at some of the charity crises, Kids Company being a case in point, one of the problems was it had an absolutely stellar board on paper, but it seemed to fail in governance. The trustees of charities that we fund are highly engaged in their organisation.
Let me juxtapose this. I visited a charity in Manchester, Haven, that deals with refugees. When I went to Haven, I found the chap who was the treasurer doing the bookkeeping and, later on, I found him stuffing the recycling bin with cardboard boxes. That is a very typical model for trustees of small charities—that they are highly engaged, really understand the front line and want to communicate. The clarity between executive and non-executive becomes much more opaque for trustees, but we do not find the trustees of local charities disengaged and we would argue that there is not an impact problem, necessarily, for most of the small charities we fund. They are telling a good impact story, and the trustees want to tell that story because they have to raise money in a much harder way than some of the big nationals would.
The Chairman: Thank you very much. We will go on to the next question, from Lord Lupton.
Q54 Lord Lupton: I should declare my interest as Chairman of my family’s newish foundation, the Lovington Foundation, which makes me particularly interested to learn from the answers to the next general question, which is: what information should donors, such as foundations and trusts, expect from charities regarding the differences that your funding has made? I will ask a supplementary because Paul Streets may be able to roll the two questions into one, which is: how do you seek to secure good outcomes for the beneficiaries, as you described it, Paul, in the projects you support and what are the main challenges in so doing?
Paul Streets: There are quite a lot of questions there. Your first question was: what should a funder expect in terms of outcome measurement and impact? If I think about what we try to do, the first thing we do is have a look at the organisation. In a sense, we are looking at the organisation and saying, “Is this an effective organisation? Does the chief executive look like they know what they are doing? Do they have a good relationship with their trustees? Do they have a degree of reserves, but not too much? Is their governance in order?” It is a due diligence process and, once we have done that, our assumption is to look then at the work. In most cases, we think that those we fund know more about the work than we do, frankly, so we have a conversation with them and say, “If we’re going to give you X amount of money, tell us how you will report outcomes”. That story ought to be based on beneficiaries because one of the questions we will ask them is: tell us how you are listening to, and engaging with, the people you are trying to deliver the service to in thinking about what you measure and then feeding it back and quantifying that. We then look to aggregate what they tell us, so we do not say to an organisation, “You are only reaching 100 homeless people. That is not enough. You must reach 150”, but we would say, “Actually, if you’re reaching 50 people and it’s really hard to reach them, that’s fine. Tell us how you are going to measure the outcomes for those 50 people in a way that matters to the people you are trying to receive the funds from”. I think that was the first part of your question. I am not sure I have answered the third part of your question, which you might want to repeat.
Lord Lupton: It was the challenges you face in securing the good outcomes for the beneficiaries, so it is partly measuring.
Paul Streets: It plays back to the earlier point about measuring, how they invest in that, and one of the things we do, as a number of funders do, is that, once we have supported the organisation, we will ask how we can help, which is not conditional and the organisation can tell us what it thinks. One thing we fund a lot is support for organisations to think about those measures. With many charities we use something simple called the Outcomes Star. It is a simple spidergram, effectively—where are you currently, where do you want to be, the conversation with you, as the person who is homeless and comes through the door, looking at how that transition is made, and it picks up some of the soft outcomes that Dan was talking about. For some of the very difficult disadvantage we are dealing with — mental health is a classic — one of the outcomes we would want is for someone with poor mental health to be in work but, for some of the most complex disadvantaged, the first outcome might be to get on a bus, so it is trying to measure some of the soft outcomes. For us, the critical bit for funders is to understand that the organisation is robust and then to have the conversation with the organisation about how it best measures the outcomes it achieves for the people it is there for.
The Chairman: Lady Scott, you wanted to add something?
Baroness Scott of Needham Market: One of the questions that comes up a lot is the amount that charities spend on administration and management. Thinking about small and medium charities, do you come across a tension whereby trustees maybe have a sense of some of the things they should be doing, but there is a cost attached to it and they are nervous about pushing up their admin costs? I just wonder if that is an issue.
Dan Corry: You are right, there is a problem where people judge charities on the proportion of money that goes straight to the front line, so people get nervous on admin costs, but the thing that causes smaller charities to not do quite the right thing, frankly, is they cannot afford it. We worry a lot about this on our consultancy side. Obviously, the really small charities cannot afford us, so we put out a lot of free stuff on the web and do cheap training courses and that kind of stuff, but it is difficult. Actually, it is not just that it is difficult to pay for it but, in a small charity of the kind Paul was describing, finding the headspace to even think it through is the real problem. The danger with saying, “They are small charities, so we will let them off the hook on all this” is that they could carry on doing something which is not having much impact. Even worse, it could be undermining what another charity down the road is doing when the two of them are going down the plughole and they should be working together, et cetera. I am loath to say that, because the Treasurers are helping stuff envelopes, it is all right for them to not report too much about impact, except with a few stories. I would go to Gen’s version, which is that, if you collect soft data, which often you want to, you can still do it well or badly, and anecdotes from your few success cases is not good enough. If you are interviewing, quite likely, a random set of people you work with, some of whom your programmes did not work for and some of whom it did, that is good evidence.
The Chairman: Gen, did you want to add something here?
Gen Maitland Hudson: I was going to add to Paul’s point about systematic feedback from the people your programme is aiming to help. First, that should be systematic and you should, as Dan says, be asking across your programme and not just picking out the success stories. That is a risk and it does happen—that you create case studies of the most successful outcomes; so trying not to do that would be one thing. The other really important point about feedback is that you should be acting on it. Do not just collect it and then leave it stored in some database, but you should be, of course, correcting, thinking about the feedback you have received and using it. There is no point in collecting data that you are not using in a programme, and this still happens too much. There is still an issue where even relatively small and medium-sized organisations are probably collecting too much data and potentially too much bad data and then they are not using it. One of the things, as a trustee, you might want to ask is: if you are collecting this data, what do you then do with it and what decisions have you made on the basis of this data; and you have got all these opinions from your service users, so how are you acting on it? It is those kinds of questions, and they do not take a huge amount of knowledge to get back to the skillsets that are required of trustees. It is about valuing information. This is a generalised skillset that people should have. Anyone serving as a trustee should be able to think about the reliability of a piece of information. We have to do this in our daily lives ourselves. All you are doing is trying to do the same thing in the context of the charity for which you are a trustee, so ask about what that data is being used for: “How has it changed the service that has been delivered? How has it improved that service? Have you then received more feedback that has told you that that has worked?” It is those kinds of questions that matter and they should apply to any size of organisation, however small.
Lord Lupton: Just following on, as a matter of interest, what do you think a good foundation or trust should do when it gets information which it regards as inadequate? Should it just close its chequebook or should it have an interaction with the charity, saying, “This is what we do not like”?
Paul Streets: As a funder, there is a real danger here, which is that we create a game where we say, “You are going to reach 100 people”; “Okay, we will tell you we have reached 100 people”. We are not auditors, so what we want is an honest conversation where, if they only reach 60 people, they tell us and we understand why and we think about what we might be able to do to help them reach 100, if that were the objective. That is the real problem as a funder. If you look at the range of organisations we fund, some of them massively over-perform in the targets they set themselves and some of them under-perform. The under-performing ones are not the ones where we say, “We are not going to fund you anymore”, but we would have a conversation with them and say, “Is there anything else we can do to help?”, which is the way we think funders ought to respond.
Gen Maitland Hudson: You want to look for the corrections, so that, if you are getting data and it is telling you that something is not working, you are able to act on it and you can demonstrate that you made changes to your programme. That is what you want, an organisation that is learning and improving.
Dan Corry: Funders come in different types. Those that are funding individual responsive-mode grants, it seems to me, should be helping the charity they are working with to get better, so they should go back and say, “Actually, how are you using the measurements you made? We think it would help you more if you did it like this”. If you are a funder who has an aim of changing opinion about something and you are funding a number of organisations to help you do that, then you will measure the impact in a very different way and you will want to understand whether you are actually achieving anything. We sometimes have funders come to us and say, “We fund all these different charities. Can you do an assessment of it?” which we can do at an individual level, whether they have been successful or not, and then they say, “Can you add it all up and tell us what it all comes to? Is it seven or 10?” It is a completely crazy question.
Q55 Lord Harries of Pentregarth: Obviously, charities vary greatly in their ability to assess impact. The less experienced ones, perhaps the smaller ones in particular, how can they best be helped? What actual form should the help for these small charities, in particular, take?
Dan Corry: There has been a programme which we have been involved in running on behalf of a number of other organisations, NCVO and so on, which I think you have heard from previous speakers about, which is Inspiring Impact. It is an attempt to try and have some tools online which are very simple for charities to use to think about their impact and then find ways through. That is an interesting programme. It was powerful in the beginning, not only because a number of groups came together but because the Government funded a bit of it to start with. The cash was nice, but the symbol that the Government cared about this was quite important. There are things like that.
As I say, the change in the incentives through reporting and so on would make a difference and, as I have said before, anything that can make data very easily available to small and medium-sized charities will help. We are going into this world of big data and social media, and it does not mean that it suddenly becomes costless, but there is more potential to do these things.
Beyond that, we recently started some governance awards with the Clothworkers Livery Company. It was the first year last year. There were three awards for a board which helped push on impact in their charity and the winner of the small one, three employees or less, was a community centre where they had always said that part of what they were about was bringing the community together. Basically, they just worked out who actually came to the community centre and they worked out that they were not bringing in the whole spread of the community, so they started to do some outreach and put on some slightly different classes to change the mix of people. That was not fancy economics or anything, but something that almost any small charity could do.
Paul Streets: First, I would like to challenge the presumption that small charities are not good at measuring impact. Our experience is that many of the small charities we fund are very good at measuring impact. What we are not very good at, as funders, is being permissive in how we think they ought to measure impact because we have a predetermined view of what it is like. Let me give you three examples of organisations we fund that measure impact in different ways which are all valid.
The first would be One25, a charity in Bristol that deals with sex workers. It reaches about 250 women every year who are on the streets of Bristol, sex-working. It has very clear outcome measures. If you look on its website, you will find a section on impact and it talks about the work it has done to reach those women and get them back off the street. It talks about the women it has in secure housing in a very quantitative, hard way that even a commissioner would understand.
If I think of another charity, Landau, it deals with young people who are severely learning-disabled in Telford. These people are so severely disabled that it cannot tell you a story about work and employment because these people are never going to get work or employment in most cases. So what it tells you is the qualitative stories of the impact it has had on their lives: the fact that John, who is learning-disabled and will live with his parents for the rest of his life, suddenly gets on a bus for the first time to go for a job interview. That is his success and that is a qualitative story.
A third charity we fund, TwentyTwenty in Loughborough, works with young people who are NEET—not in education, employment or training, which is another dreadful acronym we have created for these people—and it actually quantifies it in an economic way. It has worked with Leicester City Council and it has created a measure which says that, for every £1 it spends, it returns £5.06. These are not un-sophisticated ways to measure. I would argue that large charities are often much worse at measuring outcomes than smaller charities. What is so difficult is how to aggregate the three, as Dan says—how to aggregate the story of Landau with the story of One25 with the story of TwentyTwenty. But each one tells an impact story which is very powerful in its own way and we need to hear that rather than pretend that they are all measured in the same way.
The Chairman: Thank you. Gen, did you want to add something?
Gen Maitland Hudson: I would probably come back to a slighter higher level of abstraction than the individual charities. It is possible to help smaller social programmes to navigate reliable collection analysis and use of data, some of which is digital and some is about providing tools. There is a programme I am involved with that has been funded by the Access Foundation for Social Investment: the Impact Management Programme—being run by NPC, in fact—which is trying to look at exactly what kinds of tools can be provided. It is building on some of the work done with Inspiring Impact, but it is looking away from the theorising bit at the beginning, where Inspiring Impact was relatively strong, and looking at the actual collection and the analysis. The analysis is often the bit that gets left out, so people will be quite good at coming up with a theory that gets support for that, they then think a bit about the measures, but actually analysis is time-consuming. It is hard and you are being told that you should be triangulating measures, collecting different sorts of measures and that, as a principle, this is something you should do—but what and how? These are quite tough questions. Coming up with good, standardised, minimal ways for different kinds of sub-sectors to triangulate effectively in comparable ways is important progress we could make as a sector which, as bigger organisations and foundations, we could and should support, and we are trying to make progress on that. I think there are more systemic ways of doing it. It is also about the recognition of the different kinds of impact measurement, which is important, but it is that systemic change too and looking at where organisations are lacking resource, ability, skill and time. At the moment, it is a bit further along that journey in looking at analysis and the use of data.
Q56 Baroness Jenkin of Kennington: In part, this is just following on from what Paul has told us, but is it possible to compare and contrast the impact of charities working in different circumstances, and is the benchmarking helpful? I know that both of you have been involved in this debate as well.
Dan Corry: Within sectors, where there are a number of charities doing a very similar thing, the more you can have shared measurements and metrics, the better, because you can then learn from each other, and there are some great examples. Caada, now SafeLives, on domestic violence, tried to bring a whole bunch of organisations together to use similar metrics and they started to learn from each other. Some seemed to be doing better than others and everyone tried to see what they were doing that was different; there were some very interesting stories. Within sectors, I think you can, within reason, have some shared metrics and we are very keen on that.
What I get unhappy with is when you start comparing completely different sectors, because I think it is apples and pears at that point, if you push it too hard. It is why I am a little unhappy about some of things that Paul talked about—£1 in and £7 back, social return on investment, which is a good way of thinking about things if you follow through the process, but the numbers are often ridiculous, quite frankly. You just do not have the data to say, and you could have a massive range.
I was a Treasury economist at different times in my career and, of course, that was the dream in the Treasury—that we would be able to work out if we should give the extra £1 to DCMS or to DCLG. The answer is that it is the politicians who have to decide that and then we can allocate it appropriately, and I think it is the same in the charity sector. There has been a slight danger. I have forgotten the name, but there is a trend in philanthropy that you should be working out this kind of return and then giving all of your money to the one with the highest return. That, I think, is a bit of a crazy way to go.
Gen Maitland Hudson: There is a question about what benchmarking is helpful for. It is not helpful in the absolute sense, but you need to think about what you are going to use it for. With any kind of data collection or measurement system, there are different levels of abstraction and there are some levels where you need granular data. If you are looking at something and it is detailed, you need a lot of information. If you are a doctor and you are talking to a patient, you need to know specific things about that patient in order to treat them. If you are thinking about budgeting for a cancer treatment across a CCG, you need less information and you can come up to a level of abstraction. It is thinking about, “Helpful for what?”—what do you want your benchmark to do? There are points at which benchmarking can be extremely helpful and a relative level of abstraction can be useful for a sector, and it probably is at sub-sector level—it is not going to be quality-adjusted life years across every single possible social programme that you could run. People have tried to do this, but I think you will just end up funding malaria nets if you do that.
There are different points at which you could benchmark and it is worth while thinking about those. Some of that might be at a community level. You might want to think about neighbourhood and how you could fund locally in order to get good, connected kinds of funding through a local area, and some of that might involve some clever benchmarking. It is always important with data to think about what you want to use it for rather than to create rules that you are trying to stick to in the abstract.
The Chairman: If I could direct this at Paul, would you use that kind of benchmarking to identify gaps in social need and in community that you might then, as a funder, fill?
Paul Streets: It is a slightly different question. A lot of the funders have a conversation about what we would call “cold spots”, which are areas where we might fund less than we would wish, given need, and that is a very active conversation that funders like us have. One of the questions is: we are not development agencies, so can we support very grassroots-based community work from the ground up where there is a lack of capacity because there is an inverse relationship between need and voluntary sector capacity across the UK? That is an issue for us, as funders.
To the specific question on benchmarking, a good example of that is my own sector, which is funders, where I think we can be benchmarked. A very good benchmark for us is our efficiency—how many people apply to us relative to the number who actually get a grant? If that number is very low, we are wasting their time and our time, and actually these organisations are very short of time, so that is one thing we should not be doing.
It is much more challenging across sectors, as Dan has said, but we do a lot of comparisons by bringing organisations together to peer-review. It may also be a good example of homelessness organisations that work together around a benchmark. There is Mind, with mental health and, as Dan has said, Women’s Aid, which come together, and we do a lot of work with them to think about how they can benchmark each other to a degree, where there are comparisons, but it depends of course on the case mix.
The Chairman: Thank you. Can we go on to your question, Lord Bichard?
Q57 Lord Bichard: All three of you have touched on the issue of contracting and commissioning and maybe we could open the issue up a little more. Do you think it has changed the role of charities? How do you think it has affected impact measurement and assessment? Has it distorted it? Finally, if you agree with me that it is probably not going to go away, and maybe should not go away, how can we improve the process?
Dan Corry: Obviously, once you have been commissioned to deliver a service, your accountability is switched away from the beneficiaries to the commissioner and that can be a problem, and charities constantly need to ask themselves whether they are now still serving their mission. We had Lord Kerslake speaking at our conference last week and he went out of his way to say that, if it is a contract that you think takes you away from your mission, you can say no—which is a bit easier to say than in the real world for some small charities. Recently, the Children’s Society has decided to walk away from going for contracts for providing children’s centres because they thought they were not adding a lot of value and it was taking them away from their mission.
The strong trend in commissioning has been towards payment by results contracts. There are a lot of issues with them, although the NAO has done good work and all the rest of it. But it has brought a focus on outcomes and metrics and I think that has been quite powerful for some charities—that they have had to think harder about what they can deliver and to think far more about value for money, which we have not mentioned yet. We are talking about outcomes, but charities often say, “Look at all the outcomes I’ve created. Isn’t it great?” and you say, “Yes, but it cost you £10,000 per outcome. Isn’t that ridiculous? Couldn’t you have done it for £2,000?” That gets missed out in the charity world, quite often. PBR contracts start to bring all that in. There are lots of problems in them, and I am sure Paul will talk about some of them, particularly for smaller charities, but they are bringing that kind of ethos about measurement, project management, real-time adjustment of projects and so forth, which is quite useful. As you say, it is not going away. As far as I can see, it is growing.
Lord Bichard: So how would you improve it?
Dan Corry: Charities have to be very clear whether they want to get involved in these contracts or not because they can pull you away from the mission. A lot of them feel, though, that they can achieve more if they have some freedom via outcome-based contracts, which causes them cash flow and risk problems but nevertheless gives them a bit more freedom to deliver in the way they think is right, which is a good thing. They have to make sure the contract is right, so a lot of charities moan and say, “We’re delivering this contract, but we know it’s not really the right thing for our beneficiaries”. One hopes that they will try and push upstream and talk to the commissioners, whether they are CCGs, local authorities or whatever, and try and change the contract. If the contract is wrong, it does not matter who is delivering it.
Similarly, some charities have problem with PBR contracts because, however clever the contracts are, they will make you cream-skim a bit—deal with the easier cases because then you get the payment, rather than the more difficult cases. For a lot of charities, that is absolutely against their whole moral belief that every person is of equal worth, so there are lots of issues. The sector is dividing into a certain number of almost business charities—some people think that maybe they are not charities anymore—which are doing a lot of contract work and other charities are keeping out of it. In a lot of areas where charities used to get grants, the grants have gone, so, if you do not go for the contracts, there is nothing else coming from the public sector, which Paul has been very vocal about.
Paul Streets: We can compare, but it depends on the outcome you are trying to achieve. There are certain contracting things that are just activities: meals on wheels services, hours of domiciliary care or, in my old organisation’s case, Diabetes UK, the number of people diagnosed—very straightforward and not highly tailored. The market we are in, where the bulk of the sector is, is a third of the sector—charities of £25,000 to £1 million that often deal with quite complex disadvantage—but we think contracting and commissioning has been wholly inappropriate and very destructive. It has made those organisations look up towards commissioners, rather than out to those they should be serving—to look for simple measures, like payment by results and social investment, which the Government are obsessed with, which is a problem because it does not work in that context because it is a simple outcome measure based on a unit cost. What we have seen, as Dan has said, is that shift from grants to contracts.
The consequence of that, which you can interpret as good or bad, is that, between 2008 and 2013, government income to charities with income of between £25,000 and £1 million fell by up to 38% and government income to charities with an income of more than £100 million went up by 38%. As that shift from grants to contracts has happened, we have seen the Government increasingly supporting large organisations. Now, you may think that is a good thing. We think it is a really bad thing for people facing multiple disadvantage.
Our answer, which is unfashionable, is grants. As a grant-maker, we are very discerning. We have a conversation with the organisation about what it wants to achieve with the money we are giving, and we do not predicate the outcomes and make it play a game with us about telling us what it wants to achieve because it thinks that is what we will pay for. We have an honest conversation with it and we do not fund rubbish, but the best organisations. What we think, although unfashionable, I suspect will be back one day and we will recognise that grants actually obviate many of the problems we have with commissioning in the bureaucracy, contracts and all the stuff that ties local authorities up with the scarce resources they have. We think grants can be the answer, if they are used well.
The Chairman: Lady Barker and Lord Foulkes both want to come in.
Baroness Barker: I come from a background in health and social care and I have seen a load of contracts, the terms of some of which would make your hair stand on end, frankly, if they were presented to you as an organisation or an individual. About five or 10 years ago, there was a big trend, particularly in local government circles, for total place budgeting. Bringing us back to the issue of impact and measurement, many of the things that particularly medium-sized and local charities are contracted to do would have an impact in a way that the charity would not have the data itself to measure. They might be stopping people going to the doctor unnecessarily or maintaining people with dementia at home for another year. In the work that you do, have you ever come across a system, probably reflected as a contract, where there is a duty to share data between the statutory agencies and the charities so that you get real figures out of them?
The Chairman: I am going to ask Lord Foulkes to ask his question as well, so you can roll both of them together.
Lord Foulkes of Cumnock: Paul, you talked about grants. We are getting evidence that impact is better if you have top-quality chief executives running the organisation, yet, when you are applying for a grant, you grant-givers do not like to fund central administration. Is there not a dilemma there?
The Chairman: Both those questions, please.
Dan Corry: The first one links to the second. You are right, a lot of charities do things which they are convinced are not only good for the people but are actually saving somebody else money, and it is very hard to show that. In the case of health, that means that the public sector uses charities probably less than they should because they are not sure either. We did a bit of work recently for the Richmond Group, which is the big health charities combined together, looking at their evidence as to whether they really did have impact, and it showed that they are pretty good actually, as evidence goes. I have seen lots of government evidence of programmes, and theirs was good. It is hard to prove it, which is one reason why I was talking about these impact data labs before and the one in the Ministry of Justice. We have done a lot of work trying to get one in the Department of Health because it should be possible to say, “We work with all these people. Can we use the health records to see whether we actually did reduce the number of A&E days?”, or whatever. That is perfectly possible.
There is work going on with NHS Digital at the minute, pilots and so forth, but there are obviously privacy issues, consent issues, et cetera, and the Richmond Group is doing a bit of work in Somerset as well. There is potential for us to crack some of those problems, which is not only great for the charity sector but, ultimately, if it is true, as we all think, that a whole myriad of small charities doing their work with people with dementia and so forth keeps a lot of people out of the statutory system, we desperately need it, so we need to let this data fly. Of course, doing all of this does add to admin costs.
The metric we really want is: what is the impact that this charity creates? If they have to spend half of their money on digital stuff, IT and whatever in training their people to create that great impact, so be it, rather than spending nothing on it and, when you analyse it, creating no impact at all.
The Chairman: We will ask you to comment about the shared data, Gen, and then we will come to you, Paul, about the grants.
Gen Maitland Hudson: Yes, there is a total place equivalent impact measurement, which is called “collective impact”. The idea is that you bring a group of organisations together which may be different parts of that system, so doing little parts of a greater whole, so they might be helping with keeping people at home, they might be preventing hospital presentations, but those will be different organisations playing different roles within that bigger puzzle. They would take a collective impact approach, so they would have an approach to sharing data, they would have an approach to having a collective measurement and not only sharing data but sharing learning and, of course, correcting as a collective rather than as individuals. It is still in its very early stages and we need to pilot more of it and see how it works most effectively, but yes, there is a way of doing that and we should be doing more of it. That kind of collaboration is incredibly important not only because the small organisations will gain so much from it, but because we will then have a much better picture and we can start building the data systems that sit within that as well and allow for the sharing of information.
The Chairman: Paul, your response on this?
Paul Streets: On the grants question, you are absolutely right: grant-makers should be funding chief execs’ salaries and a number of us do. We will fund, frankly, rent and rates, if that is what is needed. We are a core cost funder and, if we think the organisation is funding work that we want to support, we will fund anything; it does not matter how glamorous it is—we are not just looking for a sexy project, as is often the case.
A very good example on those costs and how they are allocated across would be a charity, Justlife, which we support in Brighton and Manchester. Justlife has worked with the CCG to identify the impact of homelessness on A&E and, as a consequence, the CCG is paying for those outcomes because it recognises a direct reduction in A&E attendance through tackling homelessness.
The real problem here is joined-up government. Successive governments have talked of joined-up government and they have singularly failed to do it. Actually, total place budgets might allow local authorities to have that money and then allocate it where the saving was across the piece. The issue with most of the work we do is that the benefits might be in Justice, they might be in DCLG, in health, in social care—and they are probably in all those things—but none of them will collectively pay for them, which is the real problem.
Dan Corry: I would add a rider to that. I am hopeful that the devolution city deals may allow some of that total place stuff to happen and may join up some of the budgets and stop some of the silo working that stops the sensible thing happening.
Q58 Lord Rooker: I do not want to sound patronising, but I have read all the evidence and your three pieces of evidence, to me, are in the top 10% of what we have read and received in usefulness from a practical point of view.
I want to ask about independence. Is it the case that donors do not really want to be supporting an arm of the State, otherwise they will be paying more tax for a start? Given the changes that have taken place, some of which come out of contracting, as Lord Bichard said, and allegations that charities which are business charities are almost an arm of the State, how can we preserve their independence? What can the Government do, through whatever offices they have? I notice that you are quite critical of the lesser powers of the Office for Civil Society, in Paul’s piece. What can the Government do to support the charities in measuring their work that still maintains their independence, bearing in mind this other aspect of the work that many of them do, whereby they become contractors and arms of the State, which has completely reduced their independence?
Dan Corry: You are right, there is a tricky balance there. I remember that Nick Hurd was the Charity Minister at the time when the Government decided to put some money into Inspiring Impact. Quite rightly, the Government felt it was not their job at all to tell the charity sector how it should go about measuring impact or that it should definitely be doing it, but they felt it was okay to support a number of groups in the sector coming together to try and help the sector because, ultimately, the Government would like a voluntary sector that is creating as much social good as possible. It is an externality, if you like, that we would all like to happen, but no one charity has quite got the energy or resources to make it happen and we need some shared metrics as well. The Government have to be careful in this area.
There is a different issue about contracts and whether that is leading some charities to lose touch with their mission and become prisoners of the State and so on, but charities vary in that. Even with charities whose income mostly comes from contracts these days—and there are a number of them—some of them are silenced, and you think that maybe they never speak out because they are worried about never getting another contract, and some of them are very loud, so I do not think you can see a pattern which means that, because you are getting contracts, you never criticise anymore.
The Chairman: Gen, do you want to comment on that?
Gen Maitland Hudson: It is not a subject I know a great deal about.
The Chairman: We will move to you then, Paul.
Paul Streets: Independence of the State; it is a good question. We do think there are different charities and some of those that have a £100 million-plus turnover are probably, effectively, public bodies. I used to run a public body. To me, the governance model they probably ought to have is that of a public body; they are, effectively, executive agencies and receive all their money from the State. That is very atypical for all the organisations we fund, which might, at best, get 20-30% of their money from local and central government. I do not think local and central government can tell them how to measure, but they need to ask them how they should measure, which is the response we would have, and then support them to do that by very simple measures.
In fairness, Rob Wilson and the Office for Civil Society have championed small charities and are now working with them in thinking about how they can improve leadership capacity, and part of that is helping them to get better at measurement. I do not think it is about the Government dictating, as that is the contract culture that we want to avoid, but it is about helping them to use simple things like Outcome Stars and PQASSO in terms of quality assurance—some of the simple tools that are in place and can be available. But as Dan said, often they have neither the time nor the headspace to get their heads round implementing them.
Dan Corry: The Government could open the doors to all the administrative data they have and make it easily available to charities and others so that they can do an evaluation. There are tonnes there and it is just a goldmine.
Baroness Jenkin of Kennington: Why do you think they do not?
Dan Corry: They have been very slow off the mark. It is interesting that, of all the departments, the Ministry of Justice, at the time with Chris Grayling as Secretary of State, went first. I do not know exactly why. The academics are frustrated by this as well. I was on the ESRC Research Committee until recently and there is frustration there. There are privacy issues, obviously, and consent issues, but I find it difficult to understand. It costs the Government a bit to run the Justice Data Lab, but you could have a different model where organisations had to pay to use it. I think we are making some progress now, so I do not want to be too critical. We thought we had the shutters up against a lot of this, but there are better sounds from DWP and Health and Education now.
The Chairman: We are looking for positives in this Committee, so that is good news.
Dan Corry: Yes, we want to encourage them all.
Lord Rooker: On your reference to the Minister and the interest they are taking, in your evidence you actually said that, potentially, the influence of the Office for Civil Society is decreasing. In that case, whose is increasing?
Paul Streets: Arguably, whether we like it or not, the organisations that we are funding are becoming less dependent on the Government and, therefore, they are looking for less influence from the Government. The paradox is that the Government want to silence them. We saw previously the anti-advocacy clause, which we are deeply opposed to, as a funder, and we think that it is not completely off the table, which we would have concerns about.
We are trying to advise the organisations that we are working with to help to leave the King’s shilling behind and find a different way to be funded. The problem, of course, is it might be fine if you work in Shoreditch, but it is pretty tough if you are in Redcar or Merthyr Tydfil, frankly, which is the problem. Some of these areas need to take funding to continue to do what they do, so it is frustrating.
The Chairman: We have one last question, from Lady Scott. We are asking this of all the people who give evidence to us.
Q59 Baroness Scott of Needham Market: Looking forward to next spring when we publish our report, can each of you say what would be the one recommendation where you would say, “I am so pleased they have put that in”?
Dan Corry: Can I have two?
The Chairman: No, one.
Dan Corry: It is a kind of nudge towards trustees and governorship of the sector thinking harder about impact. It is saying that charities need to report annually on the impact they have achieved this year and how they are going to improve it next year, and that would be proportionate and, if you are small, it would be tiny. I think that would change the culture. At so many boards, you are lucky if you get round to talking about that at the end of the meeting. It would change the dynamic. It is not telling charities what to do, it must never do that, but I think it would have quite a profound effect.
The Chairman: Thank you.
Gen Maitland Hudson: One way to look at this is to say, “Charities should do such and such”, but there is something the Government could do, which is not recommission or extend programmes where there is good evidence that they are not working.
The Chairman: We have seen some of those.
Paul Streets: They should rethink their approach towards grants.
The Chairman: Thank you very much for all your answers and particularly for those succinct recommendations. We very much appreciate your coming. Remember what I said at the beginning: if there is anything else you want to add, please let us have it, so look at the transcript as soon as possible and let us have any comments on that. In the meantime, on behalf of the Committee, thank you very much indeed for your attendance.
