Public Accounts Committee
Oral evidence: Troubled Families, HC 711
Wednesday 19 October 2016
Ordered by the House of Commons to be published on 19 October 2016.
Watch the meeting http://www.parliamentlive.tv/Event/Index/f905703b-9df2-4dd9-9132-581b7e64bd56
Members present: Meg Hillier (Chair); Philip Boswell; Chris Evans; Caroline Flint; Kevin Foster; Nigel Mills; Bridget Phillipson; John Pugh; Karin Smyth.
Sir Amyas Morse, Comptroller and Auditor General; Adrian Jenner, Director of Parliamentary Relations, National Audit Office; Aileen Murphie, Director, National Audit Office; and Marius Gallaher, Alternate Treasury Officer of Accounts, were in attendance.
Questions 1-158
Witnesses
I: Dame Louise Casey CB, Director General, Casey Review Team, Department for Communities and Local Government; Melanie Dawes CB, Permanent Secretary, Department for Communities and Local Government; and Joe Tuke, Director, Troubled Families and Public Service Reform, Department for Communities and Local Government.
Witnesses: Dame Louise Casey CB, Melanie Dawes CB and Joe Tuke.
Q1 Chair: Good afternoon. Welcome to this afternoon’s meeting of the Public Accounts Committee. Before we start, let me give a special welcome to fellow members of Public Accounts Committees from around the world, who are here to watch our sitting today. You have an international audience today, Ms Dawes—your fame is spreading.
We are doing a review, a call-back if you like, of the Troubled Families programme, which was a policy priority at the beginning of the last Parliament for the then Prime Minister, David Cameron. We hope that we can have a thoughtful discussion today, because we know that support for families with complex needs is not new, and it is also very complex. Policies in this area have spanned Governments of all political colours.
With all the programmes that have happened, there has been some evidence of good work, including with this one, with some families, but the work of this scheme—as the NAO Report and the evaluation reports of the Department itself have highlighted—has been clouded and, frankly, let down by many systemic and evaluation failures. The lack of clear parameters and a fixed evaluation framework has undermined success. As we have seen in evidence from various organisations, a lot of the changes are behavioural, and those changes in attitude matter but are hard to measure. So we want to get into a bit of that at some point in the hearing.
As a Committee, we are very clear that, yes, we are here to look at how taxpayers’ money is spent, but we are not just bean counters. We want to look at what is happening around long-term change, and at how effective and efficient this whole process has been and will be. But at the moment there appears to be little measurement or any plan to measure longitudinally, and that is something we are concerned about, so we will want to get into that.
Before I get into the main subject area, I will introduce our witnesses. Then I want to talk to Melanie Dawes, in particular, about some of the information we have received. Our witnesses today are: Dame Louise Casey, who is now Director General of the Casey Review Team at the Department for Communities and Local Government, but who was head of the Troubled Families programme in the last Parliament; in the middle, Melanie Dawes, Permanent Secretary at the Department for Communities and Local Government, the sponsoring Department for the Troubled Families programme, which is a cross-Government initiative; and Joe Tuke—I think this is your first time in front of us, Mr Tuke, isn’t it?
Joe Tuke: It is, yes.
Chair: Welcome. You are the Director of the Troubled Families and Public Service Reform team at the Department for Communities and Local Government, which makes you the man now leading on the Troubled Families programme. Our hashtag, for anyone following, is #troubledfamilies.
Before I get into the main issues in the Report, Ms Dawes, your Department initially expected evaluation in early 2014, as you know, but then it stretched to late 2015. In August there was a leak to the BBC of some of the evaluation work in the Department, so on 5 October our Committee requested the main report and the five feeder reports into it. Initially, your Department promised that we would receive two of those, but we did not get them. Who was suppressing the release of these documents?
Melanie Dawes: First, may I say thank you for the opportunity to talk about the programme today, but—
Chair: Could you answer the question?
Melanie Dawes: But I am keen to answer that question about the timing immediately. The evaluation was actually finalised on 4 October and you wrote, I think, on 5 October. My Secretary of State was very clear that he wanted us to publish the reports as quickly as we possibly could. We had the final proofed copy of the overall synthesis report, which was the remaining piece of the evaluation, on 10 October, which was a week ago, and then we published the reports on Monday. I appreciate that that has not left very much time for consideration—
Q2 Chair: Sorry, you just said that there were finalised reports on 4 October, but we requested them on 5 October.
Melanie Dawes: Yes, and that simply was the nature of the timing. We finalised the overall synthesis report, which is the one that sits across the top of all the different elements of the evaluation. We were then thinking about how to publish those documents, and it just so happened that your letter was sent on 5 October. We have moved as fast as we could to—
Q3 Chair: That is amazing timing. But the BBC had a copy of a report in August, and it had originally been promised early in 2014, so are you telling me that two and a half years later, by a miraculous coincidence, the report was finalised the day before this Committee asked for a copy of it?
Melanie Dawes: Well, I don’t know what copy the BBC had, of course—it was a leaked document—but what I can say is that the National Institute of Economic and Social Research bit of the evaluation was finalised at the end of August, and that forms part of the overall evaluation.
You are right that we intended—certainly recently our intention had been—to publish the full evaluation at the end of 2015. What happened is that about a year ago, towards the late summer of last year, particularly with the National Institute element of the work—the very complicated and quite experimental national data evaluation—we began to get interim findings from the researchers and it became very clear that there were very many data issues that were causing problems with the work. We worked with the researchers through the autumn and the early part of this year, and we brought in an independent academic—Anna Vignoles from Cambridge University—to look at the data issues and confusions, and it therefore took longer.
Q4 Chair: This is all process, but that is extraordinary. So, let me repeat, it was expected in early 2014, yet two and a half years later it had not been published. That does not seem feasible. There was something out there—something had been produced—so who was stopping it from being released into the public domain? Was that you, Ms Dawes?
Melanie Dawes: There was nobody stopping it; it was just that it was not yet completed. I think we have learned along the way—
Q5 Chair: So was it not yet completed, or was it not yet completed to the satisfaction of what the Department wanted to release?
Melanie Dawes: The work just was not finished. These are independent evaluations—
Q6 Chair: I hope you get a discount from the evaluators. Are these documents that we have got now, which we have only had the chance to go over in the past 48 hours, or less, the same as the ones that were being discussed this summer? What has changed in them since August that they could not be published in August, once they had been on the BBC?
Melanie Dawes: The way the sequencing worked was that the individual elements of the evaluation were completed first. As I said, the critical National Institute of Economic and Social Research one was finished at the end of August, and during September ECORYS, the overall contractors, put together the synthesis report and discussed that with us. That took a month, and it was finalised at the beginning of October.
Q7 Chair: So you had other reports, which you could have released at any point once they were produced.
Melanie Dawes: We did in some areas. For example, on the cost savings calculator we published some elements of the evaluation earlier. I think it is important to see the evaluation as a whole. We decided it was best to do that and put the whole evaluation together with this political synthesis report.
Q8 Chair: You had various reports that you didn’t publish. What were you trying to hide in them that made you wait and sit on them?
Melanie Dawes: We weren’t hiding anything. In fact, I should say that I think the evaluation is very ambitious. The Department invested in a way that no other social programme has invested in an evaluation, but it was big and complex. It took longer than we expected and we have learned from that.
Q9 Chair: When did you see the first drafts of the reports? Did you alter, or request alteration to, any of the reports that were published that led into the main report?
Melanie Dawes: We always do quality assurance on research, but in the end they are independent research findings. The most difficult was the National Institute element, which was undoubtedly the most cutting-edge and experimental part of the evaluation and the most difficult to pin down and finalise, and where all the data issues really came into play. Because that was so difficult, we knew it would come under scrutiny. That was why we got Anna Vignoles to look at that and to quality assure for us not just the work, but how the conclusions were presented.
Q10 Chair: So did you personally look at how conclusions were presented in that report?
Melanie Dawes: I did not personally get involved in that process.
Q11 Chair: Which official in your Department looked at the presentation of that?
Melanie Dawes: Our analysis directorate oversees all of the Department’s evaluation work and the relationships with the contractors.
Q12 Chair: Did they go back and ask the researchers to change any of the recommendations?
Melanie Dawes: There were certainly some areas where we— In the end, as I said, we made sure that none of these judgments were made by us alone.
Q13 Chair: Just answer the question: did they go back and ask the researchers to change any of the recommendations in the report?
Melanie Dawes: Certainly not the recommendations, but in the case of, particularly with the National Institute—
Q14 Chair: But they did change something? You are indicating that they went back and asked for some changes, yes?
Melanie Dawes: We had a number of concerns about—
Q15 Chair: Yes or no—you went back and asked for some changes?
Melanie Dawes: We challenged the fundamental basis of some of how the data was treated by the National Institute in their work. It is not about the report or what was written in the report. It actually started with some quite big challenges that we felt we needed to bring out about how the data was being done.
Q16 Chair: So you weren’t happy with the report that they produced?
Melanie Dawes: We looked at the interim findings. That is not so much the report as the actual analysis. If I could perhaps give you a little bit more detail—
Q17 Chair: They are kind of the same thing, aren’t they? It’s a report by an independent academic body.
Melanie Dawes: No, I don’t think they are. Firstly, was the analysis done well, bearing in mind it was complex?
Q18 Chair: If you are paying an organisation that has a reputation for doing this work and it doesn’t do the analysis well, that is a pretty serious breach of contract. Are you saying they did bad analysis?
Melanie Dawes: It was very complicated. We think that, in some ways, they did not deal with the data as well as they could have done. For example, for three local authorities the data was not included, and in some areas, where data did not exist, and there were a number of reasons why that was the case, they assumed—
Q19 Chair: Did you pay these people to do this job?
Melanie Dawes: Yes, we did pay them.
Q20 Chair: As a result of your unhappiness did you ask them for any money back from the fee?
Melanie Dawes: We had long conversations about the nature—
Q21 Chair: Yes or no?
Melanie Dawes: No, we haven’t asked for any money back. In the end, the evaluation has been published and the work has been done.
Q22 Chair: So you are saying that you commissioned an independent report. When it came out you weren’t happy about the data analysis they had done, and you went back and asked them to change it as a result?
Melanie Dawes: It wasn’t about asking them to change it; it was about working with them and bringing in experts to look at that. As I said, it was a very complicated piece of work. No one has done this kind of thing before.
Q23 Chair: We will get on to data measurement in the other thing. You only released this at 6.15 pm on Monday. That was less than 48 hours before this hearing. It seems a long gap between even 10 October, when the final report was published—even if we are generous and allow you that—and releasing them to the Committee. Why was it so late?
Melanie Dawes: That was the quickest I have ever seen, to be honest. The final report was available and proofread, which is done independently by the researchers—that’s the way the process works—and then we had a week in which to get them out. Normally, the Government protocol is 12 weeks. We did that in a week because of today’s hearing.
Q24 Chair: Sorry, but it went online. These were documents that were there and available online that you could have given to us in confidence in the form in which you had them on 10 October.
Melanie Dawes: We did publish them as quickly as we could in that remaining week and we gave them to the National Audit Office on Friday.
Chair: We are different from the National Audit Office; we are the Public Accounts Committee.
Melanie Dawes: I appreciate that.
Q25 Chair: What was stopping you sending them to us before Monday?
Melanie Dawes: In the end, the decision was taken by our Ministers that they wanted to publish them. That then took a week of work to get all of that very complex material and the five evaluation reports, plus DCLG’s own work—
Q26 Chair: Ms Dawes, I don’t want to have long exchanges; short answers please. You had the final reports on 10 October, yet you couldn’t get us a copy of those reports before seven days later, even in confidence and for preparation for today’s hearing, knowing this was coming up.
Melanie Dawes: Well, in the end we did it as quickly as we could. I appreciate we haven’t left very long before today.
Q27 Chair: What was stopping them just being sent to us on 10 October? We have email.
Melanie Dawes: Well, in the end we were about to publish the report. What my Ministers wanted to do was to get them out in the public domain, and that is where we focused our efforts.
Q28 Chair: You have said that repeatedly, but why didn’t you get them out on 10 October to us? I don’t know why they couldn’t have gone public as well. Why couldn’t you publish them on 10 October? What happened between the 10th and the 17th?
Melanie Dawes: All I can say is that we did it as quickly as we could.
Chair: You have repeated that several times.
Melanie Dawes: It was partly about making sure all the information was there and available. These things are not as straightforward as just pressing a button and putting them on a website. It did take a bit longer.
Q29 Chair: We sometimes receive information like that in its early iteration, and then it can be published publicly, but you didn’t consider that you could send it to us even in draft form. Well, you are saying it was not draft form; you said it was finalised.
Melanie Dawes: It was finalised, yes.
Q30 Chair: So you couldn’t have sent them to us before. Why not?
Melanie Dawes: I’m sorry that we weren’t able to do that, but we decided to publish them and to get them out as quickly as we could.
Q31 Chair: Did your Ministers have to sign it off between the 10th and the 17th? Did they have oversight of it?
Melanie Dawes: They didn’t sign off, in any sense, the contents of the report. That is not what happened.
Q32 Chair: But were Ministers involved in the last week?
Melanie Dawes: They certainly weren’t involved at any stage in the content of the evaluation. It was an independent report.
Q33 Chair: Did they go through the Minister’s box in the last week to be agreed to be released?
Melanie Dawes: Certainly, decisions on when to publish are for Ministers, but they didn’t get involved in any of the content of the report.
Q34 Chair: So it was Ministers who decided to publish on Monday the 17th?
Melanie Dawes: That was their decision, yes, although it was as quickly as I feel the Department could have managed to get the documents out.
Q35 Chair: Okay. We’re not going to get very far. May I just remind you that it was 765 pages 48 hours before a hearing? Although we are all assiduous, there is a point about reasonableness there. Your answers have been, I have to say, a bit evasive, so I wonder who you are protecting in all this. There have not been good headlines for you on these reports. Are you protecting your Department or your Minister?
Melanie Dawes: No, I don’t think so. I’m sure Louise will want to come in, particularly on the content of the programme and the evaluation. To be honest, this is a big, complex evaluation, and the debate in the last 48 hours in the media has not brought out exactly what the evaluation shows, so I hope we can manage to do that and explain some of that today. I think this is a good programme that achieved some really good results in the last Parliament. We have adapted it, and particularly adapted our approach to the evaluation, so we can make it stronger for the future.
Q36 Chair: You have seen the headlines. We want to get beyond headlines in this Committee—we are obviously keen to look at things in a bit more depth—but headlines are there for a reason: they reflect some of the things that are in the report. You are not saying that the news coverage hasn’t been reflective of some of the outcomes of this evaluation.
Dame Louise Casey: Can I step in at this point? The first thing to say is that you are absolutely right that the timing does not help the Committee at all, in terms of when you got the information.
Chair: It was also slightly suspicious timing.
Dame Louise Casey: Indeed. Neither does it help people like me and others who believe we have nothing to hide and nothing to be worried about here. I would just say that what has been in the headlines for the last few days is the one element that we had problems with in the Department—I was not responsible for the Troubled Families programme at the time, but I am now fully aware of it—which is that when the draft report first came through, the organisation had put through data that was inaccurate and flawed.
The process was—this was a while ago now—that they had to take the information in-house. That is why it has been delayed. I can understand your frustration and your line of questioning. I desire to be accountable here. The process was pretty challenging within the Department. Essentially, when they looked at the information—this is the analysts, not people like me or, indeed, Joe Tuke—they found things like they had put three local authorities’ flawed data through their system. They accepted it; the Department did not. They ended up having to go to the University of Cambridge to take this information to an entirely different person and say, “Is this okay? Is this not okay?”
One of the other things that has been difficult, certainly for me, is that a lot of the comment and extraordinarily negative publicity in the last few days has been based on that one report. Appearing before you today is pretty challenging because we’re playing catch-up with one element of a 700-page report that essentially says that the Troubled Families programme has no impact. I, for one, do not believe that is true, and I think I can prove that to you, as can colleagues from the Department.
Q37 Chair: I am just saying that it was not difficult to predict, for those of you who have been around the block a few times on these things, that if something like that is in a report, you are going to get bad headlines. Were you trying to suppress that element of the report, Ms Dawes?
Melanie Dawes: No, we weren’t trying to suppress it.
Q38 Chair: I want to finish in a minute, because we could go round and round on this. There were extraordinary hoops to go through to get the right analysts to do the work they are being paid to do. It seems extraordinary that the Department could not even commission the right people and the right research, which is what you are effectively saying to us.
Melanie Dawes: Perhaps we can come on—not now, but perhaps later—to explain quite how complex that work is. I do appreciate, though, that it is somewhat extraordinary, and it was disappointing that the work effectively had to be redone.
Q39 Chair: It was disappointing—we will leave it at that for now. The Comptroller and Auditor General wants to come in. Can you be brief, please?
Sir Amyas Morse: Did you check all the research work and calculations in the report? Did you effectively audit all the findings in the report yourselves? You obviously went through all the numbers to check whether they were correctly calculated—is that right?
Melanie Dawes: No, it was not a question of going through a spreadsheet and checking numbers. It was a question, though—particularly for the national data study, which is a very unusual piece of research, we always have a conversation with the researchers to make sure we have understood the analytical basis of the work. Quite quickly it became apparent that there were real issues and judgment calls that we thought ought to be made differently. In the end, we worked through that with an independent, external evaluator as well.
Q40 Chair: I still find it extraordinary. This is a report commissioned by the Department. When you commission things—we look at procurement all the time—surely you set parameters, you talk about what data might be used and you have some rules about what is acceptable and what data is verified. Did you not do that when you commissioned this work?
Melanie Dawes: That is one of the reasons we did raise concerns: we thought that some of the data that had been accepted into the analysis was not as good as it needed to be.
Q41 Chair: This was a long way down the line. Let me repeat, this was going to be published in early 2014, so surely before the report was even in train there would have been some discussion between someone in the Department—I do not care who it is—to make sure that everyone was agreed and sighted on what sort of focus the report would have and what data it would use and make sure that the methodology had been agreed and understood on all sides. But you are saying that did not happen.
Melanie Dawes: I do not think this sort of evaluation was ever something where you could say, “We are going to do this bit exactly this way,” because it has not been done before. What is going on here is that we have taken data from 57 local authority areas, brought them up to form a national dataset and then linked them with national administrative datasets. That is completely cutting-edge. I am not surprised that the researchers found it difficult.
Q42 Chair: To be clear, did the researchers come to you and say, “We think we’ve got a problem here and we’d like to talk to you about how we are going to do this” or did they present something to you and you said, “Oh no, that’s not good enough. Go back and we’ll help you rethink it”? Which version of events was it?
Melanie Dawes: It was a more collaborative process than that. I think it was about talking about the findings of the evaluation, which at that stage were interim findings. This is last summer, not in 2014—
Q43 Chair: So they came to you and said, “We are not sure about this. We want to talk to you about it.”
Melanie Dawes: I was not part of those conversations; they were led by our analytical teams and overseen by our chief analyst.
Q44 Chair: We could get into how you procure research, but I have been a Minister and I would have been appalled if a piece of research had come back so out of kilter as you are suggesting. It is important to make sure that the clear parameters are set at day one.
Melanie Dawes: It was not completely out of kilter, but there were some judgment calls in a complex field that needed to be challenged and in the end the vast majority of the work that was done was absolutely fine. This is not a completely flawed piece of analysis. It is a very complex piece of analysis with genuine data problems where there were genuinely good questions about how to deal with those problems that needed to be discussed.
Q45 Bridget Phillipson: It just comes across that you are bashing the report because it was inconvenient.
Melanie Dawes: I can completely appreciate that that is how it may look, which is why we sought to have Anna Vignoles come in and look at it and make sure that we were not in any way prejudicing the findings. We did not write the final findings; she actually looked not just at the data but at what we could conclude from the data.
The specific point that she drew out when she did her independent work was that because of the data limitations it was important not to say that the evaluation showed that there had been no impact. What was correct to say was that the evaluation could not prove that there had been an attributable impact from the programme. That is quite a nuanced point—I do not think it has come out in the press coverage in the last few days—but that was the critical point that she sought to draw out in the work she did.
Q46 Bridget Phillipson: Neither of those scenarios is great—to say that it has had either no impact or not a particularly good impact. Neither of those would demonstrate a great use of public money.
Melanie Dawes: What the evaluation does show is that the families in the programme did improve their outcomes. It does show that quite clearly, whether that is work or school attendance and so on. It also shows that there is a statistically significant improvement in how they feel about their lives, in particular feeling that the worst is behind them.
Q47 Chair: We don’t want to go into the detail.
Melanie Dawes: I think at some at level you have to bring out the detail a bit, if you will forgive me, with one more point. What we need to put in context is that this evaluation ran for only 12 to 18 months with individual bits of data. What proved not to be possible in the research was to form a control group that would have allowed you to say, “Here’s a group of families that did not have the Troubled Families intervention but are similar to the ones that did. Therefore, how can we measure that difference?” That was the thing that the research was not able to prove.
Q48 Bridget Phillipson: But the report does also say that they were unable to find consistent evidence that the programme had any significant or sustainable impact.
Melanie Dawes: That was attributable to the programme, although the outcomes did improve, and that is shown in the evaluation.
Q49 Chair: Ms Phillipson’s point rests. We are going to move on now from the actual publication of the report. I want to come to you, Dame Louise. Will you remind us how many years you headed the Troubled Families programme?
Dame Louise Casey: From 2011 until I went to Rotherham, which would be 2014, I think.
Q50 Chair: So it has been a big part of your life?
Dame Louise Casey: Very much so.
Q51 Chair: Does the media coverage and the evaluation disappoint you?
Dame Louise Casey: I am glad that is the starting point. I want to say that there is no way on God’s earth that it would be helpful to anybody to decide to suppress people, because obviously they have access to the press in a way that they have shown very clearly. Therefore, it is just not in our interests to do that—certainly not in my interests to do that. The frustration is that it is one part of a much bigger story. The other thing is that the way it has been presented in the media—if I am honest, quite deliberately—has not got the caveats across. Nowhere does the word “attributable” appear—
Q52 Chair: When you say “quite deliberately” who are you attributing that to? To the media or to the researchers?
Dame Louise Casey: No, not to the media. Sorry, I’ve got nothing to lose in a scenario like this. Lots of comment made by those closely involved with the evaluation, who have been leading on the press in the past few days, has been unedifying. They didn’t wait until the rest of the evaluation was out. I am sure they feel suppressed. That simply isn’t true.
I am the first to say about Jonathan Portes and NIESR and their research that, after a lot of correction and sorting out, I accept the findings of the research. That is that over the timescale that they looked at the families, which was really early on in the programme, the changes in those families, which they do not dispute, cannot be directly attributed to the Troubled Families programme. You can, on the other hand, find a lot of information as to why. They had not, frankly, Chair, put any of the caveats in the public domain.
Q53 Chair: So you are unhappy with how the people who were funded by the Department to do this evaluation have conducted themselves.
Dame Louise Casey: I am. I will be honest about it: I am. I don’t want to make it a personal thing because I accept that, within the strictures of this one piece of research, it doesn’t prove what I hoped it would prove. Did I ask the Department to sit on it? No, I didn’t. I think it is better to have that stuff out and washed out in the public domain so that you can have a discourse about it. My frustration, if I am honest, is that we haven’t had a chance to set the record straight, that you are not only—
Q54 Chair: Well, you are here now and we can do that. In terms of the evaluation report itself, you say you agree with some findings. Are you happy now that the methodology was correct and that it is a proper and fair evaluation?
Dame Louise Casey: I am happy with ECORYS’ overall evaluation report, that you couldn’t publish separate ones, as you will find out when you finally get time.
Q55 Chair: We have seen how they link in.
Dame Louise Casey: You can see how they link together and you can’t take any one element on its own. In fairness to ECORYS—to which NIESR is a sub-contractor—what they have quite cleverly tried to do is knit together what the overall learning of the story is. I accept that.
Within that, I wholly accept and understand why, at the beginning of this programme, when we compared these two things, they were not able to attribute changes in the families directly to the Troubled Families programme.
I am not sitting before you here today disputing that fact. I am sitting before you today disputing how that fact has been used and how it has been used in isolation of the rest of the evaluation. No one disputes the fact that 116,000-plus families had problems and now have fewer problems. Nobody is disputing that; the stated aims of the programme. Were you to have read some of the publicity in the past few days and, indeed, what has been put out by this organisation, you would think, “Oh, the whole programme is useless.”
Q56 Chair: We are going to get on later to issues around the control group and so on, but I want to go back a little bit in history, to give us an idea of where the buck stops. First, Melanie Dawes, who is responsible for this programme? Is it you? Is it Dame Louise? Who is the main person who should be answering questions from us on this?
Melanie Dawes: Well, I am the permanent secretary of the Department that is responsible for this programme.
Chair: So the buck stops with you.
Melanie Dawes: Yes, I think it does. Clearly, Louise was in charge of the programme in the last Parliament.
Q57 Chair: Okay. I want to be clear. Louise Casey, you would have been involved in the days before the announcement of the programme. When were the first discussions about doing something with a group of families that have been dubbed “troubled”?
Dame Louise Casey: Prior to my or anybody’s involvement, in December 2010 the then Prime Minister David Cameron made a speech in which he outlined a public commitment to deal with troubled families. After the speech, the number came out as 120,000 families, based on the estimate they had at that point, which we were not involved with but we understand why they used that. It became clear to advisers in Downing Street and the permanent secretary at the Cabinet Office, Jeremy Heywood, that the Prime Minister’s commitment to tackle troubled families, as he had called them in his speech, was not necessarily going to happen in the way they wanted.
Because of my personal experience doing things like family intervention and social policies for other Governments, they asked me to have a quick reconnoitre across Whitehall as to whether or not each Department was going to move on how they might meet the Prime Minister’s commitment. I reached a conclusion that they probably were not, depending on how we wanted to meet that commitment. I was appointed in October 2011. The Prime Minister then made another announcement in December 2011, where he said he wanted to turn around the lives of 120,000 families. I was involved in all of that process.
Q58 Chair: That is an incredibly helpful summary. It has often been publicised as something that happened after the riots, but you are telling us that there was a clear plan—or at least a plan, if not a clear plan—in place before that point.
Dame Louise Casey: In fairness, I had said to them in the summer during which the riots happened—at a very low level; I was not doing a major review—“There are different ways you could look at this. You could do it writ large. In other words, do you actually want to really transform the lives, spend money and have a massive Government programme? Or you could do a classic line to take, which is that there are lots of ways Government is helping all these families, such as through early help.” I said, “You’ve got two options here. Whitehall might try to get you to do the middle one. That will be a mistake.” It was left there. The riots happened, and then I think they decided around September—
Chair: “They” being No. 10?
Dame Louise Casey: The Prime Minister and the Quad at that point. They wanted to have a much greater focus on the Troubled Families programme.
Q59 Chair: So you had a bit of a perfect storm for any poor Government Department. The PM had made a pledge, and then the riots had happened, which made it imperative. There was this figure of 120,000, which you say the Department had no part in. It is a perfect political storm. Who then had to backfill a methodology to make sure the Prime Minister met this? There was not a programme in place when he made that announcement, was there?
Dame Louise Casey: No, there wasn’t a programme in place, but to be honest, I have now worked for an awful lot of Prime Ministers and Ministers, and they quite often make quite direct policy commitments and then it is the job of Whitehall to run behind them and make sure they commit them. I don’t think that is an unreasonable way sometimes for things to get done.
Q60 Chair: But he had made the pledge about 120,000 families being “turned around”. When you looked at how you were going to deliver, you knew there would be pressure for the next Prime Ministerial speech to point out how far people had been turned around. You had to develop criteria that allowed him to hit a numerical target and this slightly difficult-to-define target of “turned around”. Who was making decisions and setting the framework for that? Presumably it was you, Dame Louise.
Dame Louise Casey: A combination of people. Indeed, some of his Cabinet colleagues were closely involved in this. Can I explain? The 120,000 figure—which a number of critics, but not too many, have a problem with—was the starting point of some of the difficulties for people involved in criticising the programme. The fact of the matter is that no one—including us, in other lives and jobs—had ever actually tried to work out how you could tackle these particular sorts of families where you have a multiplicity of highly complex problems. In fairness to the Government, I would say the best proxy they had was the old Social Exclusion Unit data—the family survey data—which I think was then owned by the Department for Education, which gives you, more or less, this figure of 120,000 across a load of cohorts. I do not think it is unreasonable that when we were developing with them what they wanted to evidence by “turned around”, they took the 120,000 figure. We then went out to local authorities and said, “In terms of kids who aren’t in school, families that are committing antisocial behaviour, and worklessness, as well as another high-cost indicator should you need it, can you stack that number up?”—and they could. The criticism that has sometimes come in is that we were criminalising troubled families—
Q61 Chair: We haven’t used that word. You are digressing a bit.
Dame Louise Casey: Sorry; I am jumping ahead. I am digressing. I am in therapeutic mode, because it has been a rough few days.
Chair: The Public Accounts Committee is great therapy.
Dame Louise Casey: I have never thought of it that way before, Chairman.
Chair: You can tell us all.
Dame Louise Casey: Essentially, it was a reasonable position for them to take. If you look at people who are poor, deprived and in overcrowded conditions—if you are not in school, for example, that is one of the likely answers.
Q62 Chair: I am aware that you know an awful lot about this. As constituency MPs, we all do too. We see this. We really need to cut through narrative and focus on what happened in this whole programme.
Obviously, you have read what some of the evaluators have said. One of the evaluators said that the whole thing became “an exercise in self-justification”. There is a feeling that you had the number, you had the Prime Ministerial pledge—we all know, especially those of us who have been in government, how you have to run to make sure that a Prime Minister’s pledge is met, otherwise it is a big embarrassment for No. 10. The Prime Minister possibly doesn’t even realise the amount of work that is being done to make sure that pledge is met, even if it is artificially met. What do you say to that comment—“an exercise in self-justification”? It comes across from what you have just described that the numbers were there—the numerical driver was 120,000; the driver was this “turned around” evaluation.
Dame Louise Casey: I am quite comfortable—I was at the time and I am now—that we could find and we would find a cohort of at least 120,000 families in this country who would meet the stated objectives of the programme as laid out. I did not—
Q63 Chair: But what about the “turned around” bit?
Dame Louise Casey: I have worked for one Prime Minister who used the word “respect” when he talked about antisocial behaviour; I have worked for another Prime Minister who decided to call parents—
Q64 Chair: How did you decide what was “turned around”?
Dame Louise Casey: This Prime Minister used the term “turned around”. For him, that was an expression about turning around a family; I do not think he necessarily thought in terms of indices.
Q65 Chair: Exactly. So why did you then make sure that was included in evaluation in a very narrow way?
Dame Louise Casey: Because it was incredibly important to make sure that this time around, the programme had a real focus and had real outcomes attached to each one of these families. In the good days, we would put plenty of services into these families. The question is: did anything change? I was well up for an outcome-based programme—to use the lingo—and very keen to make sure we reduced crime and anti-social behaviour in those families. You know yourself that the silver bullet in social policy is getting children into school all day, every day and fed, if we can possibly make it, so of course I was comfortable with those two as key indicators. Again, we all know—whichever party you represent here today—that work is the most profound thing we can do for any household. I was not backfilling anything. Yes, there was a fight and a debate at Cabinet about how many indicators they wanted to put in—I know I am about to be told to shut up, even by the Permanent Secretary—but of course there was a trade-off.
Chair: I want to ask you a question, Dame Louise. That is why—
Dame Louise Casey: I am sorry; it is just annoying to see a programme so shot down.
Q66 Chair: I know you know a lot about this and you want to tell us about it. While I joke that it is therapy, it is actually a serious Committee trying to get serious answers. Can we just go back to the evaluation? You had this pledge of 120,000. You had the phrase “turned around”, which has become this quite mechanistic approach with quite tight criteria attached. You then had to draw up a cost calculator, because one of the other measures of success was the proof that intervention would save costs in the long run. What evaluation did you do to draw up that cost calculator? That is a huge political prize. I should think every Prime Minister anywhere would want to prove the value for money of this, but it is very difficult to do. What evaluation did you and your team do to work up that cost calculator model?
Melanie Dawes: Could I come in here and tell you my observation of this, having come in just under two years ago? Yes, the Department had to move fast to put this programme in place, and yes, the rest of Whitehall had to move fast to work with them, but the payment by results mechanism that was put in place is quite clear. Yes, people can call it mechanistic, and some have, but it did actually get local government and its local partner agencies moving around some clear outcomes for the first time in a cross-agency way. So, from my perspective as accounting officer, the payment by results gave us some clarity about whether or not the programme’s outcomes were being achieved, and whether we were paying money for something we could be sure was happening.
Q67 Chair: You had a nearly 100% success rate.
Melanie Dawes: In fact, we have never said that. Local authorities worked with more families than that. That is something that the evaluation report does draw out. We can be confident from the payment by results mechanism that we know we were paying for outcomes; the question is a wider one from the evaluation.
When it then comes to the cost-benefit calculator, that piece of work was done—again, most other programmes have not attempted to do this—to try to get at the true value for money, if you like, of whether this was really going to do what local government believed it was and save money.
Q69 Chair: So, what evaluation did you do to draw up the cost calculator? You have given me a lot of words there, but what evaluation did the Department do to draw up the cost calculator? How did you work out the methodology for it?
Melanie Dawes: Joe might want to come in on the details of this. We worked with local government to work out how we were going to capture reductions in things like police call-out rates, the need for people to attend family homes, and so on. Data systems were set up to try to capture some of that data. Over time that became slightly more sophisticated and we were able to put costs on it.
Q70 Chair: Do you think the cost calculator is now a good calculator of the cash savings?
Melanie Dawes: I think it is a really good first effort to do something we have not done before.
Q71 Chair: We are trying to have a discussion about honesty. So you are saying it is not perfect—
Melanie Dawes: It is not perfect, and in particular—
Q72 Chair: What do you think needs to be done to get it better?
Joe Tuke: When we go out to talk to local authorities, the reason they find the cost savings calculator really useful to them is that it is about, how do they drive the integration of services? How do they show the benefits that working in this way brings, not just for them but for their partner agencies? In order to do that though, you need to gain information, but that is a really difficult thing to do, not just because of things like data protection but also because systems don’t talk to each other. So an awful lot of work had to be done to develop that—
Q73 Chair: That is the mechanistic part. I am not saying that it is not difficult but how did you work out what should be measured? You were talking to local authorities. What needs to be done to make it better, because Ms Dawes has acknowledged that it is not perfect?
Joe Tuke: The big thing is being able to get the data from other agencies about the reductions in particular volumes of indicators that they are responsible for. So, for instance, how do you get the police to tell you the reduction in the number of police call-outs? How do you get them to give you information about the convictions for the cohort of troubled families you have? How do you find out from the schools about the reduction in truancy? How do you get the reduction in the number of ambulance call-outs or A&E episodes?
Q74 Chair: So where are you at now? What do you need to be doing to make it better?
Joe Tuke: At the start we worked really hard with seven local authorities. This isn’t something that is easy for local areas. We managed to produce a report showing some of the ways in which they had done it. The best local authority at the time was Wandsworth. We stole their person leading on it, brought him into the team and got him to work with all our other areas. We did about 30 tutorials around the country; we did a helpline. Eventually, we got about 67 local authorities that could use the cost savings calculator to a reasonable degree. So they could use it with a random sample of families.
Q75 Chair: So it is getting better all the time?
Joe Tuke: It is getting better all the time. In the new programme, we have designed it so that we pre-populate a lot of the information for them from the impact study.
Q76 Chair: I am going back to Dame Louise. I am going to ask some quick-fire questions if I can, because I need to move on to colleagues. Who decided on the success criteria? Did No. 10 give you a free rein, or did you decide?
Dame Louise Casey: No, the three emblematic payment by results of education, work and anti-social behaviour and crime were signed off by the Cabinet.
Q77 Chair: Signed off by the Cabinet, yes, but who decided in the Cabinet? Who decided to recommend it?
Dame Louise Casey: To be honest, there was a fairly robust debate over several meetings both of the quad and the Cabinet about what they wanted in it. I was very clear that I wanted education to be in there because I genuinely think that is—[Interruption.] Also, it was very clear that we wanted to do anti-social behaviour and crime, and we wanted to do worklessness because they are the ones that actually show change.
Q78 Chair: So, you are saying the decision was really clearly the quad and Cabinet?
Dame Louise Casey: Yes, very much so.
Chair: Can I ask some more quick-fire questions about how you, Dame Louise, fitted in the system? You were in this odd role, in a way, because in lots of the roles you have had in Government you have not been quite in a departmental structure in the same way as you are now as a director general. Who did you report to on a regular daily basis and periodically? The Prime Minister? No. 10?
Dame Louise Casey: I did not have to report to many people on a daily basis. I was based in the Department for Communities and Local Government. My reporting line was to Sir Bob Kerslake on civil service and to Jeremy Heywood in No. 10. Politically the programme ran into the Prime Minister of the day and the lead Cabinet member on it was Eric Pickles.
Q79 Chair: Who did you provide updates to? If someone said, “How is it going?” did they just chat to you in the corridor or did you have to go to a meeting with somebody and say, “This is what’s good, what’s going well, what’s not going well”?
Dame Louise Casey: I think it was mainly the Secretary of State. It would have been Eric Pickles.
Q80 Chair: It would have been the Secretary of State. That bit would have been through the Department. Did you report to any other Departments apart from the Cabinet Office and the DCLG?
Dame Louise Casey: No, not at all. There was a Social Justice Committee that used to take Troubled Families as one of the things they discussed, I think every quarter.
Q81 Chair: Was that at Cabinet level?
Dame Louise Casey: That was a Cabinet-level Social Justice Committee serviced by the Cabinet Office. We went to the quad, as was, on a fairly regular basis. It felt at the time like we saw a lot of people.
Melanie Dawes: Could I add that I was in the Cabinet Office at the time, as you may remember? What I observed is exactly as Louise just said. The primary responsibility was within the Department but there was a lot of interest in the programme.
Q82 Chair: Exactly. That is why I’m asking.
Melanie Dawes: It came to discussion in a number of different Cabinet Committees, including Cabinet itself every so often.
Q83 Chair: As far as going and talking to people about things, Dame Louise, you have demonstrated today that you are very good at coming to talk to people about things that you are passionate about. That is nothing other than a compliment. You talk passionately about something but, out of all those people you were reporting to, were any of them looking at the data and asking about the metrics and the calculator and those sort of detailed things?
Dame Louise Casey: Relentlessly. We had things like the implementation unit in the Cabinet Office looking at what we were doing. I need to be clear. The reporting line and the accounting officer for the Troubled Families programme was Bob Kerslake, Melanie’s predecessor. In terms of that, I would report to Bob. The money and everything ran through the Department for Communities and Local Government.
At a policy level, we were obviously trying to work across Departments. DCLG was not in charge of work. That was Iain Duncan Smith’s area. So we had a framework by which we had to talk to and enlist help from other Government Departments. One of the things we may get on to later is that half way through the programme we managed to score a deal with the Department for Work and Pensions through Iain Duncan Smith in order to get people out of Jobcentres and into our Troubled Families programme. So it was a very cross-departmental programme.
Q84 Chair: It was 500, wasn’t it, people from Jobcentres? I need to pass on to Chris Evans in a moment. There was an implementation taskforce set up in 2015 when I think Greg Clark was Secretary of State. That was under you, Joe Tuke.
Joe Tuke: That was, yes.
Q85 Chair: Why was that set up? How was that different from some of the things that went before?
Melanie Dawes: That was essentially a Cabinet Committee. Last year there were a number of those Committees on a number of different topics.
Q86 Chair: So it was a different Cabinet Committee from the one Dame Louise Casey talked about.
Melanie Dawes: It was designed to help our Department and Secretary of State to get the support that was needed across other Departments, and also for those other Departments to ensure that they understood what the programme was delivering. It was a Committee, which is quite common.
Q87 Chair: Back to Dame Louise, among all those things, which was the most important? I do not doubt that you have a passion for this subject area but, in the end, there is a political and day-to-day reality. What was more important, keeping No. 10 happy or getting long-term change for these families?
Dame Louise Casey: Without a shadow of a doubt, I would not have left being the Victims Commissioner, which was a difficult job but one I thought worthy, and done this unless I thought there was a cat in hell’s chance of actually trying to change the system for these families, as well as help the families. It was unfinished business for me. We had set up family intervention projects around 2006, 2007. We set up 53 of those projects throughout the United Kingdom. In every single social policy job I have done, systemically, we fail multiple-need families to a—
Q88 Chair: So you are convinced that, even if you had not met short-term targets, No. 10 would have been happy.
Dame Louise Casey: They would have to be happy. The bottom line here is that we wanted to help the most difficult families. I hope we come on to this at one point. Some of the coverage and misrepresentations—
Q89 Chair: We don’t want to go back over that please.
Dame Louise Casey: The idea that we were trying to go for low-hanging fruit and easy families just isn’t true. It has now been backed up in the research that has only been published this week, which shows whom we were getting to through the programme. No, it did not necessarily fit easily with everybody that we were reaching out to these extremely difficult families, but we did and—
Chair: We are slightly digressing from the point I was making, which was the long-term aims versus the short-term aims. However, I will pass straight over to Chris Evans and we may come back to that.
Q90 Chris Evans: I just want to start—120,000 families. I appreciate that you were not involved in setting that target. It was 400,000 families for phase two. Do you think, in all honesty, the bar has been set very, very high here, and you were always set up to fail anyway? I am trying to be helpful with that question.
Dame Louise Casey: If I did the 120,000—I mean, what is really interesting about this is that what we have to try to do in public policy terms is to work out both what is costing us the most money and who we are allowing to lead the most difficult lives. We did not have to work very hard once we had the 120,000 to—basically, we went out to local authorities and said, “Do you think you have families that fit this criteria?” They did not have to work very hard at it.
So, on the original 120,000, the short-term thing would have been to go for easy families. We didn’t; we went for families with significant levels. There were days, and that was why I was pleased that we were able to get the extended programme, where I did not think 120,000 was enough, and I thought that we were not helping enough families through that programme with the right outcomes.
I have also thought in retrospect—we wanted the whole system to change, to move from being just about children to being just about families, and we could not mandate that from the Troubled Families programme. We had to inspire people to do that, we had to entice people to do that and we had to work collaboratively with people to do it.
So, on the 120,000, we did not have to work very hard to find those families, and I stand by it. I think that Joe and colleagues have led the process on the 400,000, but again I do not think that over time we will find that too difficult—
Q91 Chris Evans: Looking at figure 2 on page 7 of the Report, I find that the definition is very vague. I come from the south Wales valleys, which would be somewhere where there is a lot of troubled families, and I would know a number of families who would qualify for this.
The other situation that I worry about as well is that, when you have the interventions, a measure of success is someone going to school regularly, or someone getting involved in the Work programme or being back in work for six months. Do you think this evaluation happened far too early in the programme, because this type of work needs a number of years for it to play out? Would you say that was fair criticism?
Dame Louise Casey: Yes.
Melanie Dawes: Perhaps I could answer that question. I do think it happened somewhat early and it did not run for long enough. It was a snapshot, run over 12 to 18 months, and that is one of the reasons why there were so many problems with missing data and so on. So one of the things we have done for the new programme is to use that same data infrastructure, if I can call it that slightly technical thing, that same infrastructure of a national-level dataset built from locally reported—local authority-reported—outcomes for families. We are now running that data every six months, so we are going to be able to look at that longer-term impact in the way the Chair was suggesting earlier is important—that more longitudinal look. So we have made an investment here, through the first programme, but we have broadened it and are going to be able to use it much more systematically for the future.
Q92 Chris Evans: Okay. Before I give the figures, Dame Louise, you have already spoken about your fight to get education involved. What I’m concerned with more than anything—this is looking from a purely policy point of view—is that once those children are back in school, how are we monitoring their development in school? For example, are we looking at whether they have been disruptive in class? What is their educational achievement like? Are they subject to some sort of soft exclusion—standing outside the door if they’ve been disruptive, or whatever? When they’re in school, how are they being monitored, because that seems to be a very difficult thing to monitor?
Dame Louise Casey: That is an incredibly difficult thing to monitor. Did we attempt to do that in the first programme? No, we didn’t. That is not part of what the evaluation was, so I think that is a fair thing to say. As you go deeper and deeper into social policy, you could look at all sorts of things, of which that would be one. I think that is fair enough.
I suppose the starting point was that it was very clear that we really didn’t have to work very hard to find families where children were not in school at all, where truancy and exclusion were really high. So the starting point was, “Let’s deal with that first.” We asked them to deal with that alongside issues such as domestic violence, drug and alcohol abuse, unemployment, depression—all of those things.
So the answer to your question is: no, we didn’t, partly because we were prioritising the starting point. We were prioritising where we were starting from in a programme that was pioneering and ambitious on an incredible level, because we were not just an education programme, just an unemployment programme, just an ASB programme or indeed a programme that just looked at those sorts of families. We were trying to get to families that had multiple needs and were multiply complicated. In a way, that was our starting point. Getting the system to do that was sometimes really difficult indeed.
Q93 Chris Evans: I want to turn the question on its head. According to figure 8 on page 16, of the 1,048 families for which full data were available, 40% had three or more children, compared with 16% nationally; 49% were lone-parent households, compared with 16% nationally; and one in five had been at risk of eviction in the previous six months. Do you think we are intervening too late? Do you think they should have been identified much earlier?
Dame Louise Casey: Chris, as the sun comes up and goes down, we always intervene too late.
Chair: Please call the Member Mr Evans.
Dame Louise Casey: Mr Evans, as the sun comes up and goes down, we are intervening too late. That is the difficulty with all of this policy. At which point do you move into the system and work out where you are able to get the most traction and the most changes? Having said all that, when you look at the impact of the programme against the families with those multiple problems, having a 77% reduction in police call-outs, a 61% reduction in the risk of eviction and—dare I say it?—a trickier and more controversial one, a 46% reduction in under-18 conceptions, you can see that with those families you cannot give up on them. That would be really wrong.
Q94 Chair: Sorry, you just rattled off a number of figures. Where were you quoting those from?
Dame Louise Casey: Those figures are from the new ECORYS evaluate—the long evaluation. They are in the synthesis report. In a way, that is what is quite interesting because—
Chair: We have a lot of documents.
Q95 Chris Evans: In the evaluation, is there any further breakdown? I noticed that 71% of families have a health problem. Is there anything on disabilities that could be shared with the Committee?
Joe Tuke: In the new programme we will have a much better idea of that, because we are looking at a broader range of issues and collecting more information. We have information on a number of health problems—for instance, around people’s mental health issues. We know that over half of the households in the cohort and in the new Troubled Families programme so far have reported mental health problems. We will have a much richer amount of data to play back to local authorities, as Melanie said, every six months, and will report that publicly as well.
Chris Evans: Before I move on to Phase Two—sorry, did you want to come in?
Melanie Dawes: Sorry to interrupt. I think you were asking whether we should have intervened earlier. One of the things that local authorities certainly said about the first programme was how many more families there were who they felt needed help and who might not be quite at the stage that was needed for the first programme. That is one of the reasons why we broadened the criteria for the second programme. The overall case load is expected to be a lot bigger because we are dealing with a broader range of families slightly earlier in the process.
Q96 Chris Evans: Is this programme dependent on how well the caseworker does their job? It seems to be very much dependent on the relationship between the caseworker and the troubled families. As we have seen in the probation service, if the probation officer is overworked there is a tendency for things to go wrong. Does the programme win or lose based on that relationship?
Joe Tuke: That is the most important relationship: that between the key worker and the family. What is different about this is that it is not just a probation officer who is working with an individual member of the family who perhaps has an offending history. Here we have got a dedicated family key worker who deals with all the family and can see the interconnectedness of the problems and can see that the offending may have a link to the domestic violence that the parent is experiencing, or may have a link to mental health problems or whatever it might be. It is that key worker who has the most important relationship. There is quite a lot in the evaluation.
I am sure that Members talk to their own local authorities and get to meet some of these key workers; there are some great examples of public servants who have got a great attitude of persistent, assertive working with families who have shunned services in the past. They create a trusting relationship with the family—a sort of “tough love” approach. The evaluation shows how that practice, which we encouraged through the first programme, really showed improvements over the course of the three years of that programme. I think we are seeing that again. We put out there, in the first programme, about these are the key characteristics—
Q97 Chair: You talk about some very good public servants, but are you sure, and do you do any evaluation, or do you require local authorities to look at the quality of key workers, casework leaders, in those situations?
Joe Tuke: We do not mandate the approach to them. We encourage that, but we pass and share good practice across the piece, and we know that, as they do in social care, they will have supervision in place around that. Ofsted, of course, comes in and looks at children’s services departments.
Q98 Chair: Ofsted would evaluate these examples?
Joe Tuke: Ofsted will come in and look at children’s services departments, which would include early help, which generally is where family workers are.
Q99 Chair: I am sure Chris Evans is going to pick up on this, but it seems like there is a different set of professionals doing the same role, so they would be monitored and evaluated differently.
Melanie Dawes: One of the things that the evaluation brings out is that one of the positive impacts of the programme is that there has been an investment in a new workforce here. There were some family workers there before, but this has been a very big increase. As for many other areas of local government services, we do not specify; we inspect. We were relying here on the payment by results outcomes framework to make sure they were getting results.
Q100 Chair: Before I bring Chris Evans back in, the Comptroller and Auditor General wants to come in.
Sir Amyas Morse: I just wanted to ask something, looking forward. These reports could not show a link through the national data sets between the programme and the improvements that they had observed in troubled families. I am trying to say that carefully, to try to get it right.
Melanie Dawes: That is correct.
Sir Amyas Morse: Thank you. Do you expect that it will be possible to demonstrate such a link in future? It is quite important to get that straight. If you cannot answer it now I sense you are going to need to answer it at some point quite soon; otherwise we could be off on a set of inappropriate expectations again.
Melanie Dawes: I think for me, to be honest, that is the six million dollar question. We have—and perhaps we will get into this—so many difficulties with the data, and with this question of what is the counterfactual that you are assuming would otherwise have happened with these families, that it is always going to be very hard to prove beyond statistical doubt, which is what the evaluation was seeking to do, that it is the programme itself that has led to the improvement. Personally, I think we do need to be realistic about whether this quite complex national data set will show that, or when it will show it, but we are committed to carrying on trying. I think one of the big legacies of this programme, which the LGA have brought out in their submission to you, is the investment that has been made in data, which is unprecedented—joined-up local service data at a local and national level.
Dame Louise Casey: In terms of your question, Mr Evans—about family intervention workers and whether you can guarantee that if those workers are superb they can help those families transform—first of all, one of the learnings for me would be that I think we need to institutionalise what we think family intervention actually is at a much more formal level. It has been my belief ever since I went to the Dundee homeless families project in Scotland, when I worked at Shelter, so at least 25 years ago, that if you look at what those workers do and you try to codify and explain it, you would have—we have a legion of them now, but we have done it through inspiration. We wrote something called “Working with Troubled Families” where we said there are five factors of family intervention. I can list them now, but I won’t. They will be in endless bits of paper.
They essentially said it is all about the relationship between the worker and the family. They cannot just deal with one individual in that family; they have to deal with them all. They are dedicated to those families. They do odd things; rather than assessing they might order a skip, so you get the rubbish cleared out of the house. It is a very interesting way, I think, of working with families that we started in the Respect action plan with the family intervention project, and that we have been trying to promote through this programme.
Chair: And which organisations such as Family Action, based in Hackney, have done for a while.
Dame Louise Casey: Family Action in your constituency, Action for Children in Ms Flint’s constituency, there is another version in Bristol—it is quite interesting.
Chair: We are in danger of getting—we are not a policy committee, so I am sure our sister Committee will be interested in this.
Dame Louise Casey: But it is value for money writ large, Chairman.
Chair: Fantastic. We will take that on board. I think you have made the point.
Q101 Chris Evans: This is my last question before I move on to Phase Two. Figure 10 shows the cost and saving per family. These figures are presented as gross, and there is the claim that the Government have estimated that they will have saved £1.2 billion by March 2015. There was an assumption—the average gross saving was £11,200 across seven exemplar authorities. Can you explain how the seven exemplar authorities were chosen? Do you think the sample was too small—I know it has gone higher now? Why were they not just presented as nets? That is quite a simple question really.
Melanie Dawes: I will ask Joe to comment on this perhaps. This is the same work that Joe was describing earlier that we did with local authorities and that was so new for them. That was partly why we had to build it up, with those that were able to, in a more incremental way. I want to emphasise that these figures are indeed gross. They are not net figures and we are not claiming that they are net savings. We are simply presenting them as they are.
In order to say that they are net, we would need to be much clearer than we are able to be about the counterfactual. That point actually lies behind a lot of the issues we had with the evaluation and it is intrinsic to the nature of what we are trying to do here. I do not think it is a failure of the evaluation; it just makes it difficult actually being able to say what would otherwise have happened to families. That is why we cannot say for sure that this a net saving to the Exchequer. If we had them in the room, local authorities would say that they thought there were net savings, but it is very hard for us to prove that.
Q102 Chris Evans: So you are saying that the figures should never have been presented anyway, because they are meaningless?
Melanie Dawes: We would rather give you the data that we have, but I absolutely do not want to over-claim for it; I want to be absolutely clear about that.
Q103 Chris Evans: We are all politicians around this table, and if there is an estimated net saving of £1.2 billion, the Government are going to be okay with that, so really that figure is just worthless.
Melanie Dawes: It is not worthless; it is what it is. Some of the data that lies behind it is in this table, and some is in the cost-benefit report that we published on Monday. As that report says in its summary—as I just described—it is a gross estimate drawn from local authorities that are working with this data for the first time. It is quite new. We think it is valuable, but I do not want to over-claim for it.
Q104 Chris Evans: It is not the first time that has been used. Even the then Prime Minister was saying that you had helped 116,654 families and that therefore the programme was a success. Where did you arrive at that? I am looking at paragraph 1.21, which says, “The data showed that a total of 116,654 families eligible for the programme had met the criterion” of reducing crime, and the Prime Minister hailed this as a success. That seems a massive figure. You have helped 116,654 out of the 117,910 families you identified. Where does that figure come from? To be honest, if that is what you have turned around, you should be running every single Government Department.
Dame Louise Casey: Let us be clear that the figure you are referring to is the figure for the number of families who met the stated objectives of the programme, and for which the local authority received not only an attachment fee—to use payment-by-results language—but the outcome fee. It was suggested in the financial framework that local authorities reached more families, although we would not pay them for them, in order both to reach that and as part of a system change. I don’t think they did masses more, but our sense around the country is that people did more in order to meet that.
We need to be clear: we know that 116,654 families basically had their lives changed by the programme according to the payment-by-results system. That is a 99% success rate. You could ask, given where we are, why it was not a full 100%, because they could actually reach out to more families in order to meet that. But I would have probably lived with that as the person responsible for the programme.
If I may just comment on the money at the moment; I was responsible for the Troubled Families programme at the point we published that report. This programme was managed a lot by hearts and minds. We wanted the system to change. I wanted people to feel part of something that was pioneering—doing things differently—and was about implementation.
I was meeting families on a daily basis. I did two or three visits a week, and I would meet families that I knew had police call-outs up to their eyeballs. Frankly, some of their neighbours would talk to me on the way in and say, “Are you going to bloody get rid of them?” And yet, during the course of the programme, it was clear that their children were in school, they weren’t down the doctors, they weren’t doing police call-outs. Somewhere in there, there has to be a cost saving. Scientifically, you don’t have counterfactuals, but it was part of saying, “Come on, everyone. We can do this.” Part of it is saying, “There must be cost savings in this.” It is also about saying to the public sector, “You have to shine a light on your money.” You can’t just say, “Let’s spend money on families.” You have to say, “So what are you saving?”
I think it is the most difficult part of the whole evaluation, and as the ECORYS report shows, we have been unable in the final furlong of this particular evaluation of the first Troubled Families programme to sit before this Committee with the type of information we would want. I think that will be very difficult for the future, because I don’t see a country where I put some people with absolutely nothing in a corner, where nobody touches them at all and they are not involved in any—
Chair: Okay. We are not suggesting—
Dame Louise Casey: It is quite important that that is a lesson for the future—we cannot work our way through that easily.
Chair: We are going to talk further about evaluation.
Q105 Chris Evans: I have one short question before we move on to Phase Two. Paragraph 1.19 of the Report says: “The Department checked 2,367 claims in total and found 115 of these claims to be invalid.” There was a further review process that led to a further 214 invalid claims being identified. This is quite simple. What do you mean by invalid claims?
Joe Tuke: That could be something like an administrative error. If you are claiming for a result, you have to make sure, for instance, that all the children in the family are attending school above a certain threshold. It may be that when we looked at that claim, three of the children were and one of them was just below that threshold, and it was a mistake in the claim they had made. We might then say, “Withdraw that claim.”
Chris Evans: So just to clear things up, there are no instances of fraud or anything like that.
Joe Tuke: No instances of fraud.
Q106 Bridget Phillipson: To go back to figure 10 about the cost saving and the way the data have been used, Ms Dawes, you seem quite relaxed about extrapolating from seven local authorities much wider savings, yet your attitude towards the independent evaluation and the data issues was perhaps a bit harder.
Melanie Dawes: I wouldn’t want to overstate the issues we had with the National Institute element of the research. It is really important for me to emphasise that. That piece of the research was cutting edge. It was experimental. It was, by definition, a statistical research programme, and those programmes have to run to certain standards, so it was looking for quite hard-edged results. To do that kind of work with the data uncertainty that this programme inevitably was subject to, given that it was so new even to be collecting the data in the first place, means that the job of the researchers was really difficult. That was the issue we had about how you then interpret the results. What I am saying about our own piece of analysis is that I don’t want to claim too much from it. It is what it is. I think it is useful, but we are not resting big numbers on it.
Q107 Bridget Phillipson: The Government did claim that £1.2 billion was saved as a result, on the basis of data from seven local authorities. On the one hand, the Government were quite relaxed when the data backed up a pre-existing belief, but not so relaxed when independent research called into question some assumptions about the effectiveness of the programme.
Melanie Dawes: We are certainly not claiming any definitive cost savings from the programme, because we have not been able to do the counterfactual analysis that would put those figures—
Chair: I think Ministers have.
Q108 Bridget Phillipson: The Report we have says: “The government used this data to claim that the programme had already saved taxpayers an estimated £1.2 billion by March 2015. This assumed that the average gross saving…in the seven ‘exemplar’ local authorities would be the same” and so on. The Government have claimed that they saved more than £1 billion.
Joe Tuke: That Report was early on in the programme. It makes it really clear that these are gross savings, and there are whole annexes about the caveats, saying that that is the case and we were not able to have a proper counterfactual for it. So the Report itself is very clear that these are gross savings. The extrapolation was done from the seven areas as an indicative amount.
The big thing about that piece of work was, as Dame Louise was trying to suggest, that we were trying to encourage local authorities to go through a more rigorous process of identifying the costs and benefits of their programme, partly so that they could engage partners in this and we could get more integrated services. It was really hard—we were only able to get seven authorities, at that very early stage in the programme, to complete the cost savings calculated to a reasonable standard.
Since then, we have done more work, as we have talked about, and the report that we published on Monday does that for 67 local authorities and still talks about gross savings because of the difficulties with the counterfactual.
Q109 Bridget Phillipson: Doesn’t making grandiose claims about the effectiveness of programmes undermine the whole concept of some of this work? We all want to see families turned around and help and support given where there are difficulties in people’s lives, but if Ministers go out there claiming that it saves £1.2 billion and clearly that cannot be evidenced, that undermines the whole basis of that social policy intervention.
Melanie Dawes: I am confident that the department’s report was as Joe described—the caveats were in the report. Then there is just the question of how those results are expressed. But the results in the report were clear, and I think the caveats were there.
Q110 Chris Evans: Mr Tuke, I want to move on to you and focus on Phase Two of the programme. When you took over this programme, what was your impression of it? What had been done well and what had gone wrong? I want you to be totally honest.
Joe Tuke: I should explain first that I was working on the first Troubled Families programme with Louise as well, so I cannot claim I came in new to this. When the new Troubled Families programme was announced, we had time to go and talk to local authorities about their experiences of it and how they would like to improve or expand on it if we had a second phase. The big thing they wanted to do was to get more overtly to a broader set of problems. So although we know that in the first programme families had on average seven significant problems, the actual eligibility criteria were more restrictive, as we have learnt.
In this case, we are saying that actually the big problems are around violence and health and getting to younger children, because in the first programme mainly it was families with school-age children because of the criteria. So it was getting to those children. And, more overtly, getting to those children who were considered to be needing help in some way: perhaps children in need etc.
So it was a broader set of problems that we were able to look at, and that gets us into a bigger cohort as well. I think the most important thing was allowing local areas to identify the families they were really confident were of most concern and cost them most in their local area.
Dame Louise Casey: Can I add something to that? The other thing that I felt we wanted the next programme to include was longer follow-up. Part of the problem is that once people claimed the results, we knew that most of them were continuing to work with those families. We have individual examples of that around the country. In the second programme—the roll-out programme or expanded programme—they are going to track those families for a much longer period of time. That is the right thing to do, getting to the under-fives, getting to domestic violence and tracking families longer—
Chair: You have anticipated some of our questioning on that.
Q111 Chris Evans: Is it a learning outcome of Phase One that there has been too much of a drive towards short-termism rather than long-termism and that really what will show whether the programme fails or succeeds is what is happening in 10 years’ time rather than in 10 months’ time?
Dame Louise Casey: I personally feel that that is an unfair characterisation. It would be unfair to say that we were chasing short-termism over long-termism. If we were doing that, we would have concentrated on the PBR and we would not have gone for wholesale system reform, we would not have pushed reform of family intervention and we would not have done lots of things that we did during the course of that programme.
Q112 Chair: Dame Louise, it is not unfair for Mr Evans to raise that when all the measurements of a turned-around family are all short-term measures. That is what I think Mr Evans was trying to say. You cannot disagree with that.
Dame Louise Casey: Indeed, I do not dispute that. I would just caution that in order to meet the criterion on education a child had to be in school for a full school year. When we use the phrase “short-termism”, people think weeks and months. In order to claim that you have succeeded with that child, it is over a year. I am just being honest and saying I wish we could track them for longer, that’s all.
Q113 Chris Evans: That is the point I was raising. If you look at the target criteria, they are being in a work programme; being in work for six months and being in school for an entire school year. If we look at examples such as my earlier question on educational attainment, how many GCSEs does a student pass at the age of 16 when you have intervened at the age of six? Those are the types of things I was talking about. Are those learning outcomes from Phase One?
Joe Tuke: They certainly are. We will be getting a much broader set of outcome information. Both at local level, the plans which key workers have with their families are across a broader range, so they have outcomes that they set against each of those six problems that the programme is tackling. We are tracking that at national level through our evaluation as well.
What is interesting about the first programme is that, although the outcomes we were measuring have a more limited timescale through the payment by results, when you are out and talking to local areas, for them, yes, there is a time when they can claim a payment, but that is an artificial point in their work with a family. They carry on working with those families if they need to and then they step them down. The evaluation talks about excellent stepping down and exit arrangements to ensure that the support is sustained—sometimes by voluntary organisations, et cetera.
One of the districts, Warrington, for instance, has tracked its families who claimed payment by results from the first programme. Only 3% have had to be referred back into services for some significant levels of support.
Although the outcomes set in the first programme may, in your terms, Mr Evans, be more short term, I think the actual practice was that they continued to be supported and the improvements sustained. We will be measuring that more in the new programme.
Q114 Chris Evans: So there is an exit from the programme at some point?
Joe Tuke: Certainly, local authorities want to be able to step people down to a lower level; they want to reduce dependence on the service. That is the whole point.
Q115 Chris Evans: My question on all this section of the report—I remember when I was a candidate in Cheltenham in 2005 and knocking doors and people were going on about—
Chair: No anecdotal evidence.
Chris Evans: It is not quite anecdotal. When I was knocking on the doors and people were talking about antisocial behaviour, I was talking about Tony Blair’s respect agenda. Then I remember a speech by Gordon Brown when he was talking about individual intervention. This is not something new; it is not a new concept, as Dame Louise said earlier. Is there not a concern that you are giving money to local authorities to do something that they are already doing anyway? Is there a danger there?
Melanie Dawes: There is a perfectly fair question here about what some people call “deadweight”. Was all this going on already? I think the evaluation shows that what was really different and what was achieved through Troubled Families was a very different way of working. It may well be that some of the agencies were working with the families already, but they were not doing that in a joined-up way that was around the family rather than the individuals in the family. The bit of the evaluation that looks at process is very clear that this is quite a transformational new set of services that is now going on in local government with its partner agencies.
The other thing that is very new that was not happening before is the focus on outcomes. Again, I think it is a very question for us as to whether a PBR framework focuses you too much on the short term.
Chair: PBR is payment by results.
Melanie Dawes: Yes, whether payment by results focuses you too much on the short term. In the end, the programme is trying to strike a balance. If you do not have any kind of outcome focus, it is really hard to galvanise this new way of working. Equally, you always have to make sure that it is not too narrow.
Q116 Chair: We should just reflect that we received evidence from the Local Government Association and others that talked about joined-up working. We also received evidence that suggested that it was not universal and that the work was a bit patchy in places.
Melanie Dawes: Yes, I think you see again the ECORYS synthesis report brings out that some local authorities are great at this—
Chair: We will perhaps touch on this a bit more in good practice sharing.
Melanie Dawes: Yes.
Q117 Chris Evans: That question was generated from looking at sections 1.6 and 1.7 of the report. The Troubled Families programme was based on the Dundee Families Project, as Dame Louise mentioned; 53 family intervention projects were set up in England from 2007, I assume as part of the Labour Government’s Respect agenda; and the community budget programme was from the coalition Government. So all this work has been going on for a very long time. That is where that question comes from. What makes this different?
Dame Louise Casey: As, unfortunately, the person who has been involved with all the ones that you just mentioned prior to this one, I think that the difference with this one is that it is part of every single local authority in England, so it is not about specific projects in specific areas. Yes, we set up 53 family intervention projects under the Respect campaign. They were lovely projects but they were boutique projects, and when they lost favour, they more or less disappeared.
People like me and Joe—because we had lived through that era—were determined to ensure that this programme was much more about changing the mainstream system, and trying to come at things such as family intervention and whole family in a very different way.
You are right to ask, Mr Evans, whether—even having delivered the Troubled Families programme and being pretty happy with the fact that we changed the lives of 116,000 families—I think we have completely sorted out the way you might deal with these families. No, I do not. Do I think we got every single family that you referred to in your constituency? No, I do not.
This is a learning process. It is really difficult when you are running social policy and a financial framework, when you are spending taxpayers’ money, which is entirely different from spending your own, and when you have to work through a system in a different way.
Q118 Chris Evans: The crux of the whole thing is, are you really saying that a family is turned around simply because they manage to qualify for a payment?
Melanie Dawes: What we are saying is that those outcomes were achieved in order for the payment to be made. That is very clear. From the Department’s perspective, it is a question of what you mean by “turned around”. I point to one element of the evaluation, which is in the overall synthesis report—
Chair: I am just going to bring Caroline Flint in on this.
Q119 Caroline Flint: Would you find it helpful if the phrase “turn around” was dropped?
Melanie Dawes: It is a question of the overall presentation of the programme.
Dame Louise Casey: As the person no longer responsible, I think it was right for the time, Ms Flint. As the programme goes forward, it will be better, particularly because the new programme has a much broader range of indices where you are measuring—
Q120 Chair: Are you saying yes or no, Dame Louise?
Dame Louise Casey: I am saying, yes. It would probably be helpful to think about what you called it next. It worked at the time. I would not use it in the future.
Chair: Melanie Dawes?
Melanie Dawes: I just want to be a bureaucrat and give you some statistics, if you don’t mind.
Chair: I’ll indulge you.
Melanie Dawes: One of the things that I find most striking about the evaluation is that families report a 17 percentage point improvement in answer to the question that they are confident that their worst problems are behind them. That is quite interesting. I am not saying that means “turned around”. I am just saying that families—
Q121 Caroline Flint: The phrase “turned around” has a certain connotation. We need to be careful that, despite very good work that is happening on the ground, it does not lead to complacency about what families will still need.
Chair: We are going to come back to Ms Flint in a moment, but her point about overselling and apparently under-delivering is pertinent. Chris Evans, you have one last point and then we will move on.
Chris Evans: I have three more questions.
Chair: Quickly, then. At Welsh speed, please.
Q122 Chris Evans: Phase Two of the programme will run from 2015 until 2020, and will cover 400,000 families. Was that figure decided on beforehand? Was any evaluation done of it, and if so, what was the point of evaluating something that you had already decided on anyway?
Joe Tuke: The figure of 400,000 came from looking at the broader set of problems that I mentioned earlier, which we sought to tackle in the new programme. We then looked at the available datasets from which we could try to calculate the number of families. Part of the problem is that there are not administrative datasets that look at family problems, so you have to use survey evidence. We used the best available survey evidence—four or five different household survey types—to triangulate those problems and come up with an estimate. Our analysts did that work and came up with 400,000. A little bit later, we double-checked that against some local areas—small geographical points— which reinforced our calculations. We are reasonably confident that about 400,000 of those families have those combinations of problems.
Q123 Chris Evans: Figure 4 shows that a local authority attracts £4,000 a year when a family is recruited according to the payment-by-results regime. In a programme like this, do you have a genuine fear, as I do, that this could create a perverse result, in that people are so desperate to identify these families to gain the money that they will report on them? Are you alive to that risk, and what checks and balances do you have in place to avoid that happening?
Joe Tuke: My experience is that there is no evidence whatever of any local authorities or anybody else gaming it or going into it for the sake of trying to get funding they are not entitled to. The funding in the first programme was for families that moved from a position where they had problems to one where they no longer have the same problems. We haven’t seen that at all.
In the new programme—you talked about the 400,000 families—although we kept the payment-by-results model, the amount per family has reduced. It is now £1,800 per family, which reflects the broader range of needs that you see within that cohort.
Melanie Dawes: Can I just answer your question? Yes, we were alive to the risk. It is a perfectly legitimate question for us. As Joe said, we managed it by making sure we did not overpay, so the payment-by-results payments in the first programme, and even more so in the second programme, are not very high as a proportion of the costs that are actually incurred in delivering the programme. We also sought to check in the evaluation that the families are genuinely in need. In fact, the evaluation shows that they are even more in need than the characteristics we were looking for. But I think that is a risk we need to be alive to.
What local authorities say is that, although the payment-by-results payments were a good and helpful thing, the real incentive is that they know that if they invest in prevention through the programme, they are more likely to reduce costs in the longer term. In the context of tight budgets, that is a particular reason why they engaged so productively with this programme.
Q124 Chris Evans: I promised myself that I would try not to mention Wales this afternoon, but unfortunately I have failed. I have been looking at the Families First programme in Wales. That programme is a little different, in that the money is given up front, which means that local authorities are not looking for families. Have you studied that project? Do you think, off the back of that, that the Troubled Families programme is financially robust enough for the future challenges? That is my final question.
Joe Tuke: I can’t say I have studied that in detail. I am aware of the Families First programme in Wales. It has similar aims, in terms of working with whole families. There is a debate about the new programme that we are alive to, which is whether or not payment-by-results will continue to be the best method of incentivising the activity we want. We are already exploring that with some areas of the country to see whether or not there is a different financial model.
It has been really useful today to gain a focus on outcomes, because that focus on outcomes was not there. When we started with this programme, Louise and I were going out to areas, and there was not always a notion of why people were providing a service to a family and what the end was that they were trying to achieve. In some cases, dependency was being maintained on services; people were not saying, “Why are we all investing money with this family? What is the end we want?” Payment-by-results really did help to focus that. If you talk to key workers around the country, at the start they were probably a bit alarmed by this, but over time they got used to it and it gave them a really helpful focus.
Q125 Philip Boswell: I have just three questions: defence and criticism, compare and contrast, and the funding model, which I am concerned about. The programme, of course, has its defenders, such as the Communities Minister, Lord Bourne, although he talks more about his belief, rather than any evidence-based analysis. One of the critics, who was mentioned earlier, is the author of the NIESR report, Jonathan Portes, who wrote on his blog that the programme is, “a perfect case study of how the manipulation and misrepresentation of statistics by politicians and civil servants—from the Prime Minister downwards—led directly to bad policy and, frankly, to the wasting of hundreds of millions of pounds of taxpayers’ money.”
Louise Casey, you said earlier that you agree with some of the report and some of what Mr Portes said. I understand it is a difficult time at the minute. How much of that statement would you agree with?
Melanie Dawes: Perhaps I can answer that as the Permanent Secretary of the Department.
I can’t answer for Jonathan Portes and I don’t think that he would say that all those comments are about the evaluation—some of that is expressing a personal opinion—but it isn’t something that I agree with. I think for me the question of value for money is central to my view of the programme, and that starts with whether the payment-by-results mechanism showed that we achieved the outcomes for the payments we made. The answer is yes, it did. Those payments were audited at local level by local authorities and then we did spot checks to ensure that the system was working properly.
The second value-for-money question is about whether or not savings were achieved, and we have already spoken about that. We have some estimates of gross savings; we can’t say for sure that they are net savings. I would add that no other programme has been able to show that either in this kind of field. It is a very difficult question to ask. We have attempted to ask it and we continue to ask it with the new programme.
Q126 Chair: You talk about it being a personal opinion, but this is someone who did quite detailed evaluation and got access to detail that possibly the NAO get, but we don’t see. He came up with this as a personal conclusion, if that is what it is. Is it not a bit devastating for you?
Melanie Dawes: I don’t think we can answer for what everybody’s saying about the programme, but what we can answer for is the Government’s position and what the evaluation shows—
Q127 Chair: Are you disappointed? I asked Dame Louise at the beginning whether she was disappointed.
Dame Louise Casey: I am disappointed that an individual seeks to undermine the programme in the way described—an individual who has had access to the programme through one of the six reports as part of the evaluation. If I am completely blunt and honest—I always am in this building—I actually feel that in the last couple of days, they have misrepresented their own research by not putting the caveats in the public domain or being very clear about what we cannot prove, of which this is part. Nobody disputes all of these changes, but in the first six to 12 months of looking at these families, we can’t attribute those changes to this particular programme. Nobody in the overall evaluation disputes whether these families were changed or whether system reform was changed or whether police call-outs to these families were reduced by 77%. I know I am not supposed to make this personal, but there is a part of me that sometimes wonders. I wish we had got all of this out earlier, because then you could have read all of it and every newspaper in the country could have read all of it and I would not be sat here being asked about a blog, which at points feels rather personal. That is all I’ll say.
Chair: Thank you. I think you’ve made your point.
Q128 Philip Boswell: Okay. On comparing and contrasting, following on from my colleague Mr Evans’s question, the Dundee Families Project was set up in 1995. It is a similar programme with similar objectives—except it worked. How carefully did you consider the proven Dundee Families model before setting up this initiative? Why not just copy it?
Dame Louise Casey: The first thing to say is that I am very fond of the Dundee Homeless Families Project, which is what it was called at that point. It has changed over the years. We tried to replicate it in the Respect era, where we set up 53 projects that were modelled on that. Again, we can claim that they were pretty good and pretty successful.
This time round, there were two things that were very different. First of all, we did not want to just set up a load of individual projects. Neither did we have the money to do that. Let me be absolutely clear: we had £4,000 per family for five in six of the families we were reaching. We never ever had the £10,000 needed to potentially do something like family intervention.
The second thing that I would say is that I am not sure—and I remain to this day unsure—that all of those 120,000 families necessarily need to be living in the Dundee Project. I do feel that the results that are in the overall ECORYS evaluation report show that the system reform is very powerful. Families did have their lives changed very significantly. The impact of the programme is contained in the ECORYS synthesis report, and I feel quite proud of the way we managed to ensure that we got overall system reform and wholesale change in the way that people looked at families. The legacy of Dundee is that in 2016 there is a wholesale approach to how you deal with families in England—now there’s an irony.
Joe Tuke: To reassure Mr Boswell, the other thing we did was second Gill Strachan, who worked on the Dundee Families Project, into our team, to go around the country to help train family workers in the approach to working with families. There was good learning.
Chair: A win for Scotland.
Q129 Philip Boswell: Given time constraints, and the fact I am no professional in this field, I will avoid trying to dig into the key features and specific differentials. My colleague Mr Evans touched on long-term strategy and short-term results, which I will come on to in my final question. A long-term strategy is about results for families, and I have no doubt that you are working towards that. However, the short-termist nature of this seems to be about results for Government and civil servants. Is it so that, as this Report suggests—I am sorry to mention it again—this initiative was an exercise for the benefit of the Government and civil servants’ image, rather than about efficiently helping the troubled families it purports to? I am talking about the way it was presented and the timing of it, rather than the long-term goals to help, which doubtless you have.
Melanie Dawes: I just don’t recognise the characterisation of this programme as something that was just done for presentational reasons, either by civil servants or by Ministers, including the former Prime Minister. This programme was absolutely about improving outcomes for families. Whether Louise, Joe, the local authorities or the key workers that I have met across the country, everybody involved in this is trying to improve outcomes for families. The programme did achieve some improvement in outcomes for families. Everybody knows, agrees and is trying to make the programme better and to learn from the things we wish we had been able to do differently first time around. I just don’t recognise that characterisation of the programme as something that was done for those ends.
Q130 Philip Boswell: This is leading into something that specifically concerns me about the pricing model. One lesson from Scotland is the title itself; the Scottish Government do not class families as “troubled”. Like yourselves, they are working hard to improve outcomes for children in society—of that there is no doubt. On the set up, you said that there are a lot of problems with troubled family initiatives and they do or they don’t work. We seem to have got this very right when we set up the Dundee Families Project. I looked at what we did and we seem to have carefully monitored genuine metrics, managed funding throughout and carefully submitted regular detailed reporting before any financial adjustments over many years.
There are key reports 2006, ’09, ’11—compare that with after only one year. The initial funding was £448 million, and then an additional £900 million was added one year later. Does anybody think that was anything other than a manipulation of statistics for political gain? Clearly, that was not a carefully calculated commitment to expand a proven, tried and tested initiative that was seen to be delivering after only one year.
Melanie Dawes: I am not an expert in family intervention work, as my colleagues are, but clearly the Dundee Project was a good project that achieved good results and is something we have learnt from. It is important to draw out that although the Troubled Families programme evaluation is saying that there is no attributable impact from the programme when you do the detailed statistical analysis, that is not an analysis that the Dundee Project did. In fact, it is not an analysis that any other programme has ever done. So the Troubled Families programme tried to measure itself by a much higher bar than is usual in social policy. That is important for us to explain.
Chair: Which I think we have acknowledged.
Q131 Philip Boswell: In closing, specifically, one year after initial funding was put in place it was trebled, before it was proven to be working, whereas in Dundee they took a long time to adjust it and apply it—or so it would appear from a non-professional perspective. Why is that?
Melanie Dawes: There are a number of things to say about that. First, if we want to achieve sustainable outcomes, it is a good thing that it was actually increased and that a longer term commitment was given to it quite early on—
Philip Boswell: Yes, applied to something that’s working.
Melanie Dawes: Otherwise we might have found that this was just a temporary investment that local authorities put in, and I am not sure it would have had the sustained impact during the last Parliament that it was able to have. From a local authority perspective, that early commitment—that this family intervention approach was something the Government wanted to continue to invest in—was actually very welcome and helped to ensure that the project’s value was longer term and not just temporary.
Philip Boswell: Certainly local government will take whatever money it can get at any time from Government.
Melanie Dawes: But can I also add that the Treasury was very clear with both the first programme and the second programme that, as they gave that funding, they wanted a full evaluation done. I don’t want to credit them too much for the commitment to the evaluation, which is a pretty good feature of the programme, but they were certainly influential in making sure that the evaluation was done. They were always concerned to make sure that the value for money analysis was done. That was always on the table as part of the discussions.
Philip Boswell: Just for the record, this was an initiative where it was payment by results. Money was thrown in because the statistics were coming out quickly and the additional money went—
Chair: You have made your comment, Mr Boswell. We can leave it at that. Caroline Flint, very briefly if you can.
Q132 Caroline Flint: Family Action, which sent a submission to us, has acknowledged that looking at issues around domestic violence and mental health are a good, positive contribution to the second phase. What analysis have you done of the original first phase families and of how much the domestic violence and mental health problems were present? Did that lead you to add that to the second phase?
Dame Louise Casey: I will try to find the statistics, but we did a first look at the family monitoring data—is that it, Joe?
Joe Tuke: Yes.
Dame Louise Casey: A third of families were suffering domestic abuse in the first programme. Also, the other thing that I was concerned about was the under-fives. We needed to use the next programme to get to kids who were not of school age because, in the first programme, clearly, the emblematic criteria—in and out—were school-age children. In terms of mental health issues, 42% of adults have mental health issues. In the 2014 “Understanding Troubled Families” report, we started to be able to see this picture of health and health in children overlapping. So if the parent was ill or disabled, their child was also more likely to be ill or disabled. We wanted to learn from all of that.
Q133 Caroline Flint: On the criteria for Phases One and Two—trying to pinpoint certain things in families to help local authorities identify these families—in the work done in Phase One, how much analysis is going on in terms of how they arrived at the point where they fitted the criteria in the first place, and in terms of what to do earlier to stop it happening if children are not attending school and parents are out of work? Is there analysis of that? How did they get there to qualify to be a troubled family? What work is being done to get ahead of that?
Dame Louise Casey: The new programme is specifically designed to get ahead of the problems we were dealing with in the first programme. We were dealing with emblematic problems through the payment by results that were three, but remember we were dealing with families with an average of five, which would include the ones you are concerned about. The second programme has a wider set of criteria that include the things you are concerned about so we can get to a broader range of families and, frankly, we can get to them earlier. I am looking at Joe because he runs this now, although I am talking about it, but essentially a lot of the second programme fits much more into an early years offer made within the local authorities, in terms of where it is structured in children’s services, and that is the right place for it to be.
Q134 Caroline Flint: Can I ask Melanie Dawes a question? My colleague wants to come in. You have had lots of different programmes over many different Administrations. Would you be prepared as the permanent secretary to go back to Ministers and say that what we need to have is a more long-term sign-up to family intervention programmes, or whatever you want to call it, over a longer period in the same way we have things like pensions, which are intergenerational with different Governments? Would you be prepared to go back to Ministers and recommend a much more long-term programme?
Melanie Dawes: In the end, that is a policy decision for Ministers, but in some ways—
Chair: If you can answer this one—
Melanie Dawes: It’s rather like some of the conversations we have about devolution in this Committee. Sometimes we ask for an approach that is really just about common sense—does it make sense to join up services around the family?—and we ask that to prove itself.
Caroline Flint: People have been paying a lot of money and not applying common sense.
Melanie Dawes: We do not ask the current way of doing things to prove itself in the same way, and that is always worth bearing in mind. Do I think personally that there is evidence that this approach has an impact? Yes, I do. I think we can point to a number of different programmes over the year that have shown it in the smaller boutique programmes that have preceded the Troubled Families programme. I also think that we have good evidence that this programme has transformed services already. It engages local government and it has begun to engage families themselves in a new way. I do not think we should underestimate the importance of local government being engaged.
Chair: I am afraid we will have to come back. If we have finished with Caroline Flint’s questions, we will come back with Ms Phillipson. We will be five or 10 minutes. I am sorry about the interruption.
Sitting suspended for a Division in the House.
On resuming—
Chair: Okay. We are now quorate. We are broadcasting again. We will let Miss Dawes take her seat; it looks like she has slipped out. Welcome back to the Public Accounts Committee on Wednesday 19 October 2016. We have just had a Division on the rights of EU nationals on the Scottish National party’s Opposition day. I am now going to hand over to Bridget Phillipson to continue the questioning.
Q135 Bridget Phillipson: We have talked about council funding and the significant cuts councils are facing. With that backdrop, how realistic do you think it is to expect councils to take on this work on a longer term basis? You referred earlier to trying to get councils to embed this approach to their wider work. Is that realistic?
Melanie Dawes: What we tried to do in the first programme, and have done again, is to give some funding, particularly focused towards the start-up costs. In fact, that element—what we call the transformation grant—has increased in the new programme, recognising that this is partly about ongoing service provision, but it is also about investing in the data and the systems that local authorities need to get this new cross-agency work going. The analysis we have done suggests that, in the first programme, the payments were worth about 40% of the actual costs of working with the families. Local authorities would of course have liked more, but I think they have felt it is clearly in their interests to invest in this because it saves them money in the long term.
Q136 Bridget Phillipson: But it is not necessarily just in their interests. That is the difficulty. Much of the savings that will be made will be of benefit to other agencies but not necessarily to councils.
Melanie Dawes: Yes, I think that is true.
Dame Louise Casey: If you look at somewhere like Manchester, I think that they, through positive interventions with the families, managed to reduce the number of children in need by a very significant percentage by using the family intervention approach, which meant they could be stepped down. Clearly, yes, there are savings for local authorities, but you are right to say there are savings for other agencies as well.
I think there is a broader point here, which is this: to be blunt, local authorities, like everybody else in the public sector, are dealing with quite significant cuts, and they are therefore having to reshape the way they do their services. I think one of the reasons the first troubled families programme was embraced was because they could see that they couldn’t do all of this on their own. They can’t walk away from children who are in need or are on child protection plans; they quite rightly have a statutory responsibility to those children and they need to do that well and are tested on that.
Essentially, the programme was of its moment. In fact, I thought it was interesting that, in one of the interviews, a lot of people used words like pioneering and zeitgeist. It is interesting. It was of its moment because they know they are facing exactly the sort of situation that Ms Phillipson is referring to. They are facing down the line and asking how they are going to cope with the sorts of problems they have got and the money they have got. The ECORYS evaluation—the overall evaluation report—I think shows the fact that we wanted to pioneer large changes in local government and others about how they did their work, and actually the ECORYS report shows that they did.
Q137 Bridget Phillipson: But local government has to be properly funded for that to happen. We heard earlier about the need for investment in a new workforce—those key workers who work with families—but in Sunderland we can’t recruit social workers. As in many part of the country, we have a major problem when it comes to children’s social care and work with families, and with councils fulfilling their statutory functions. Would it have been better to fund children’s services departments properly? It is not an either/or, but if we are not properly funding core council work, is this programme ever going to have the impact that it should?
Melanie Dawes: That goes to the overall question of local government finances. We had a bit of a discussion about that last week, in fact, at this Committee. The analysis that we fed into the spending review was that it was a challenging time for local government, but the pressures on adult social care were the big area where there was concern. The Government did recognise that with the social care precept and additional money for the better care fund. On the question of children’s services, I do not think we would recognise the characterisation that they were seriously underfunded.
Q138 Bridget Phillipson: Well, if you would like to visit Sunderland, I can show you the problems we are facing.
Melanie Dawes: It is clearly a challenging environment, but we would not recognise that overall characterisation. In investing in family intervention, what we are doing is bringing the other agencies on board in a way that really helps local government; that is one of the things it really welcomes.
Q139 Bridget Phillipson: But this has gone hand in hand with—
Chair: 25% cuts.
Bridget Phillipson: Yes, massive cuts, at a time when programmes around intervening early that are shown to have worked, like Sure Start, have either been cut back or, in some areas, almost ceased to exist. I appreciate that this programme is wider than that, but again, we talked about the need to intervene earlier, and much of the money and resources that went into intervening earlier are actually no longer there; it is not happening.
Dame Louise Casey: I would say two things to that. First, we are not suggesting that the first Troubled Families programme, which I was responsible for, was the panacea to all the difficulties facing local government—austerity and everything else—but it is helpful, and it has definitely been part of what it has had to deal with.
Secondly, if you look at somewhere like Sunderland, which you have raised, the last Ofsted report, though it found it inadequate overall—I realise that it is in special measures and has been transferred out to a trust and all the rest of it—nevertheless actually rated it good, in terms of what it was doing around early years. It is trying to re-establish what it does in that space. I am not for a moment defending Sunderland local authority to you, but I am saying that these things are always much more complicated than they might appear to be.
Q140 Bridget Phillipson: You do not need to defend Sunderland to me. I am not saying that they are not doing very good work, but the reality is that they are facing a very challenging financial climate, which means that they are finding it incredibly tough to do even some of the statutory interventions that they are required to do with families, where families are referred and there is child protection involvement. Going beyond that is hard.
Dame Louise Casey: I think that is right, but I would honestly say that if we thought that everything that happened in the world of early years before the cuts was perfect, that would not be right. The idea that some of the families getting help under both the old scheme and the new scheme were going to go to children’s centres and Sure Start is often quoted to me. A lot of the families that were targeted through the Troubled Families programme, including in places like Sunderland, Newcastle, South Tyneside and North Tyneside, were being reached out to by Troubled Families, or whatever it was called locally, because we knew they were not coming to early years. These things are always much more complicated—and, dare I say it, I don’t think the problems with Sunderland children’s services were just down to money.
Q141 Bridget Phillipson: No. In terms of the outcomes for families, going back to this issue around the percentage of families turned around, figure 6 on page 14 shows that the total percentage of families turned around in the north-east was 100%. Is that credible?
Dame Louise Casey: What we have tried to explain before—unfortunately, we are running behind the discourse of the past couple of days—is that the 100% is about the number of families for which the local authorities made a claim, having met the stated objectives of the scheme for those families. That is why they could get 100%, and why we are saying to Mr Evans that those could look good on paper if you overspan it, but the truth is that most areas should be at 100%, because they should be working with enough families that they are able to make those claims. I have always been very specific that the 117,000, which was our goal, was the number of people whom they could make claims for, and who had all the problems. That is not an insignificant number. It is not an insignificant achievement. I come back to the fact that just doing that with 116,000 families is fairly phenomenal.
Q142 Bridget Phillipson: But we are talking about people who have been the subject of repeated unsuccessful interventions in the past. Many of them are, by definition, known to a variety of services. Everything else has failed, yet suddenly this programme appears and it has a wonderful success rate.
Dame Louise Casey: That is because, when you get the time to read the ECORYS evaluation report, one of the things in it that gives me great comfort is the fact that they did the process evaluation and they showed very clearly the type of stuff we wanted to happen—a whole-family approach. As for the reason why many things have failed in the past, let’s take domestic violence as an example: if all you do is move the woman and her children to a refuge at midnight, rather than dealing with everything that is happening in that family and bolstering up so that she does not go back again—it is as important as an intervention as anything else.
Yes, Joe’s right; I’d forgotten the very wonderful Gill Strachan, whom we recruited into our team, to get her and others to go round the country training people in how you do family intervention. That is why some of it worked better than before. I am not saying it is perfect, but some of it definitely made a difference.
Q143 Bridget Phillipson: But are the measures just not ambitious enough? Are they insufficiently ambitious? If you have to meet only two of those three, you are potentially talking about quite a lot of families, without getting into a dispute about the numbers and how you arrived at them.
Melanie Dawes: If you are getting into a question of, “Does that mean ‘turned around’?”, I think that is subjective. Ms Flint asked us earlier whether we thought the phrase “turned around” was helpful, and Louise gave a clear answer.
Chair: You could do the same, Ms Dawes.
Melanie Dawes: I gave a bureaucratic answer, which probably wasn’t very helpful. But I think what we are saying here is that this is the number of outcomes that was achieved—the number of families where the outcomes we set out to achieve were achieved: children stayed in school for a year longer, somebody in the household went into work, there was a reduction in antisocial behaviour or another indicator that the local authority thought was significant. It is as set out in the programme. I think it is then a question of: was that truly sustainable? Does that really mean ‘turned around’?
Q144 Bridget Phillipson: And would it have happened anyway?
Melanie Dawes: What we are not able to say definitively, with statistical certainty, is that it would not have happened anyway. No other programme has asked itself that question, of course; we haven’t been able to answer it either. We have said that we cannot answer it, yes or no, but what we can say is that we achieved the outcomes we set out to achieve in the payment-by-results programme.
Q145 Bridget Phillipson: Finally, I appreciate that trying new things involves taking risks. Sometimes things will go well, and sometimes they will not. I do not think anyone will want to deter Government from trying things that are about a new approach, but doesn’t the way this programme has been presented, the claims that have been made about it and the amount of money that has gone into it undermine the whole approach to that kind of programme in the future?
Melanie Dawes: Louise may want to come in on this. It is unfortunate that over the last two days we have had the debate that we have in the media.
Q146 Bridget Phillipson: It is not about the last two days. You just have to look at the report. We can all see how Government have claimed this project as a wonderful success, and the vast amounts of money that have been spent on it. We all want great outcomes for families in difficulty. This is not about the coverage or the evaluation or whatever else.
Melanie Dawes: But we have had the improvement in outcomes, and the evaluation says that. What we cannot do, with this very statistical point in mind, is prove definitively that those outcomes are attributable to the programme, rather than to the quite complex world in which the families live, with interventions and services that are already provided to them. Personally, I think that was just a very, very high bar for us to set ourselves; no other programme has done that.
In terms of the actual programme, though, I hope we have been quite clear, including in the evaluation and in the cost-benefit analysis work that we published on Monday, that we are trying not to over-claim here in the documents that the Department has produced. For example, we wouldn’t use a percentage success rate; I don’t dispute that that is arithmetically correct, but it isn’t the way we would describe the programme.
Chair: Very briefly, Dame Louise, and then we will need to move on to another question.
Dame Louise Casey: The thing is that, alongside Ministers, I have publicly represented this programme and its predecessors for three or four years. Essentially, the question is: did we oversell and under-deliver? I honestly think the answer to that is no. Did we change the lives of 116,000 families? Yes, we did. Did we pioneer new ways of working with families that meant that local authorities were working in a whole-family way, rather than on a child-by-child basis? Yes, we did.
Did we do something extraordinary? At the time, I had my own feelings about the payment by results scheme. I’ll be honest with you: I thought, “Dear God, what is payment by results? Will that work?” What it did was galvanise people to work in an incredibly difficult way, so that, actually, we were knitting together people from the police and so on. It wasn’t just people having partnership meetings and mashes; people actually had to work with families in order to do something in a different way.
I know, with the benefit of hindsight, that it is always easy to say, “You over-promised.” We didn’t over-promise; we said that we’d try to sort out 120,000 families, and local authorities and their colleagues across the country sorted them out. Nobody seems to be disputing that in terms of evidence, and nobody disputes the fact that, actually, the impact on those families was monumental, in terms of the seven problems that each of those families had—they had an average of seven very, very significant problems—and the family monitoring data shows that.
What we can’t prove, as Melanie has been consistent about throughout this, is that the NIESR research says that you can’t attribute all of those things directly to the Troubled Families programme—you can’t not attribute them, either. No, we can’t. Not with that piece of research. Do I still believe it was the right thing to do, and did I oversell it? I still feel that what we did was the right thing. Honestly, I do.
Q147 Bridget Phillipson: Ms Dawes, on the basis of the evaluations and everything you have seen, should the programme continue?
Melanie Dawes: Yes, I believe it should. In the end that is a decision for Ministers, but the evaluation is very clear about the transformation that has already been achieved in how local authorities and their partner agencies are working. It is clear that there are statistically significant improvements in how families feel about their circumstances, and it is also very clear—in many ways this is one of the most important legacies of the programme—that we are now operating with data and with an outcome focus in these complex local service areas in a way that is new. Those things, for me, are very compelling results to have achieved in addition to the outcomes that have come through the payment by results mechanism. Do we need to improve and adapt? Yes, we do, but I think that throughout the last few years this programme has adopted quite a humble approach to improving what it does, and it has done that in partnership with local government. That is not that usual for central Government, but it has been a good feature of this programme.
Q148 Chair: Can I just ask you a few quick questions to finish off? You have talked a bit about data—we always talk about data on this Committee—and you have talked about the difficulty of building national data sets. Given that local authorities are doing this work anyway in various forms, such as through education and children’s services, and the DWP is doing work to identify these people and the police have that information, and I appreciate that data sharing was a major win in this—it is one thing that everyone has acknowledged—why was data such an issue from the beginning? Surely with the data sets that you knew were available, you could have set up an evaluation structure drawing on what was there. At least you would have had better raw material to work with from the beginning, rather than trying to build it from scratch.
Melanie Dawes: Joe might want to come in on this, but what wasn’t there at the beginning was an ability, at local level, for a council to share their data with DWP—in fact that data gateway, as we call them, came in quite late in the programme—and with the police, and so on. That infrastructure just wasn’t there. People tend to assume that it is there, but they didn’t have systems, data protocols or sharing protocols. We had to legislate in some areas to allow that data to be shared, particularly on the DWP side. That is one of the things that the programme has done.
Chair: I recognise the data sharing, but the point is more about the fact that the data, even if it wasn’t shared, was there. You could ask a local authority, “How many children have you got on the child protection register?” You could ask them about school truancy rates, and you could probably map some of those individuals together within one authority, which is an easy way to share the data, yet it became a big issue. It has come through very strongly in today’s hearing that getting the right data has been problematic for your research, for the evaluation and for proving that taxpayers’ money was spent wisely, in effect. I still don’t understand why, with the amount of data out there, even if they couldn’t talk to each other, it couldn’t have been collated more effectively.
Joe Tuke: In the past, a lot of the family intervention projects that we have talked about and that were the forerunners for this used local qualitative information, whether from local systems or from local keyworkers, to ascribe their success. We were looking to get a much more robust basis. This was the attempt to use national data systems for that, rather than just relying on the local provision of data. Where there wasn’t any national administrative data, such as on domestic violence or on antisocial behaviour, et cetera, we collected that information from local authorities and brought it together. That is represented in the evaluation as well. I don’t think we should underestimate the real difficulties that there were in bringing that sort of data together, and that we still have. The NAO Report and the response to the PAC’s previous hearing recommendations give an idea of some of the gains that we have made, with health, the police and DWP jobcentres, in getting through some of that but it has been a real, hard slog. It would be wrong to suggest that there was an alternative way to structure all this from the start.
Q149 Chair: Okay. We always talk about data on this Committee and I’m sure we’ll come back to that. I hope we can deal with this fairly quickly but it is quite an important issue in terms of the evaluation. How realistic was it to secure a proper realistic comparative group, given that, of the pool of people you are talking about, most of them will be having some intervention, even if just through the GP or whatever? Was that a big issue in measuring it? Perhaps you could explain a little, Melanie Dawes.
Melanie Dawes: You have identified exactly the problem that we had in doing that very robust analysis of what would otherwise have happened. We did not have a control group; we never set out to put to one side a group of families who were not going to be worked with through the programme. I think that would have been wrong, but as a result, it was then hard to say exactly what the programme did compared with what would otherwise have happened. We looked at the national administrative data work. The researchers looked at families who had similar characteristics to those who were in the programme, but it was an estimate, so it was subject to that level of uncertainty.
I think that problem will be with us throughout this work. As I say, it is very unusual for social policy evaluations to solve that problem, by their very nature. That is why, in reply to Sir Amyas’s earlier question, I answered with some realism, I hope, about what we will see from that particular element of the evaluation in future.
Q150 Chair: I want to ask you this question, Melanie Dawes, as you are responsible overall for this. You said at the beginning that the buck stops with you. You have a seat with colleagues across Government to talk to them about this. Do you think this is going to be almost impossible to analyse and evaluate? What do you think can really be done? We have talked about all the various initiatives over the years: good will, good intent, bits and pieces of good practice. Is success possible in this arena, in all honesty? Where do you see this going from now on?
Melanie Dawes: I am an optimist and I am optimistic about this. I started my career as an economist so I have some understanding of the techniques here. I am an optimist here because it is new. This is the first time that anyone has set out to do this kind of national statistical analysis. Our partners at the national institute were doing it for the first time. It is a very exciting piece of research. I am optimistic that we will get the data issues better under control and that we will be able to show more progress in future. But it is always going to be about estimates.
Q151 Chair: Moving on from the data to the actual impact on people’s family lives, do you think it is too big a challenge for Government centrally to get a grip on, or do you believe we will get better results over time?
Melanie Dawes: Yes, I do. I was asked earlier whether I actually believed that there was enough evidence that would lead me to feel confident that we should continue with this. While that isn’t a policy question, personally, yes, I do think that there is sufficient evidence.
In Government, we need to recognise sometimes what we can and can’t measure. The notion that it is better to work with the whole family in a joined-up way, to use a data and outcome focus is common sense.
We then have to be sure that when we are putting money behind those approaches we know that we are achieving value for them. Which is why we set up a payment-by-results mechanism, so that we can show that, albeit that that comes with some issues and limitations as well. So we are always striking a balance here but, yes, I do feel confident.
Q152 Chair: Touching on payment by results, because it may be connected. We touched on longitudinal studies and, Joe Tuke, you talked about being able to track progress. How will you track progress? If I am a mother in a family that has got challenges, how are you going to track what happens to me in my lifetime and perhaps the outcomes for my children?
Joe Tuke: At a local level there is initially a plan agreed between a key worker and a family, where they will set those goals around what they are trying to achieve with the family. They will track that and use their own local administrative data but also key worker assessment. Some things you can’t get administrative data for and so they use key worker assessments. At national level we collect data from the local authorities on a cohort basis. They send us details of every individual in every family they are working for and we, with our partners in the Office for National Statistics, are then able to match that against national datasets and link that back into families. We can look at activity from five years prior to intervention to look at the amount of criminal activity, what has been going on in schools and so on, through the interventions and then for a period of up to five years after the intervention.
Q153 Chair: To be clear, if you had a schoolchild now in this programme who had truanted, then met the first target of a year in school without any absence—or with limited absence—you can track them for another five years. So if they were 15, you would be able to track them on to possibly going to university or an apprenticeship or something like that. Would you track where they went what the outcomes were?
Joe Tuke: If the information is available in the national pupil database, we can check them against that for a period of five years from now. Past that, we will have to get into new arrangements and new funding, but we have the agreements in place that we will do that, and we will look at that every six months. That does not give us the counterfactual against whether we can attribute that—
Q154 Chair: But the real prize is if you break it generationally. So five years sounds great, but it is not really long enough.
Dame Louise Casey: The answer in part is what Ms Flint was saying. The goal, which Mr Evans was also talking about, is to look at something over generations. No, we are not attempting to do that in the Troubled Families programme. Do I think that would be a really useful thing to do? Yes, I do. Do I think you would be able to deal with the counterfactual issue of finding people you have not dealt with? I think you would find that hard. Do I still think it is worth tracking whether someone helped through family intervention, or whichever programme, actually has their kids in Russell Group or whatever is the goal? I think it is worth it, definitely. That is not something that we have commissioned from Government at the moment.
In the same way, I have lived through political cycle after political cycle and some of the things that you want to happen that go through all those cycles are not straightforward to navigate and land, and the whole-family approach and family intervention is one of them. I do think—I can say this now because this is probably the last time I will appear before the PAC, in this role at least—that if you could recommend that we did something like family intervention in the longer term and you recommended how we looked at that in the longer term and we could look at whether something like payment by results really does fit in the next scheme, that would be helpful, because all we all want to do here is exactly what you said, which is help families.
Q155 Chair: I just happen to remember being told about a decade ago about the social science budget being slashed and a lot of these studies being stopped, but let’s not digress. One quick question for Ms Dawes: there is now an annual progress report requirement in legislation. When will that start? When is the first annual progress report coming out?
Melanie Dawes: It will be published before the end of March in this financial year.
Chair: Just to be clear—March 2017?
Melanie Dawes: That is a legislative requirement.
Q156 Chair: In what form will that be? Will it be published online, or paper copy?
Melanie Dawes: It will be a report to Parliament.
Q157 Chair: Have you worked out what metrics you will be using to do that, given your concerns about the evaluation so far?
Melanie Dawes: Not in detail, but we will be expecting to report against the key outcome measures for the new programme. That will include the number of families already in the programme and any early data we have on the outcomes.
Q158 Chair: Will it track trends over time?
Melanie Dawes: I think over time, as we build up the picture, that is the sort of thing that we ought to be able to include, yes.
Chair: We have had an interesting hearing today. You have picked up some of our concerns: there are poor datasets, but I think you have acknowledged there are issues there; and there is poor evaluation. We know it is very difficult to evaluate, but, as Ms Phillipson highlighted, everyone wants these things to succeed—the key thing is proving it.
I have to say, speaking as the Chair of a Select Committee of the House of Commons, the slow or delayed release of the documents stored up some problems for you. It certainly does not look transparent or give the impression of openness and of wanting to share that information, given that it had been knocking around Whitehall for some considerable time. I hope you will take that back to the Department and to colleagues that it gives the appearance of hiding things, and concern about progress and suppression. Those are not words I like to use. I hope that Government is trying to be open, but it did not come across that way because of the way the documents were released.
Dame Louise, you talked about changing the lives of 116,000 families. Of course, that is what everyone wants to see happen. We are not sure yet whether the proof is absolutely there in turning around. As you heard from Caroline Flint, I think that “turning around” phrase is a bit of overselling, because one intervention that achieves the results in the narrow criteria is good, but is it absolutely changing the lives of that family in the long term? I think, on a note of agreement, that is what we would all ultimately want to see from a programme like this.
Thank you very much for your time. Apologies: that was quite a bit longer than we expected, partly because of the vote. Our transcript will be up on the website in the next couple of days uncorrected and our report will be published before Christmas.