Examination of witnesses
Jill Rutter, Jonathan Breckon and David Halpern.
Q16 The Chairman: Ms Rutter, I am sorry that I did not greet you outside; I thought your assistant was you. I apologise. She did not contradict me—you had better watch out. Welcome to you, to Jonathan Breckon and David Halpern. You are three experts in precisely the area we want to talk about today. We are delighted to have people with so much experience, and such diverse experience, as you bring to bear.
We have a set of questions that you have seen. We will probably stick to them but may digress occasionally from time to time. The first one, as you will see, is rather broad ranging. Quite simply, how well do the Government use evidence to make policy and develop legislation? I do not want a one-word answer, “Badly”. It is an opportunity for you to open up and get off your chest one or two things that you are afraid you may not have the chance to say later. Shall we start with you, Ms Rutter?
Jill Rutter: The simple answer is that it is quite variable. There are some examples of evidence being used quite well. Most people regard the way in which the Government went about making their policy on pensions under the Labour Government—the Turner commission—as the gold standard, the rational policy-making model of spending a long time, partly for political reasons, gathering your evidence base, subjecting that to external criticism and then spending quite a long time developing policy proposals and building consensus around them. At the other end of the scale, there are clearly some decisions where the evidence base is difficult to find or, indeed, very thin. It was quite interesting that in the foreword to the Social Security Advisory Committee annual report, Paul Gray, who used to be chairman of HMRC and a second Permanent Secretary at the Department for Work and Pensions, drew particular attention to what the SSAC thought was the extremely poor—or at least ill-exposed—evidence base behind a lot of the Budget welfare decisions. So there is a big spectrum.
Since 2010, with the coalition, there has been quite a concerted attempt to produce more evidence. I am sure that David will talk about the What Works initiative and things like that. He will probably disagree with me on this, but there has been more willingness in slightly second-order, more technocratic, less politically contested areas to engage in bits of experimentation, trialling and things like that. Those are all very welcome developments.
There are some things where government just has to act in the absence of an evidence base. Where the Government are working in a bit of a blind area, what is really interesting is the provision that they put in place for capturing information as the policy develops, so that they can tweak as they get feedback on what is working and bearing out their underlying hypothesis and what is not.
David Halpern: Thank you for inviting us. The basic point is that there are lots of good ideas that do not work, but seem very plausible, through legislation and other government practice. We have to embrace a fundamental humility to acknowledge that. That is the point. Often we are overconfident about what will and will not work, whether it be that passing a law will affect people’s behaviour in some significant way or that using a tax instrument will have a given impact. Broadly, that is what lies behind the What Works idea—to embrace and run into that question by saying, “Actually, we often do not know what will be effective or not”.
In principle, that applies absolutely to legislation. The point that Jill made as an aside is right; perhaps we will get on to it in more detail. The What Works stuff pointed in a direction away from legislation. It is about all the many choices that are made by head teachers and others about the best way to teach maths, the best way to spend a budget if you are a local economic partnership, or the discretionary judgments that apply in the criminal justice system, such as whether the police should wear a body-worn camera. You do not need to go to legislation. There are many questions where you can say, “What is the most effective thing to do?”
It is a really good question to ask about legislation itself, as well as the role that legislation can play in making something possible. The discussion about social work is a good, topical one; maybe we will get on to it. How do you reconcile the basic issue—the really deep thing—that very often we do not know what is most effective? The only way we will find out is by testing variations in practice. That is not really how laws are generally thought of. They apply for everyone, all the time. How do you square that paradigm with, essentially, a scientific one that, in order to find out what is effective, we have to vary our practice?
We are definitely making some progress. Has it really come to bear in a deep way in relation to legislation? I do not think so. My own work, much of the time, has been as head of the Behavioural Insights Team, as well as national adviser on What Works. We spend a lot of time avoiding legislation. The point is that there are so many other things that you can do. Indeed, a prior question is, “Are you sure that we need legislation at all? We could probably get this effect more effectively in some other way”.
The Chairman: You said that there are a lot of good ideas that do not work. Can you define areas, sectors or particular types of problems of that kind?
David Halpern: Yes. We can take them from almost any domain, although the back story will be that the more frightening ones are those where we do not know whether they work at all. One of the most notorious examples, often rehearsed, is Scared Straight in the criminal justice system. In fact, when I was writing up the book ‘Inside the Nudge Unit’ a little while ago, I was struck by the fact that the London Evening Standard had a big piece saying that we were going to pick up kids who might be getting a bit more troubled, put them in handcuffs in the back of a police van and show them prison. The idea was that obviously that will scare them straight. The programme is run on a massive scale across the world. In the US, it is done in huge volumes, for a few dollars a time. Two systematic reviews showed that not only does it not work but it makes kids significantly more likely to offend—in fact, by some estimates, 60% more likely to offend. It is a beautiful idea. You can see why it is a lovely idea; it just turns out empirically to be disastrous.
That is probably true of a lot of things that we do. Even in medicine, which is normally held up as the great example, with a strongly empirical base—not least since the work of Archie Cochrane—we still get examples of it. A term that is sometimes used, even in medicine, is the so-called parachute test. You should not do something because it would be unethical to run a trial on parachutes—to have half of the Committee jump out of a plane with a parachute and half without one, to see how it works—but the ‘parachute’ defence is often used, because it is seen as self-evident that a practice is more effective.
There are now some very famous examples of things that were once considered self-evident. Ben Goldacre has given an example to do with head injuries. Until recently, if you had a head injury you would be given a steroid injection. There is a very good case for that, because swelling on the brain is very dangerous and steroids should reduce swelling. As it turns out, two randomised controlled trials done in recent years show that you are more likely to die if you are given a steroid injection after a head injury. Do you see what I mean? The basic point is that there are many such areas. The frightening thing is, what are all the areas that we have not tested and where we are probably doing harm?
Jonathan Breckon: The use of evidence by government in policy and legislation is not good enough. I want to bring up a few examples. It is not good enough at using the social science that is out there. The UK is blessed with having a world-class social science base and a phenomenal supply, not just the What Works centres and the Behavioural Insights Team, but many others—including, from here, people such as the Parliamentary Office of Science and Technology. It is just not used enough. It can be cherry picked and distorted.
We are keen to recommend that we be a little more suspicious of single studies and that we look for wider bodies of trustworthy evidence. That is why the What Works centres David heads up are so crucial: they can do what are called systematic reviews—a technical phrase—bringing together large bodies of trusted evidence. Critically, such reviews also show you where there are gaps—the things that we do not know. Going back to the humility question, we need to be up front about what we do not know. We need to use more of those bodies of evidence.
We also fall down when we commission new evaluations and evidence as we go along with new pieces of legislation or policy. They can be of poor quality. The National Audit Office looked more deeply at some government evaluations. Half the evaluations it looked at were poor quality. It was particularly worrying that, the worse quality it was, the more likely it was to have stronger recommendations for different actions. Do you see what I mean? The weaker it was, the more confidently the evaluators would say, “We have some clear policy recommendations”. Supply and demand are critical.
Finally, I would love to answer your question in a more evidence-based way, but I cannot. The reason why I cannot is that it is so hard to find evidence behind particular policies and legislation when you are on the outside. An initiative we are involved in with the Institute for Government and Sense About Science, which David kicked off, is looking just at whether you can show the workings behind particular policies. It is extraordinarily hard. If we take something like the Department for Education’s Educational Excellence Everywhere White Paper and look at even really simple things such as the referencing, we find that it goes nowhere. It goes to a splurge of different, random bits of evidence—so we cannot find the evidence behind the particular policy.
There is a lot of good practice, but we have found examples where even policies that have good evidence behind them have forgotten to show it, so we cannot even reward the good practice. In South Africa, they now have White Papers that have a section that shows the evidence. That may be something you want to look at. It can be empty, which is fine; you are being transparent about the fact that there is no evidence or it is of very dubious quality—it is a think tank with some wishy-washy survey—but at least you have something to get your teeth into. That could answer your question.
Q17 Lord Hunt of Wirral: Mr Halpern, given all the work on behavioural insights, to what extent should public opinion be listened to? Several times we have responded as a legislature, as with the Dangerous Dogs Act. Now, post-Brexit and post the American elections, people say that it is all going wrong because we are not listening to the public. I have a horrible feeling that the public would want us to undergo the parachute test. How could we listen better?
David Halpern: I will give you my personal view, which is obviously not necessarily government policy. There are mechanisms that can solve that issue. Personally, I think that there is a particularly strong case for the use of what are sometimes known as deliberative forums, which bring a sample of the public to hear evidence on an issue. Effectively, it is like a legal jury. You could take 100 people. We were involved in one in Australia, interestingly, on obesity; we help other Governments these days. There were 120 people, who were exposed to evidence from a whole load of experts and other stakeholders and asked to give a view themselves: “What do you think?” You get a very different, much more sophisticated view from the public when they have heard about the issues, not least from one another. They are asked to engage in the trade-off.
One of the ways we should respond to these kinds of challenges is by introducing similar sorts of sophisticated mechanisms for the public to hear and give a view, not necessarily a binding one, that addresses some of the concerns. You mentioned it specifically in relation to behavioural stuff. It is especially true there, because often it concerns everybody’s behaviour out there. There is naturally a degree of suspicion sometimes about whether you really want the Government nudging you, and using these more subtle approaches. Adair Turner’s work to change the defaults on pensions is a great example. One of the things that underlines it as an excellent piece of work was that there was very good-quality consultation. In some ways, the public, when given the evidence, gave permission to make the change—to change the defaults on pensions. And by the way, it has been extraordinarily effective; 6.5 million extra people have started saving for a pension since 2012.
Jill Rutter: In the wake of Brexit—not in the wake of the Trump victory, but in September—the Institute for Government did some polling. We asked people to what extent they wanted politicians to consult experts when formulating policy. More than 80% of people—there was no difference between leavers and remainers—said that they still expected government to take on board the views of experts. They also wanted politicians who delivered on their promises. That was the other big message from that polling. They also had great confidence that this Government would deliver. That was quite an interesting message as well.
Q18 Lord MacGregor of Pulham Market: What are the constraints that prevent government departments from proposing evidence-based policy and legislation? Is there a lack of capacity within government for dealing with that?
Jonathan Breckon: The capacity issue is about the smart demand from government, civil servants, your advisers and staff to interpret and engage with the evidence. As the Alliance for Useful Evidence, we have been piloting, with Civil Service Learning, training of the policy profession. Here, with the Parliamentary Office of Science and Technology, we have been helping clerks, researchers and other staff in both Houses with their capacity to interpret evidence, and to know what is rubbish and what they can trust—really digging down. That kind of intelligent consumption of evidence is the bigger thing; that is where the capacity issue is. In the supply, there is almost too much evaluation, with academic papers and research. That is why things like the What Works centres are so crucial; they synthesise it and put it into an actionable form, but we still need that almost absorptive capacity.
Lord MacGregor of Pulham Market: Is the in-house capacity sufficient?
Jonathan Breckon: I think it is. The in-house capacity is more about staff demanding the right things.
Jill Rutter: It is an interesting question. Departments all have reasonably sized analytic teams. There is a quite big analytic capacity in government departments. There is a question of how far that capacity is deployed in policy-making. Sometimes the view is, “These are the sort of people we will put in a box here and consult when convenient”. There is another quite interesting question about the willingness of civil servants to challenge Ministers when they do not necessarily think there is a great evidence base behind the policy that Ministers are suggesting. That never happened in our day. If you do not feel that there is a licence to say, “Okay, but that is not actually what this suggests”, it is quite difficult.
Jonathan says that there is a strong evidence base. I think it is quite variable. We have just been doing a project on tax policy-making and whether it is a question of supply or demand. There is good work done by the IFS, obviously, but across academia as a whole there are very few people who do really serious academic work on tax policy. There are little pockets but not a hugely massive evidence base out there. In a sense, the Government can create more of that if they are more willing to go outside and get engaged with academia, so that academia knows what questions the Government are asking.
There is also always a real problem with timing. In 2012, we did a report, Evidence and Evaluation in Policy Making: A Problem of Supply or Demand? In it there is a quote that was given, unfortunately, under Chatham House rules. At one of our seminars, a Minister said, “The problem is, if there is no evidence that is immediately out there and I want to make a decision, I am up against the political cycle. A reasonable forecast is that I am probably in this job for 18 months or two years. I want to get things done. I need the evidence now. If I ask you to get the evidence for me, it will take you six months to develop a proposal and a bit longer to get it funded, and you will come back with the results in a couple of years’ time. By that time, I will be Fisheries Minister”. Actually, he was not Fisheries Minister by that time—he would have had time to use it—but there is a real conflict between what is there now and what Ministers need now. Ministers are always going to have to make decisions with only partial information.
The Chairman: Before I bring in Dr Halpern, Lady Dean and Lord MacGregor want to follow up.
Q19 Baroness Dean of Thornton-le-Fylde: I had a question about technology, but we may as well sweep it up in this, as Dr Halpern referred to other consultations that could take place for evidence. In this world of technology, is there a place for better use of technology in developing proposed legislation? We heard last week from the Hansard Society that it is doing some work that may help. There is the other argument—from the Institute of Chartered Accountants, I think—that you do not want to overengineer the thing, because then you are also restricting it, in a way, to people who have full access to the technology. Is there a way we could use what we have at the moment or, perhaps, different technology? Have you looked at that?
David Halpern: A bit. Can I sweep it into a slightly wider point? On the earlier question, I would point to three kinds of areas or particular lacunae. There is some evidence that is good stuff. Of course, it varies between domains. Generally, there is a shortage of evidence about which interventions work. You can find a lot of evidence on numbers, what is going on and what the problem is, but then you have to make a leap into what we are going to do about it. That is when you start running out of evidence pretty fast. Of course, that is partly the point of the What Works approach. What can we do about that?
There is then an issue to do with capacity inside the Civil Service—having the skills to do it. In principle, if you are doing policy, you can induce variations, for example, rather than going for a single solution. If you are doing a rollout, you can use something called a step-wedge rollout—you can randomise the sequence of the rollout—which will tell you whether or not something is effective. That can be harnessed to generate evidence, but traditionally our civil servants have not had those skills. We had a separation between our analysts and those who do the policy, which is a mistake. Chris Wormald, I and others are trying to do a lot about that, to retrain our entire policy-making profession to have those skills, and therefore generate the evidence.
Jill touched on the other point, which is not to be dismissed. There can be ambivalence about why you would want to generate evidence in the first place. It can look like all downside. If you are a Minister, you think, “Do I really want to pause to find out? Do I really, really want to know whether or not this thing worked? Am I going to find out in two years that my precious, superb idea did not work?” Even in a local authority people may think, “I’m going to have extra hassle to find that out. Who is going to benefit? Everyone else, but I, personally, will suffer some cost”. So there are some political factors, too. I do not think they are overwhelming. We now have a world where there are Ministers who know what a randomised controlled trial is, for example, which is an amazing development.
On the technology point, if you mean it in a very particular way about consultation, it can definitely help, but normally it brings with it the same issues. I alluded to the example of the consultation, or deliberation, on obesity that I held in Australia, in Victoria. That used technology very heavily. People received a lot of information and discussions were done online, which greatly lowers the cost, but you still have to crack who is the sample. Consultations are always dogged by that. Whether they are done online or in some other form, the people who come forward are definitely not a representative sample of the public. You must actively crack that issue.
The Chairman: Lord MacGregor, you wanted to follow up.
Q20 Lord MacGregor of Pulham Market: Before I ask my questions, I ought to explain why I was smiling wryly when Ms Rutter was talking. She was my principal private secretary when I was Chief Secretary to the Treasury. She was an absolutely admirable one and understands some of the issues she started to touch on. That is what I want to address. I can understand the points you are all making about well-prepared legislation over a period of time and so on, but frequently Ministers are faced with an immediate, urgent situation. The Dangerous Dogs Act was mentioned. I had to deal with that. I had to deal with BSE and a lot of health issues, where it was very difficult to get evidence within the space of 48 hours. Our officials did very well to try to find it. There was a huge public outcry, much debate and demand in Parliament for immediate Statements and immediate reactions. You have to deal with that. Is there a better way of dealing with it?
David Halpern: It is nice to have a Blue Peter “something I prepared earlier” in evidence terms. We can try in a serious way to anticipate the evidential needs and harness our academic communities to answer some of those questions. Of course you are right. Sometimes, even with the best will in the world, we will not have that. There will also be manifestos or Ministers who want to take a line on something.
Lord MacGregor of Pulham Market: That is the other one I was coming to.
David Halpern: The corollary is that of course we should try out new things, but when we do that, can we put in place a mechanism that will tell us whether or not they are effective? I go back to the humility point. There is a deeply interesting question about how it interacts with a matter of law. If you buy the argument about humility and uncertainty about effects, wherever possible you want at least to test, for example, variation in dosage or approach and to have that possibility in legislation, so that you are not locked into a fixed position for a national rollout, but have the capacity for learning in the system, to tweak and adjust practice going forward. In those situations, where there is an imperative to act, we should at least try to look at what mechanisms we put in place to learn and refine going forward.
Lord MacGregor of Pulham Market: What happens very often is that civil servants, who are in touch with all sorts of bodies, try to find the best experts as quickly as they can.
David Halpern: Yes.
Lord MacGregor of Pulham Market: But sometimes the thing comes at you out of the blue.
Jill Rutter: There is a really interesting question about trying to seize the space. This is something on which you will be much more expert than us—we are mere civil servants. Could you get to a stage when it was acceptable to say, “This is what we know. This is what we do not know now, and this is how we are finding it out. In the light of what we think we know now, we think this is the best approach”? I realise that interacts in a quite difficult way with the certainty of Parliament wanting to know what it is passing. It does not give Ministers a whole bunch of discretions to make up policy as new evidence emerges. What you really want is to be more adaptive and find ways in which, as new information and evidence emerges, you can come back and do something different, if your first go at it was not the best in the light of the emerging evidence.
One really interesting thing about the question of public engagement is whether we can take the public debate to a more sophisticated level, where it is okay to say, “This is not blindingly obviously the right answer that will work in all circumstances and that I will defend” or whatever. As David said, an evaluation that tells me that it was quite a good idea at the time, when in retrospect it did not work, is one of the worst things that can happen to me politically, because then I get headlines about wasting money by cancelling a programme and things like that. It is better to duck and hope that my successor in three months takes it on and makes the difficult decisions.
It would be interesting to see how you could have much better acknowledgement that Ministers have a really difficult task to do, which is to make decisions. It is one of the big disjunctures between academics and politicians. Jonathan might talk about this. Academics will always say, “On the one hand, on the other hand; caveat, caveat, caveat”. A politician says, “Yes, but I am in a binary world. I have to do or not do. That is the sort of decision I have to face”. Getting those two worlds to talk to each other better is crucial to the better use of evidence.
Lord MacGregor of Pulham Market: The other thing I was going to raise, of course, was the manifesto issue. Government is about politics. Politicians have their own views on what they think policy should be. They also respond to the public when working out manifestos. The manifestos contain a lot of policy proposals that are politically driven, in the long run. I know that the Civil Service is now trying to understand what is going to go into manifestos—
The Chairman: There will be a question on party manifestos later. Can we hold it until then?
Lord MacGregor of Pulham Market: Okay.
The Chairman: Thank you very much. I want to bring in Lord Pannick and Lord Judge quickly. Then we shall try to improve our productivity.
Q21 Lord Pannick: My question is a variation on the theme that we are developing. How often do you encounter Ministers or civil servants who say, “The evidence is ambiguous or unavailable, so I am going to go with my political instincts—my experience. That is why I am in politics”? Is that a recurrent theme? Are they ever right?
Jill Rutter: I will give you an example from my time at the Treasury. I used to be press secretary to Ken Clarke, who was an excellent Chancellor of the Exchequer. Basically, he never thought that the Treasury and the Bank of England understood at all what was going on in the real economy. I remember various debates about the forecasts going into the Budget. At the time, we did not have the OBR; we have now made it a much more technocratic process, subcontracted to Robert Chote and his friends. At the time, it was billed as the Chancellor’s forecast; it was a first draft, but it was the Chancellor’s forecast. I remember very strong debates when Ken Clarke would say, “I just do not believe what you tell me is going on in the economy. It does not feel like that in Nottingham”. He was probably more often right than not right.
That is one of the huge values of our system. We have talked about a specific sort of evidence, but we should not be too narrow about the sorts of evidence. Ministers, as constituency MPs, are a vital source of a different sort of evidence. It is Ministers as constituency MPs who say, “Something is going wrong with the Child Support Agency. This is making a lot of people really worse off and is not working as the legislation intended”. Optimally, you want a blending of that sort of experience with, “Do you have a hypothesis about why this might work? Is it really a general problem? Have we specified it right?”
Getting the problem definition right is a really important starting point for a lot of policy. We jump to say that there is a problem. When we were doing a thing called the advanced policy school and asked a bunch of officials in the Home Office what they thought was one of the biggest reasons why policies went wrong, they said, “Because too often you jump to the solution without understanding which problem you are trying to address”. A solution without a problem will never work.
A lot of civil servants, rightly, defer to the political judgments of Ministers. They are not trapped inside an SW1 bubble. They have a much stronger view of what is happening on the ground. That is vital input and the evidence agenda should not crowd it out.
Q22 Lord Judge: My question goes back to the problem of urgent legislation. There has to be an answer and there are 48 hours to do it. Is there anything to be said for using a sunset clause when you have to legislate in those circumstances?
David Halpern: Yes, there is a strong case for doing so. It is precisely the sort of mechanism that addresses the issue of putting something in place to figure out whether it was in fact effective.
As a small aside, when the Behavioural Insights Team was created in 2010, it was built with a sunset clause—not in legislation—that it would be disbanded in two years unless it was able to show at least a tenfold return on the cost of the team and so on. Frankly, the presumption was that it would not work. It turned out that it was very effective and we were able to show that, so the PM decided to double it rather than shut it down. In fact, the sunset clause could be used much more widely, but we need to link that to the kind of evidence that would determine it.
We should bear in mind that, if there is a blanket application of something and there is no variance, we may struggle to figure out whether it was effective at all. That is why the sunset clause is required—it is a means of ensuring you incorporate methodologically sophisticated tests of efficacy but then it starts to conflict with the presumption of a rival principle, normally expressed in terms of fairness: everyone should be treated the same. That makes it a lot harder to find out whether or not the thing worked. I realise that it would be as difficult to sit in a courtroom and say, “Your honour, spin a coin about the judgment that you will use on a given individual”. But you can see why, at the margin, you need a bit of that in the system to be able to answer the question. The answer is definitely yes to shutting down things that don’t work well, but we should note what we would need to do to be able to answer the question well.
The Chairman: We must move on.
Q23 Lord Norton of Louth: I have two questions, which derive from some of the points you have made. You made the point that you can take evidence and identify the problem, but you may not be able to get evidence to determine consequences. Now that we at least have post-legislative review three to five years after enactment, I wonder whether a partial answer to the point you identify would be to stipulate, when a Bill was introduced, the criteria by which we would know whether or not it had been successful.
The other question picks up on points that have already been made, particularly by Jill Rutter. We cannot discuss this with the politics left out. That point has clearly been made. There are going to be tensions. For some Ministers, the evidence is, “If we do not do this, we are going to lose votes”. There will be that tension. How do you resolve it? Where is the point at which you say, “I am sorry, Minister, the evidence is overwhelming that it will not work”? Do you at least put on the record that it will not work, if the Minister insists on carrying on with it?
Jill Rutter: Back in 2011, we wrote a report called Making Policy Better. One of the things we suggested—it has not been taken up—was that when the Civil Service thought that there was not a robust enough basis for proceeding with a policy for spending public money or requiring private resources to comply with a regulation, it should be able to seek what we called a policy direction. In the same way as you can get an accounting officer direction on value for money, regularity and feasibility, which was added as a criterion in the 2000s, after tax credits, by the Treasury, we thought that you could get the Civil Service to take more responsibility for saying, “Yes, we think that there is an adequate basis for proceeding”. Ministers want to proceed anyway. You do not want unelected civil servants saying to Ministers, “You cannot do that”. Ministers will always want to do innovative things. We do not want to constrain policies only to things that have been done somewhere else before. We want policy innovation, but when people think that there is not a good enough basis for proceeding, they should be able to seek a policy direction and put on the record that they did not think there was an adequate evidence base. As I said, that idea has not been taken up.
Much more systematic compliance with the commitments on post-legislative review would be inordinately helpful. One thing that changes a ministerial incentive set is when a Minister thinks, “Okay, I know that this is not really the right thing to do, but I can see short-term political convenience in it. What is the disincentive to doing that?” It is about being able to do things that say whether someone is acting in the absence of a respectable hypothesis. Things not working will happen, if you are innovating. Things that were never going to work and that you should not have tried, and wasted all our time and money on, are the ones you should really be called out for. If there is much more systematic knowledge that you personally, not your successor, will be asked to account for in a post-legislative review, there will be a tipping of the scales—a slight change in those incentives.
The Chairman: Mr Breckon has been waiting very patiently to get in.
Jonathan Breckon: One of the common misunderstandings we come across, particularly in our training of civil servants and others—charity leaders and people in local government—is about the term “evidence-based policy”. There is often a misunderstanding that evidence will trump everything else. That was never the model. The model was inspired by medicine and the very eminent, brilliant David Sackett, who died a couple of years ago. It was always a triangulation between professional judgment, the wider science and what you saw in front of you; it could be your patients, your constituents or your stakeholders. There is no easy way out of that triangulation between research, professional judgment and the stakeholders. Those models are everywhere. The police use them. Teachers use them. There is even something called the Centre for Evidence-Based Management that uses them. There is a lot of guff spoken in the management field and business, and they are trying to get more robust. So we must have that model.
I go back to the digital point. The question of digital platforms is crucial. I sit next to a digital democracy team, which I believe has submitted written evidence. The crucial element is not only getting the sampling right, as David said, but having deliberative forums where you can take people on a journey, particularly with research that is ambiguous and hard to deal with. There have been some evidence checks in Parliament that have opened up the review of evidence. There was one on the Department for Education, for instance. It was a great initiative, in that it was an opportunity for the public, as well as academics, to review the evidence around a particular government department. What happened was that a very passive online forum got captured by parents lobbying on things like term-time holidays, summer-born children, et cetera. I am a parent, so I know about that, and I am in these discussion groups, thinking, “Great, we have an opportunity to lobby them”. But they did not understand the evidence. Deliberative democracy is even more crucial in the complex area of research than anywhere else. We need to look at that.
The Chairman: Dr Halpern, do you want to add anything? It is not obligatory.
David Halpern: No, I will shut up for now.
The Chairman: Thank you very much. We will move on.
Q24 Lord Judge: I wonder whether we can go back to the pre-legislative stage—to the manifesto. The Government of the day always regard their manifesto as practically holy writ—except the bits they decide that they do not want to implement for one reason or another. My question relates to whether party manifestos and the policies espoused in them should be based on solid evidence. There are follow-up questions. Should they be, if they are not? Can they be, if they are not, for the reasons you were discussing with Lord MacGregor?
Jill Rutter: We have had an interesting exercise trying to do the evidence benchmarking that Jonathan mentioned, which Sense About Science is leading, using our joint evidence transparency framework. At one stage, we were thinking about doing departmental rankings. One of the things that civil servants said to us was, “You cannot judge us on that, because those are manifesto commitments. We do not use evidence on manifesto commitments”. What we have actually found from doing the benchmarking is something quite interesting: there is no uniform rule on manifesto commitments. When a manifesto commitment is, “We are going to try to meet this objective”, they are quite good at exposing an evidence base. The problem is when the manifesto commitment is to a means, not to an end. Then you jump instantly to that.
One of the problems of government or opposition is that you might tie yourself—particularly if you are not a governing party forming a manifesto—to very specific means without being able to discuss them with the people who will have to implement them, so that you end up with something. The Institute for Government works with opposition parties to prepare them for government. One of the pieces of advice we give is that they should set out strategic objectives, priorities and so on, but not tie themselves to very specific commitments, because they might find when they get into government that there is a better idea.
One assumes—rightly, I think—that Ministers come into government to do things that will have the effects they want. They should be interested in making sure that they are implementing the headline, flagship promises in their manifestos in the best possible way, rather than sticking to the absolute letter. The Cabinet Secretary has a figure that there are about 540 manifesto promises in the 2015 Conservative manifesto. I do not think many of the public could name more than three or four of those, so I am not sure how literally we should take that.
Some policies are just statements of values. On those, it should be clear that it is a statement of values and is not being introduced for an evidential reason—it is just because you think it is the right thing to do. If you just say that, you exempt yourself from evidence-based scrutiny.
Jonathan Breckon: We have real, practical experience. We did something called the manifesto check, when we looked at all the manifestos in 2015, including those for Plaid Cymru, the Greens and UKIP. I have been invited to speak at UKIP. We did them all, so we had very practical experience. The trouble with looking at the evidence behind them, in a very practical way, is that it goes back to transparency. One party—I will not name it—had a whole background literature that it was happy to give us, partly because it was worried about bodies such as Full Fact and others, which jump on parties and say, “Let us tease out the detail of their manifesto. What does that mean?” It was very hard practically to check the evidence behind the manifestos. It is such a political operation, even down to the date when they come out. We were trying to check them with the heads of policy from all the political parties, in a very consultative way, but it was really hard to do. I cannot answer your question in an evidence-based way, but we checked all the manifestos. We did it with a great website called The Conversation UK.
Lord Norton of Louth: On the manifesto point, presumably there is a particular problem if there is a coalition and post-election bargaining on what will go into the Government’s programme. There is a problem with testing and producing any evidence when there is compromise, rather than concession.
David Halpern: Exactly. There are normally some mechanisms that kick in. I agree with Jill in general. If commitments are specified as objectives, with a bit more wriggle room, it is better. Yes, you want to improve outcomes for young people, with New Routes. et cetera, but do you want exactly 3 million apprenticeships? Do you want a bit of wriggle room?
In 2015, there was an expectation that there would be negotiation, as that had happened the previous time, and a lot of things would drop. In many northern European systems, in particular, that is the way these things get hammered out. If we wanted, we could supplement it in other ways. There has always been a case for having a Department for the Opposition, where you would give them some access to evidence in another way. In the Netherlands, the equivalent of the OBR does fact checking and looks at the economics and the numbers beyond that, to see whether or not something is likely to be effective and at what cost. It is theoretically possible to put all those things in place. They are not that difficult. It might help to address your concern.
Jill Rutter: Effectively, however, it hands over the timing of Dutch elections to a bunch of Treasury officials. When we had an event where the Dutch CPB talked about that, even we thought that it might be a bit too far for us to go. Basically, it tells the political parties how long it will take to do the manifesto checks, and the date of the election follows from that.
The Chairman: Okay, we will not follow that example.
Q25 Lord Pannick: On the subject of transparency, is there any justification for government not to publish the evidence base for legislative proposals, absent some special factor such as national security or foreign policy?
David Halpern: It is hard to come up with one. I have been a civil servant. I am no longer a civil servant, although I am pretty close to being one in my day-to-day life. I am very much on the side that we should publish as much as possible. It generally helps in the medium term to set out what your case is. Can there be examples? Theoretically, there could be something that was commercially sensitive or whatever, but that is rarely the case. Indeed, the research community itself could go further. There is a strong case for saying, for example, that you should publish research protocols in advance, with what you think is going to happen, rather than adding your hypothesis retrospectively, after you have the answer and the evidence. I am quite ultra on that side. I struggle to find many examples of why you would not publish.
Jill Rutter: You are committing public resources, so you ought to tell people the basis on which you have decided to commit those public or private resources. There are relatively few exceptions. If it is a national security exception, at the very least you might want to talk to the Intelligence and Security Committee, on Privy Council terms, about the basis on which you are doing it. There is a good case for doing that.
Even more interesting is the extent to which you could divorce producing the evidence base from the policy adjustments. The way evidence is used in government is one you will be much more familiar with, David, as evidence in a court of law. I use the evidence to make my case and win my departmental battles, rather than necessarily giving a very objective analysis of what the optimal outcome is. It would be very interesting if on some of these issues—not the examples that Lord MacGregor gave of things that require immediate action, but on some longer-term actions—the Government put out the evidence base first and said, “This is what we think the problem is”. You then use that as the basis for thinking, “If we all agree that that is the problem, what are the potential ways we do it?” That is one of the things that the Turner commission did very well. Too often, we use evidence to make the case for a solution we have already come up with. We conflate that. The evidence might be a bunch of slightly cherry-picked things that support the case for doing X, rather than a proper consideration of what the problem is and what the choices are.
David Halpern: It used to be called “spray-on evidence”.
Lord Pannick: To what extent is there strong resistance within government to transparency on publishing the evidence base? Are we pushing at an open door?
Jonathan Breckon: It depends on who you talk to. Some are very supportive. We find some analysts, for instance, who say, “Fantastic. This is an opportunity. I have this evidence”. I was speaking to a very senior ex-analyst from the Department of Health who was desperate to get the material she was getting to Ministers into a wider public domain. There are others who find it politically awkward, so it depends on who you speak to.
I go back to your first question about the wider transparency question. You might find this too radical, but I think that we should be up front when there is no evidence and say, “That is a blank. This is a values-based belief”. We live in a democracy. That democracy should trump evidence, as regards my beliefs and my normative approach. Be up front. As we said before, the problem is the spray-on evidence, or sometimes just a mess when you cannot even find the evidence. I would rather be up front and say, “That is the blank spot. We are doing this, but let us not pretend that it has evidence behind it”.
Jill Rutter: A frustrating thing in the benchmarking exercise is that there are a lot of cases where departments say that they have reviewed evidence or whatever, but they do not share it with anyone else. You almost have to take it on trust that they have looked at the evidence and made the right judgment. Maybe, but if they have, they should be able to convince us that they have.
The Chairman: Does the Cabinet Office intervene in that situation, to find out more about the evidence?
Jill Rutter: That is a question for our What Works national adviser to the Cabinet Office.
David Halpern: It can. We have Cabinet government. Ministers can say, “What is your evidence?” and push for it.
The Chairman: I was not thinking of the Ministers. I was thinking of the civil servants in the Cabinet Office.
David Halpern: It is erratic; it depends. There is a degree of scrutiny that occurs in internal and official discussions all the time, where you might push on it further. That happens quite a lot. I think that in general we should get the evidence out there and let it be tested more generally.
One concern that people might have is that sometimes you reveal your hand very fully by saying that you have asked for evidence on the following thing, but generally I think we should do it. I would go a stage further; Jonathan alluded to it. My view is that Governments should set out the things that they do not know but would like answers to. Even stronger than putting out your evidence is to say out loud, “We know that we have a problem”, whatever it may be—drugs, for example—“but we are not sure what would work, particularly around intervention”. You can immediately see why Ministers and Governments would be hesitant to do that. That is partly the answer to your question. If you put out a list of all the things we do not know the answer to, the Daily Mail will make real mischief: “Oh my God, what do you know?” It is actually the right position to be in, to lay down the challenge to a wider research community by saying, “Come in and help us to answer those questions”, and to build a bridge.
The Chairman: Staying with the evidence theme, I bring in Lord Maclennan.
Q26 Lord Maclennan of Rogart: I would like to ask Jonathan Breckon a question. The Alliance for Useful Evidence has stated that “policymakers should use more systematic reviews”. Is there any evidence of standards or requirements that departments must meet in drafting or introducing legislation? If not, should there be?
Jonathan Breckon: There should be standards. At the moment, there are standards of evidence per se that are used in the wider research community. There is also advice from the Treasury in its Green and Magenta Books about the best ways of going about research, evaluations or cost-benefit analysis to make the case. When we do our training, I am amazed how few people in the Civil Service have heard about the Green or Magenta Books.
The problem with the standards of evidence, and how they impact on legislation, is that in the UK I have counted 12 formal hierarchies of evidence. They tend to have things like systematic reviews and randomised controlled trials at the top. At the bottom, it might be that you just collect a bit of data. They are a way of judging how trustworthy evidence is, particularly evidence of impact. There are 12 in the UK, with possibly a 13th coming along. That is excluding all the international ones. There is something called GRADE, or the Maryland scale for scientific advice. I am sorry to get so boring and technical, but it is important. It would be a nightmare for you to say, “We must have a single standard for evidence that we can apply for legislation”.
Where we should have some kind of standard is on transparency. That is doable and fixable. In our joint work with the Institute for Government and Sense About Science, we found that some of it would be undergraduate stuff. Just do your referencing. If you were an undergraduate, you would be failed—D minus—if you had not even shown the research that we know is available, let alone things that are harder, where it is hidden, obscured or cherry picked. We could have standards on transparency but not on the quality of evidence related to legislation.
The Chairman: Does either of the other witnesses wish to add anything?
Jill Rutter: For some legislation, the Government require impact assessments, which are often produced at a quite late stage, to be submitted to the Regulatory Policy Committee. It is only a partial standard, because it applies only to regulation that affects business or civil society. It does not apply to spending and things like that.
The other place where you might look for challenge is the Treasury. The Treasury should be satisfying itself, on big proposals anyway, that they represent value for money. If there is not a decent basis for thinking that something will deliver the effects it is supposed to have, it is difficult to see how the Treasury would not challenge that. Of course, the Treasury does not always win its battles on these things, as we know.
David Halpern: For lots of reasons, including the correct role of politics, we often make a leap with relatively little evidence. We could at least make progress on transparency, to show our workings in a way that is testable, so that other people can examine them, using the mechanism we touched on before. That is fine. If you are going to make a leap, that is okay, but what mechanism did you put in place to see whether it was effective? That seems to me a reasonable aspiration to have for government.
Lord Maclennan of Rogart: I would like to know whether there should be some follow-up to transparency. Should there be a civil servant’s concerted view about the lack of evidence, if there is complete transparency?
Jonathan Breckon: There should be a good nudge. What is reassuring is that departments themselves and other parts of the government system have been taking up almost unilaterally, or have found very useful, the transparency framework we have been working on with Sense About Science and the Institute for Government. If I have understood your question correctly and it is about whether a civil servant should push it and follow it up—
Lord Maclennan of Rogart: No, it is about responsibility for following up.
Jonathan Breckon: Who should have responsibility for that? I am not sure, in government. I am outside government, very deliberately, so I would like to be that nudger. Internally, could you have a tsar for transparency? David, could you imagine such a role?
David Halpern: If government really wanted to do it. We have done it in other areas. There are other ways. The natural place might be the chief scientists, who are now a fairly mature network. We might say that we should ask, first, what was the evidence on this particular issue, and secondly, have we been open about it? At the very least, it is reasonable for departments to say what their position is on transparency, and to answer the question that was asked earlier.
The Chairman: Lady Dean, you wanted to come in briefly.
Baroness Dean of Thornton-le-Fylde: I have a quick question. Everything we have been talking about has been pre-legislation. Dr Halpern, in your opening address you referred to the fact that sometimes legislation is passed that is not necessary or does not work. There have been instances, of which universal benefits may be one, where the Government tried something out—I call it piloting; you probably have a different term—in one or two areas of the country, which influenced the final application of the legislation. Do you think there is an argument for doing that more frequently—not in every case perhaps, but for substantial changes on issues such as welfare benefits and universal benefits and a whole host of things?
David Halpern: I clearly do think that. My role in government, as national adviser on What Works, is partly to champion that approach. You mentioned benefits. Lord Freud has been a great and passionate advocate of more empirical methods. When he pushed through some of the reforms, not only did they try to get evidence before, where they could, but they even put into legislation the capacity to carry on experimenting. That is exactly the right place to be.
Could you do it in lots of other areas? I will give you a concrete example that Oliver Letwin got very interested in. It is a small one, but it is illustrative. It concerns licensing arrangements for small outlets; the example often given is a community centre or a church hall. Do they need to get a big licence to serve a couple of bottles of wine? The question arose, why could we not test it? Why could we not test variations in the legislation? If there was a very light touch—in fact, if you could just say online, “I am going to do the following thing”—would that be enough? What if there was nothing at all? Could we then track the number of drunken folks in the region coming out of the church hall or whatever accordingly?
We rapidly get pushback, especially from the legal establishment, which finds it slightly abhorrent that we would deliberately vary practice, of law in this case. In principle, it applies to law or, indeed, any other practice. We might want to do that in order to establish efficacy. My own view is that we should absolutely do that. When people have objections, we should remember that that is how we made progress in medicine. When it is literally life or death, we have been prepared to vary practice to find out whether something is marginally less or more effective.
Q27 Lord Hunt of Wirral: Dr Halpern, could you tell us a bit more about the What Works centres? I have great respect for NICE, but I had not realised that it was a What Works centre. Could you tell us how it has been built up and to what extent it could be extended?
David Halpern: I will be delighted to do so. You are right. NICE was created in 1999. Retrospectively, we now refer to it as a What Works centre, because that is what it does—it says that on the can. It does not instruct clinicians what pills to give you, but it gives you some sense of relative efficacy. It rather raises the question posed by Archie Cochrane in his famous tome in 1972, when he said, “I have been critical of my medical colleagues, but look at everyone else”. How many judges, teachers and so on are running randomised controlled trials? You might say, “It’s a fair cop”. Why do we think that when it is a matter of life and death we will test and experiment, but we will not do it in relation to education and so on? If that is the question, the What Works centres are the attempted answer.
In the last five years, we have constructed a number of them. They vary in their design, but basically they are in the business of stimulating the generation of better evidence. They are almost always focused on a given profession or group. They are also about transmission—putting evidence in the hands of that profession—and building the absorptive capacity to understand the difference between good and bad evidence. Education is possibly the best example. The What Works centre in education is called the Education Endowment Foundation. It had its fifth anniversary yesterday. The background is that perhaps once a decade in education there was a good-quality trial of an intervention. Those are probably the numbers we are talking about.
Since 2011, there have been 127 large-scale randomised controlled trials in education in the UK, involving more than 7,000 schools and nearly 1 million children. For the first time, we are starting to be able to answer these kinds of questions. What is the most effective way of teaching maths? Are school uniforms effective? Are teaching assistants effective? By the way, it has thrown up lots of incredibly interesting surprises. School uniforms turn out not to be very effective. Teaching assistants, on whom we spend £4 billion a year, look as though they are not especially effective. On the other hand, teaching an hour of philosophy to a young disadvantaged kid improves their reading, writing and maths by the equivalent of an extra term of school at age 11. Who knows what it does for their philosophy?
You will never find the answers to those kinds of questions sitting in an armchair or reviewing the existing variants; you have to do what NICE has done. We now have centres covering education, criminal justice, local economic growth, well-being, better ageing and early intervention. Some of them are smaller. The DfE is in the process of producing one for children at risk as well.
Lord Hunt of Wirral: What about older people?
David Halpern: We have one called the Centre for Ageing Better, which is funded by the lottery.
Lord Brennan: We are looking at parliamentary scrutiny of evidence-based policy arrangements. The Government make a legislative proposal, Parliament considers and enacts it, the Government execute, and thereafter nobody seems to have an adequate check, either in Parliament or elsewhere, on what has happened. The crucial step in that process is Parliament, without which the event would not occur. If transparency aids competent legislation, what do you think, for example, about having a Joint Legislation Committee of Parliament to which, after a Queen’s Speech, the Government should present pre-legislative proposals in White Paper form, with an evidence-based annexe or whatever it might be, to avoid Committee-stage consideration where, without transparency or adequate evidence-based policy information, you have a lot of uninformed and untestable opinion being used by everybody as a secure basis for legislating? That does not seem to work. My last point is, what happens afterwards? In any other walk of life, with matters of such consequence you would have results-based analysis, so why not here? What can Parliament do?
Jill Rutter: I would like to start. We have been doing a project jointly with the Chartered Institute of Taxation and the Institute for Fiscal Studies on making tax policy better. Tax policy is almost an extreme case of some of these issues, because of the dynamic of the Budget process. One of the things that we are thinking of recommending is that there need to be evidence sessions in the Finance Bill Committee on what measures are trying to do—what is the intent and purpose, and how have the Government decided that they are the right ways of trying to do things? Tax policy is one of the prime cases where the purpose behind the measures that the Government introduce is often extremely hard to discern.
You are absolutely right. A pre-legislative stage that said, “What is the evidence base for the problem you think you are addressing and for the particular proposals you have?” would be enormously helpful in guiding the subsequent scrutiny stages. It would also act as a real incentive to Government by saying, “Get your act together before you bring the proposals”. If you know that you are going to be asked, it concentrates the mind wonderfully, to make sure that you have a very good story to tell and have worked out where the weaknesses are. It would be very useful to bring in external witnesses as well.
On a lot of ongoing policies, we have seen some initial steps in evidence-check exercises. Jonathan mentioned the Select Committee on Education, which did one yesterday on grammar schools. It got in academic experts to give their views on the evidence base behind selection; it brought in Nick Gibb, with two chief analysts at the Department for Education alongside him, to question them on the evidence base behind the Government’s new proposals on education. That is a good use of Parliament.
I am not sure whether you want to sunset absolutely everything, but you could have a clear, fixed-point review when Parliament says, “We expect you to come back”. There will be different periods for different sorts of legislation, but you could say in the legislation, “This will be subject to a proper post-legislative review at this point”. Maybe a really good role for a Joint Committee would be to look at post-legislative reviews.
There have been one or two developments on tax policy. The work that the House of Lords Economic Affairs Committee now does in looking at some of the draft clauses of the Finance Bill and saying, “What are these trying to achieve?” seems to be quite a lot of progress compared with the previous experience of parliamentary scrutiny of Finance Bills. The more Parliament can show that it is interested in the evidence base, the more Ministers, who very often take their cue on what is important from Parliament, will be encouraged to think that it really matters to them and their reputation. So Parliament has an absolutely vital role to play.
Jonathan Breckon: It is a terrific idea. The Parliamentary Office of Science and Technology is reviewing the current use of evidence in Parliament. The review is not ready yet; I tried to get an early copy so that I could share it with you. It is looking at the current practice, but there may be some recommendations about additional institutions or ways of doing things. The pre-legislative stage would be a great one. The follow-up would be very useful as well. A really practical example happened outside—it happens to come from the Institute for Government. At one of its policy reunions, it looked at things like the Sure Start centres and children’s centres, which have had cross-party support, but it was crucial to learn how evidence had been used, and in some ways quite distorted. It was very much an evidence-based programme to help in the early years. Lots of people will be retiring and leaving, and we need to capture the lessons learned, even though it is years after the fact. It would be fantastic if that were a recommendation from your Committee.
David Halpern: I have spent a long time in No. 10, under more than one Prime Minister. It is pretty clear that sometimes you introduce legislation because you can do it and it is a way of creating an intentional spotlight. You then hope that you will figure out the detail when you are in flight, as it were. There is some logic to doing that, because you want to call attention to something, but you need to make sure that there is scrutiny.
We are talking about evidence very broadly. One of the key points is that evidence is not like democracy; not all evidence is equal. We need to move to a world, both inside Whitehall and in Parliament, where people are able to discern the subtler differences: how good was the method that gave rise to the evidence? Randomised controlled trials are by no means the only good tool, but the reason why they are often cited is that they are a particularly powerful form of evidence around interventions and are very rarely used in most fields. That is at least a tell-tale. It will not be a bad thing for Ministers to live in a world where they are occasionally prompted about the quality of the evidence, not just about whether it exists at all. It has ramifications through the system. When Ministers start demanding it—when they come back from Parliament or an interview to the Whitehall department and say, “What the hell was my evidence?”—it ricochets throughout the system.
The Chairman: We have already covered technology, so I will move to a question from Lord Norton.
Q28 Lord Norton of Louth: In its 2004 report on Parliament and the legislative process, this Committee proposed a Joint Committee on post-legislative scrutiny. I want to come back to the other end—the input side—to look at whether there should be generation of more coherent legislative standards. A few years ago, the Leader’s Group on Working Practices in this place suggested that there should be a Legislative Standards Committee. Of course, the Better Government Initiative variously put in evidence and recommended that there should be the generation of legislative standards covering the preparation and presentation of Bills. Do you think that would be desirable and achievable?
David Halpern: I have to defer to your knowledge of those procedural matters, which is much greater than mine. I spend a lot of my life trying to avoid legislation, as I explained. There are enough problems outside legislation that we can try to bring evidence to. One thing I would say is that it does not have to take years and years. Sometimes it is possible to build this in real time.
To give a concrete example, in the US there has been a lot of interest in the specific issue of food labelling—whether restaurants should be required to have calorific information down the side of the menu and so on. Superb work has been done by Christina Roberto at Harvard, literally in parallel to the drafting of legislation, on running trials to see whether or not it makes a difference. Incidentally, the result was that it depends what side the numbers are on. Whether they are on the left or the right makes a massive difference as to whether they have any impact on behaviour, because people just happen to look at one side in particular.
The point is that it is possible to live in a world where we can introduce evidence at lots of stages. We should seek to do so wherever we can. Let us do it pre, during and afterwards, where we can. As to the best mechanism, I feel that I need to defer to those who are more expert than I am.
Jill Rutter: It may be a good approach. I never did much legislation, because I came from the Treasury. The Treasury is now the top legislator in government, but when I was there we did not do huge amounts. There is always a danger with anything that it might degenerate into a tick-box exercise. We have seen that with impact assessments, to an extent; you make all your policy decisions and then commission the impact assessment from a bunch of outside consultants at the end of the process, because you know you have to do it. I have done things where I have required people to fill out sustainability checklists. You know that it is done right at the end of the process, rather than what you really want, which is people regarding it as an integrated part of the conversation.
There is one thing that Parliament could very usefully do. It may be completely unrealistic politically, but we need to get into the notion of good and bad failures, which I think comes from Tim Harford’s book Adapt: Why Success Always Starts With Failure. There are good failures, when something was really worth trying, was quite well founded and there was perfectly good reason to believe that it was worth doing, but it just did not turn out like that and we all learned from admitting the failure. That is opposed to the bad failure, when it was never going to work, should never have seen the light of day and should never have got anywhere near Parliament.
It would be enormously helpful if we could find a way of giving Ministers and civil servants more space to fail well and learn from that, rather than everything degenerating into stuff like, “You are a bunch of incompetents wasting money”. The House of Lords might be a better place to try to do that. We do not want government that is extraordinarily limited in its ambitions, because we will never find out new things. It would be a terrible thing if we said, “You can only do things if you can meet all these tests now”, because we learn a lot from experimentation.
We can argue about the consequences, but if you had asked for evidence on council house sales in the 1980s under Mrs Thatcher, would you have found any? Evidence for what? I do not know. Another example is privatisation. Governments do big things. We sometimes want Governments to do big things and to try things. We do not want to be a block on progress.
Jonathan Breckon: I am not sure whether having some kind of evidence standards to go with your standards of good legislation would work. They would work if they were about transparency. More important is the point about absorptive capacity—intelligent consumers helping and working with you and all your staff and colleagues to interrogate what is untrustworthy and what would be useful. That is not easy. Tick-boxing standards might not be the best way.
Lord Norton of Louth: I take that point. On Jill Rutter’s point, one of the recommendations of the Better Government Initiative for what would be on the list would be to look at risk. You would be encouraging the Government to reflect on that. It would not just be a tick-box that they had looked at risk. You would be asking, “What is the assessment of risk?” There is no harm if they recognise that there is a risk but there may be a case for proceeding.
Jill Rutter: I have argued that there is quite a strong case for the Government to publish their risk registers. If there are no risks on your risk register, you are asking the wrong questions. A much better conversation about where the uncertainties are, where the problems might arise and stuff like that would help everybody to do things better, at the end of the day. A lot of it would probably point to doing a bit less as well. That is certainly true on the tax side. We think that doing less would be a key early step towards doing better.
David Halpern: I would make the case strongly, as we certainly do with Ministers when things come to Cabinet, to look for every opportunity to induce some variation. You say, “Minister, that is such a great idea. Why would you do only one variation? Let us try five”. Potentially, you could do that in Parliament and look for opportunities to say, “Do not do one”. There is a concrete example with right to buy. One of the issues we were looking at in discussions on how many more billions we should spend on the subsidies was that the forms are written in legislation. Literally, the forms that you have to fill out are written in legislation. There are lots of very talented people here, but are you sure that you got the best possible variation? It is unlikely. Why on earth would you have locked that down?
The Chairman: I am afraid the clock is against us, and there we must finish. We knew that we would benefit from hearing from you three today. You have not disappointed us one little bit. It has been fascinating. We are very grateful to you for sharing your insights and experience with us. It has contributed enormously to the work that we are doing. Thank you very much. If you have nothing else to do later today, I suggest you give Washington a telephone call—they might be glad of the contact.
