Work and Pensions Committee 

Oral evidence: Effectiveness of sanctions, HC 1308

Wednesday 10 May 2023

Ordered by the House of Commons to be published on 10 May 2023.

Watch the meeting

Members present: Sir Stephen Timms (Chair); Debbie Abrahams; Shaun Bailey; Siobhan Baillie; Neil Coyle; Steve McCabe; Nigel Mills; Selaine Saxby; Dr Ben Spencer; Sir Desmond Swayne.

Questions 1 to 55

Witnesses

I: Dr Patrick Arni, Senior Lecturer at the School of Economics, University of Bristol; Dr Serena Pattaro, Research Fellow, School of Social and Political Sciences, University of Glasgow; and Tom Waters, Associate Director, The Institute for Fiscal Studies (IFS).

Examination of witnesses

Witnesses: Dr Patrick Arni, Dr Serena Pattaro and Tom Waters.

Q1                Chair: Welcome, everybody, to this meeting of the Work and Pensions Select Committee and for an evidence session on benefit sanctions. We are very grateful to the three witnesses who have joined us for the panel this morning. Could each of you tell us very briefly who you are, starting with Patrick Arni?

Dr Arni: Thank you very much. I am a senior lecturer at the University of Bristol. I am a labour market economist and my clear focus of work is active labour market policies and job search behaviour. In fact, I was a bit involved in consultancy for the DWP for this report.

Dr Pattaro: Good morning, everyone. I am a research fellow at the University of Glasgow and part of a research team within the Scottish Centre for Administrative Data Research, which is part of Administrative Data Research UK infrastructure. I am the lead author of the first scoping review of the international quantitative evidence on the impacts of benefit sanctions, which looked at labour market outcomes and wider outcomes. I am also part of a research team at the University of Glasgow. We still hope to receive data from the Department for Work and Pensions to assess the impact of benefit sanctions on health outcomes, including deaths and hospitalisations, but also—let’s say, more generally—the use of health services in Scotland.

Chair: Thank you very much. I am going to ask you about that particular exercise in a minute.

Tom Waters: I am an associate director at The Institute for Fiscal Studies where I work on labour market policy and tax and benefit policy.

Q2                Chair: Thank you all very much. I will put the first question to each of you. What do you think benefit sanctions should be designed to achieve?

Dr Arni: I think the core purpose of a benefit sanction system should be in fact to support compliance of individuals with the rules, to establish the conditionality principle that has many of these labour market policy systems. It should be mostly a tool for enforcement of these rules to establish a good behavioural collaboration.

A second purpose is to also ensure what we economists call to cope with moral hazard. That means that if we are in welfare systems, supportive systems, there is a certain tendency that because of the insurance support people may deliver smaller amounts of job search effort, of engagement in the system as they would without benefits. Therefore, I think we have these supportive measures in that case—the stick measures if you wish—that would ensure an additional level of commitment to the research effort and to cope with the rules of the conditionality system. I see that as the key.

Q3                Chair: Thank you. In the UK system, as it stands at present, to what extent do you think those purposes are achieved?

Dr Arni: In general, they are achieved to quite a good level. If I do some comparisons across other countries in Europe, in general the UK has quite a good balance between what I called before the carrot and the stick—so the supportive policies and the restrictive enforcement policies—and also in comparison to others a relatively systematic monetary controlling system about requirements to search for jobs to engage with the system.

In that sense, I think it is quite a good balance. I guess we will discuss later on the details of the design of the sanctions but, in general, I think the main ingredients to have well balanced conditionality systems are there.

Q4                Chair: Dr Pattaro, what would you say the purpose of sanctions are and to what extent are those being fulfilled or achieved at the moment?

Dr Pattaro: I reiterate what my colleague Patrick Arni has said. One of the core purposes of a sanctions regime is to ensure and encourage people to move back into work. Also, we need to establish whether the right balance of measures is in place, the restrictive measures versus the supportive measures are in place to ensure that wider outcomes or negative outcomes that can be engendered by benefit sanctions are not reached.

We need to focus on the right balance and to establish that we need to bring in the wider outcomes. By wider outcomes, the harmful consequences that benefit sanctions may have, for example, in material hardships, financial problems or food deprivation or food insecurity, but also the detrimental health consequences that sanctions may have not only for adult claimants but also for their children and family members. Therefore, I think that we still need evidence. Perhaps my colleagues may agree on this, that we still need to increase the evidence on this.

Tom Waters: I will say simply that if we are going to have a benefit regime that has job search conditionality, where we require people to search for jobs to get benefits, there is a question about whether or not that is the kind of policy we want to have. Taking that as a given, something like a sanctions regime is needed to enforce that.

The downside that has been highlighted by Serena is that you risk causing low levels of living standards, potentially very low levels of living standards for a period for claimants, so I think that is the fundamental trade-off in setting sanctions. You can tighten sanctions policy and that enforces a conditionality regime more strongly but you risk quite low levels of living standards for a period for claimants.

Q5                Chair: How do you see the current balance in the system?

Tom Waters: To a substantial degree, I see that as being a political trade-off about how much one is concerned about low levels of living standards versus getting people into work and so on, and so I think that is a kind of political judgment. As Patrick was saying, it is helpful to look internationally to compare what other countries are doing to get a sense of scale.

Q6                Chair: We will explore that during this session. Dr Pattaro, the 2018 Work and Pensions Committee report called for research on the impact of sanctions on finding work and the level of earnings—and that research was carried out and has now been published—but also research on claimants’ wellbeing. You touched on this in what you said a minute ago, because you were for some time in discussion with the Department about carrying out such a study. Can you tell us what happened about that?

Dr Pattaro: Yes, I can tell you a little bit about the development or the timeline for our project. We have been in discussion with colleagues at the Department for Work and Pensions for quite some time, since 2013, through our engagement with colleagues from the Department for Work and Pensions to basically establish the research design of this project.

In fact, this project was meant to address some of the key areas of priorities for the Department. That is, looking to investigate more comprehensively the impact of benefit sanctions on health outcomes, including deaths by suicide and other causes, but also hospitalisations and GP prescriptions, for example, for antidepressants. It is looking at health outcomes, including mental and physical health conditions.

We have been in conversation for quite a long time and we have obtained a set of permissions, in principle, that were in place. For example, we obtained permission from the Scottish Public Benefit and Privacy Panel for Health and Social Care in July 2017. We also obtained permission from the UK Statistics Authority’s Research Accreditation Panel in February 2020, and ministerial approval from the Department for Work and Pensions in April 2018 under Esther McVey.

This ministerial approval was then subsequently reversed in July 2022, due to the fact that there has been a change in the legal landscape within the UK, with the introduction of the Digital Economy Act, for which there was a need to establish the legal basis for data sharing of administrative data in this area. That mean that basically our permissions lapsed and we had to present an application again.

Q7                Chair: You are saying that last July the Department said it was not going to provide the data?

Dr Pattaro: It wasn’t going to go ahead because ministerial approval was denied.

Q8                Chair: There was a press report that I was quoted in, in March last year, and that included a quote from the Department saying, “We agreed in principle to release the sanctions data to researchers but this required formal accreditation” and so on and it said at that time, “We are now considering the data request” but the Department has subsequently said no to you, has it?

Dr Pattaro: At that time I believe colleagues were referring to the fact that because there has been a need to obtain accreditation for all the data infrastructure within the UK, the Scottish data infrastructure needed to obtain accreditation through the UK Statistics Authority. That was achieved subsequently. I can provide the exact date. Everything is now in place, but we have been denied—

Q9                Neil Coyle: You were denied it last year, but have you gone back and asked for further permission at all?

Dr Pattaro: We are still in communication with colleagues because we have other projects that are going ahead in less controversial areas. We have not been advised to apply immediately. We were advised to wait for a better climate, let’s say.

Chair: That is very interesting. Thank you very much.

Q10            Dr Ben Spencer: I have a couple of questions about the methodology of the report but, before going into those, I want to ask you a bit about what you make of the DWP’s response to it and what it said about some of the limitations or the strengths of the report. I don’t know if you have seen the response from DWP, but what is your take on it?

Dr Arni: I guess you are referring to the context notes that were published together with the report. I can comment a bit on the—

Dr Ben Spencer: Yes, the key paragraph, as highlighted I think by you and Tom Waters: “The difficulty in fully and robustly evaluating the impact of sanctions, the draft report’s methodology cannot fully account for differences between sanctioned and non-sanctioned claimants that affect their employment and earnings prospects, nor does it account for the deterrent effect of sanctions”.

One point seems to be: what are the unknown variables that haven’t been measured as part of this study and what effect is that having? Another point is the systematic differences between the non-sanctioned and the sanctioned population, although interestingly enough in the draft report it doesn’t have confidence in the key values on those two groups, or absolute numbers for that matter, so it is difficult to know if those differences are systematic or not. Nor does it consider the impact of sanctions in the round, which isn’t measured by that. It is responding to those three points.

Dr Arni: I think there are two components in that one. One is a purely methodological one that you mentioned, so the question of the perfect comparability of the sanctioned and the non-sanctioned individuals, and I agree with what is written there. With the scientific standard of that kind of frontline research in that area there you would go a step further methodologically than what they were able to do to model this so-called unobserved heterogeneity, which helps to establish a better comparability between the groups of the sanctioned and the non-sanctioned. We can still assume that there may be a certain unobserved difference in, for example, attitudes, behaviours among these two groups.

We see it already when we look a bit into the social demographics of these groups. They are not the same, so it is not random who gets sanctioned, hopefully. That is clearly a step that is missing. Now, you may ask how this biases the results, probably. There of course I cannot tell in detail because we need to run these methodologies but we know the tendencies from other such studies, so we have done these comparability checks in the context of other studies in work that I have done.

We see so-called negative selectivity, which means by the fact that this unobserved heterogeneity was not fully controlled the first group of effects—so on exit out of Universal Credit or out of unemployment spells, the effect may be a tendency to underestimate a bit. If we correct it further this effect may be a bit more prominent.

The question then is we observe in that report two destinations, so whether people exit the unemployment spell into earnings or whether they exit without earnings. How these two distinguish that is hard to say without really modelling that part. Then with the second outcome on earnings we would rather expect a bit of difference; the opposite, in fact—that the slightly negative earnings effect that we find in the report here could also be a bit overstated towards the negative. If we further correct there, the negative earnings effect may be a bit smaller.

Maybe a last note on that, I think that the earnings effects—also from the experience from other studies—may suffer a bit less from this comparability issue because, as we have seen with other data, the amount of control variables that they are using in that study was quite extensive about the past employment and past earnings of the individuals. From experience from other estimations, that helps quite a lot when we then look at earnings, so post unemployment earnings. I would expect there a smaller bias as compared to the exit rates.

That is one component. The other one goes a bit beyond the methodology but it also refers to this note that you mentioned. There are quite a few dimensions that could not be covered by this report. The key one mentioned prominently there is the so-called deterrence effect of the system. That refers to my statement before that, in fact, that would be the main goal, that in an optimal system only the deterrence effect would do the job. There are some methodologies that would allow an approximate estimation of the impact of that deterrence effect. We did that in a study in Switzerland, for example. There is also an example in Germany. That was not possible in that report. Purely from the coverage of the effects I would say this is in fact the most prominent effect, probably, in the sense that the deterrence operates on all individuals who are under conditionality, by the fact that they are in the system the deterrence effect should help them to comply more. That would cover everyone, whereas the sanction in position effects that we have in this report focuses on a few percent of people who really get sanctioned.

In that sense, just comparing who is affected, the deterrence effect has a big importance and that part is missing, and that is also the core of these statements on limitations that this is not covered.

Dr Pattaro: I will add to what Patrick has mentioned here that, yes, there are limitations in the report but overall, according to the evidence that we reviewed in the scoping review study that I mentioned at the beginning, the study provides an overall reasonable assessment of the labour market outcomes. I say this because, let’s say, there is substantially an agreement between the evidence that was reported in the study by the Department for Work and Pensions and the evidence that we found.

For example, although we do not find in the international qualitative evidence that sanctions lead to slower employment rates, our evidence shows that, for example, we have, as Patrick has mentioned, longer-term negative effects of people moving into lower quality jobs, so characterised by lower earnings. This is in evidence. We also find mixed evidence of negative impacts but also no impact, so zero impacts in the area. This is probably reflecting what Patrick mentioned earlier.

We also saw increased rates of transition into non-employment and economic inactivity. That is also reported in the study by DWP colleagues. Overall, although there are some methodological limitations, the study provides a reasonable assessment. We should not regard it as misleading.

I think that to determine the contribution of the deterrence effect—I go back to my previous point—we need to consider the longer-term impacts that the sanctions have, which were not, from my understanding, investigated in this report. One of the report’s main limitations is looking at the short-term impacts, so within three to six months post exit from benefit sanctions. We really need to look at a time horizon that is beyond the short-term impact. In my view, it is still an empirical question as to whether the deterrence prevails, and so we really need to look beyond and bring the wider impacts into the picture. Thank you.

Tom Waters: I will say two things. DWP was directly trying to measure the effects on those who get sanctioned. Basically, what it is trying to do is compare those who get sanctioned to those who don’t. For this kind of approach to work, you need it to be the case that those who were sanctioned and those who weren’t would have had the same employment outcomes had there been no sanctions imposed. The question is whether or not that is a likely assumption to hold and the answer is that it is not a likely assumption to hold because anything that might affect whether you get a job and whether or not you get sanctioned will cause a problem.

For example, if one claimant does fewer job searches than another they are more likely to be sanctioned and they are less likely to get a job. However, in the DWP study that looks like just purely the effect of being sanctioned, not the effect of doing fewer job searches. Similarly, if you have a child who gets sick, that might limit the number of job searches you can do. It might also make you more likely to get sanctioned because you might not be able to turn up for the interview at the jobcentre. In the DWP study that just looks like the effect of being sanctioned.

The international evidence we have, certainly on average, shows that being sanctioned increases the likelihood of getting into employment. I think the comment you read out from DWP is correct, that it is unable to control for these sorts of factors, and so I do not think that in that sense the study tells us much that is particularly helpful.

Then in the wider effect of the deterrent effect of sanctions, I also think the comment from the DWP is right. The threat of sanctions will affect people’s behaviour, even for those that never get sanctioned. Again, there is wider research on that that generally does show a positive impact on employment. This comment that you read out from DWP, which I haven’t come across before, essentially summarises quite well the problems with the actual report.

Q11            Dr Ben Spencer: Thank you. Building on that, and mindful that this is a draft report—it says that in the title that we have managed to get out of Department for Work and Pensions, with much effort and trying—having looked at it and, notwithstanding the methodological problems, do you think this would survive peer review, from your point of view?

Are the methodological problems, given that you must peer review literature in this area, so problematic that you think it needs substantial changes? With your professional peer review hats on, what would you like to see different in this report? I assume it is because it says on the title “draft report”. What would you like to see different in it?

Tom Waters: I don’t think it would survive a peer review, at least by a reasonable standard social science journal. It is mentioned somewhere in it—and Patrick was kind of alluding to this earlier—the possibility of modelling the likelihood that someone gets sanctioned. There is a way of doing that that can lead to a robust estimate.

There is a footnote about it, which says something like it might take months to do or something and so they have not done it. If they did that that would be a very big improvement and that could well make it a robust study, in my view.

Dr Pattaro: I will reiterate the fact that there are limitations in the study but I think there are limitations with other similar studies in the area. Even with the studies that apply more robust methods, because most of the studies in this area rely on strong assumptions, larger sample sizes is achieved by using linked administrative data, but it is also the use of strong assumptions that may address the issues with alternative explanations that may explain the link between sanctions and the outcomes that are observed.

Overall, I reiterate that there are limitations but, as with many other studies in the area, the study needs to be improved. The authors have identified the areas that need to be improved themselves, but overall because we see the study reporting largely or being in agreement with the evidence that we find from other countries in the areas, I would not disregard it as misleading. It needs improvement but, overall, the results are in the ballpark of what we see from international studies, I would say.

Dr Arni: I agree with both components that were mentioned. Indeed, if we were to go for peer review for a scientific journal, such as The Economic Journal, it would not pass because it is in too much of a draft state and we would need additional methodological improvements or further steps.

However, I also agree with what Serena said. I would not disregard the evidence that we have. I see that there is a certain bias on the size of the effects, but as a more qualitative picture of where the effects are going, it is still very valuable if we want to investigate the sanctioning position effect. In that sense, I would not put too much face into the exact quantitative levels of the effects that are reported, but I think that it is valid as a first step to see what these effects look like in tendency.

Another note on that: I mentioned at the beginning that I did a bit of methodological consulting for the team who implemented this report at DWP. That was in the level of about two or three days when I met with them to talk about implementation of the report. I must say I think that they carefully implemented it given the conditions they had. They only had a restricted amount of time. They would have needed more collaboration with other academics and a larger project to be able to go a step further to do the rest of the methodology, as Tom mentioned. I think that the way they did the basic modelling and treated the data and so on looks reasonable to me. Therefore, I would not disregard these results but take them with a pinch of salt because of the quantity.

Q12            Dr Ben Spencer: Thank you. That is really helpful. To preface my final question, my questions about the peer review component were not intended to cast aspersions on the people who put in the hard work to put it together.

As we all know, the peer review process is important to improve literature to get the best outcomes going forwards and the most robust evidence. That is the bit that sits uncomfortably with me because, given the limitations of this report that we have discussed, the question is how can you draw its conclusions if you take it with a pinch of salt? That causes a real problem for policymakers because we are now going to make meat out of this in deciding what is good or bad in a very binary way regarding sanctions regimes and deterrence. However, you are saying from the start to take this with a pinch of salt because we did not get through to the scientific level.

I personally think there is a normative level of having deterrence and sanctions regimes that is separate from the efficacy of those detentions and sanctions regimes. I think that it is right that people who are healthy young men without kids—if we take this at face value—who seem to be the group more likely to be sanctioned from the start, if that is true, because there are no confidence levels. You wonder what is going on there and what is going through the system.

My concern is that if we are to make meat out of a report like this, it feels like it needs to get through the process so it can stand on both of its legs under scrutiny. What are your thoughts about using draft reports to start making meat about policy?

Tom Waters: In other settings, DWP have had external peer reviews on this sort of thing. Just recently, a report came out on the effects of the benefits cap which was peer-reviewed. I looked at it briefly. It looked pretty robustly done to me and I think that it was a good effort. In the past they have put out a contract for someone external to do the research. Those sorts of things are worth doing, whereas if they cannot be certain that what they are putting out will be of high quality, it runs exactly the risk you say. It gets picked up and much is made of it when it probably should be taken with a substantial amount of salt.

Neil Coyle: I will quickly add that it is not that there is any reluctance or lack of willingness on the part of academics or think-tanks to get to a more robust study, and it is certainly not welfare claimants or organisations that sit on it. It is DWP that is fundamentally required to give you access to the data and support you to get to that point, isn’t it, just to be absolutely clear here? Thanks very much. I thought that was the case.

Chair: You will be coming in next, Neil, after Nigel Mills.

Q13            Nigel Mills: On that theme, what would a study that could prove this conclusively look like? I assume you would have to have a part of the country with a large number of typical claimants and a benefit regime with no sanctions. They would be told there would be no sanctions, and you would presumably have to use the resource you spend administering sanctions on more support. You would have to do it for a long time to see if you got better outcomes. Is that the only real way you could ever prove this?

Dr Arni: Yes. You are describing the optimal case for a researcher. Often, we would like to run randomised control trials. I think that is what you are referring to, that you would randomly assign people to the treatment group—those who are sanctioned—and to the control group, but obviously it is not feasible to randomise people into penalties in the context of sanctions. As a tendency, I think that you mentioned the core dimensions where we would need to improve the evidence to get a more comprehensive picture. It was also already mentioned by my colleagues.

I would also see two core dimensions added. The time window—a broader coverage over a longer time period to be able to look into long-term effects, as you said. We know from other studies—and in fact I recently saw a study on Germany where they looked at five-year effects—that some of these negative effects on earnings and employment do not disappear after a year. They are surprisingly consistent over time. That is the main finding in this new study in Germany. I also saw it when I worked in Switzerland for two or three years. These can be persistent effects. That is something we want to avoid because if individuals with low labour market attachments—people who are struggling on the labour market already—get stuck on a relatively adverse employment trajectory in the long run, their income loss over time adds up to a lot, even if it is a few per cent. It would be valuable to have that long-term perspective in the report.

The other point is about the deterrence effect because that covers everyone, as Tom also said. Everyone is affected by the deterrence effect of the system. To get an estimate of that requires a certain type of additional data to make it possible. I think that is the other big dimension to extend.

Q14            Nigel Mills: The challenges to this that I see are that, first, you have to know what you are trying to achieve in your jobcentre. If my policy was to get people out of there into any kind of job as fast as we possibly can, just get rid of them, that is very different to wanting someone to get into a career where they can have a long, sustaining job with chances of promotion. You would probably see sanctions being more effective in that first policy than in the second.

The other problem you have is the funnel of people. Most people who lose a job go to find one without troubling the jobcentre, but some people end up claiming benefits for a short time and then find a job for themselves relatively quickly, and then it gradually gets longer and longer. If you are planning this, you would probably want to use sanctions quite early in that process for people who should get a job but are not. You do not really want sanctions late in the process for those who have been there for a while and who are trying because you probably end up making it worse for them.

How do you take all of that into account? If you are in my system A, sanctions will be more effective than if you are in my system B, won’t they? That is the challenge.

Dr Pattaro: That is a valid question you have. Going back to what you said earlier, we have ideal methods that are quite robust, such as randomised controlled trials, but there are also estimation techniques that we can apply and they mimic experimental designs—the so-called quasi-experimental design. This is an area where studies should improve by utilising more of these methods. This could potentially address the questions that you raised.

I have been in conversation with colleagues from the Department for Work and Pensions for a while now and data exists. The data is there. They have the Work and Pensions Longitudinal Study, which has been used extensively for a long time. It is possible to combine data with data from other Government Departments, which would enable the study of these persistent impacts or knock-on or spillover effects in other areas of life of people, for example children’s education and health. In the case of our project, we are linking data from Public Health Scotland. The data is there and there is the infrastructure as well. We have obtained accreditation for this. The possibility is there to investigate. These are important empirical questions that need proper investigation.

Q15            Nigel Mills: The other question I have on this is if you have a very generous welfare system where someone could live quite comfortably, you might need more of a deterrent to do that than if you have a pretty frugal one where someone has a pretty miserable standard of living, because that is quite a deterrent in itself, isn’t it? I cannot pay the bills if I do not get a job. I do not really need to worry about you taking more money off me. I am going to do everything I can to get myself out of this mess anyway if I can. How do we factor that into what we are saying here?

Tom Waters: I think that any of these sorts of studies only tell you about the effect of the policy in the world that they are implemented. Sometimes we talk about external validity. How far can you read that out into a different situation? That is why you need to be a bit careful when you are looking at the effects in other countries because the situation is different.

I think that this is an area where—at least when we are talking about employment—the consistency, the effect is always in one direction, at least among high-quality studies. That gives us a bit of extra confidence, but you always have this issue that if you employ sanctions in a Scandinavian welfare state or somewhere that has much higher level of out-of-work benefits, that is likely to have a somewhat different effect to applying it to the UK or another setting. I think that is a reason why it is particularly valuable for people to be able to do this research in the UK in the current setting, because that provides us the best information for policymakers here today.

Q16            Nigel Mills: Dr Pattaro, I was reading a paper you published last year on the quantitative evidence for this, which appeared to conclude that there was not all that much. I think the conclusion you found from other studies was effectively that short-term sanctions do what we want them to do. They get people off benefits and into work. In the longer term, it is a bit more difficult. Is that a fair summary of what you found in that paper from last February?

Dr Pattaro: Yes. The key findings in labour market outcomes are positive effects in transition to employment in the short term, but this comes with a considerable cost because of pushing people in the longer term into lower quality jobs that are characterised by lower earnings, we said, but also higher transitions into non-employment and economic inactivity. This is for the labour market.

For wider outcomes, although the evidence base is more heterogenous, the quality of the evidence is weaker because the quantitative evidence is based on standard methodologies. They do not account for quasi-experimental design or experimental designs, for example. None the less, we consistently find that benefit sanctions lead to increased material hardship, financial problems, food deprivation and detrimental health outcomes for adults and their children. We have detrimental health outcomes for children and increased issues in behavioural problems and school attendance issues, so educational outcomes from that point of view, and, I would say, poorer cognitive achievements for school-age children. We found behavioural problems for adolescent children in that sense. This is it in a nutshell.

Q17            Nigel Mills: I am just reading the report. I think that quite early you cite a lot of studies. I think there was Kluve, 2010; Filges et al, 2015; Crépon and van den Berg, 2016; Card et al, 2018; Vooren at al, 2019; Yeyati et al, 2019. There are quite a few of these. You say, “Very broadly these studies suggest significant effects of an increase in rates of both benefit exit and job entry in the short term”. I think that is roughly what they did at DWP but then, to be fair, you go on to say, “Others suggest that, in the longer term, there may be higher risks of economic inactivity or a return to employment benefits, and worsening job quality”. That is roughly consistent with what this report tells us, isn’t it?

Tom Waters: No because this report says it reduces employment.

Q18            Nigel Mills: It reduces exit into higher rate work, doesn’t it?

Tom Waters: It says that getting sanctioned reduces your likelihood of being in any employment, which is inconsistent with the evidence and I think is the reasons we have been talking about.

Dr Pattaro: This is the only area where there is inconsistency, as Tom said. We have a prior report from the National Audit Office, which was published in 2016. The study was conducted based on data on the prior legacy system, so different from the Universal Credit system. What was found there is that the likelihood of transition to employment for people who were claiming jobseeker’s allowance increased and the opposite was reported for claimants of employment and support allowance.

In the case of the report, as it was also highlighted by my colleague David Webster from the University of Glasgow, it may be possible that the evidence that we see in the DWP report is picking up this effect that is probably related to people who are sick and disabled with long-term conditions. The investigation does not identify the effect for people who are sick and disabled and none the less subject to conditionality. It would be beneficial there to investigate this effect by extending the investigation or the study.

Q19            Neil Coyle: You mentioned the group subject to conditionality. The DWP study indicates that young people under 26 and claimants with children do worse under sanctions. Why do the panel think benefit sanctions appear to have a greater negative impact on families with children and young people?

Dr Pattaro: There are two mechanisms at work. One is through the financial or material route. Basically by reducing the financial resources, the living standard decreases and people are not able to afford basic necessities, including food, heating or electricity. The second mechanism is through the increase of anxiety and depression for the parents. This may affect the relationship with children. For example, by increasing problems with parental practices, there would be consequences of child abuse for children and health consequences also for children, but also more generally problems with developmental aspects of children, such as cognitive, social and behavioural development.

In our review, on the impacts on children and young people we found that 16 studies reported on the impacts of children, mainly pre-school children but also adolescents, as I said earlier. The main results—and I would stress these—are consistently negative impacts in child maltreatment, including cases of abuse and neglect, adverse children’s health outcomes, including an increase in GP consultations and hospitalisations, increased foster care placement, problems with school attendance and poorer cognitive achievement for children. These are in a nutshell when we are looking at children’s outcomes.

Q20            Neil Coyle: Do those studies examine the costs to the NHS, for example?

Dr Pattaro: The vast majority of studies that we reviewed, I would like to emphasise, are based in the US context. There were no cost-effective assessments of the service.

Q21            Neil Coyle: Tom, you mentioned earlier that someone with a sick child may find it harder to avoid being sanctioned. Coming back to that same question about why people with children and younger people seem to be worse affected, can you outline your thoughts as to why that might be the case?

Tom Waters: Yes. A large proportion of sanctions are for missing a meeting with your work coach. There are lots of things that might make you more likely to miss a meeting with your work coach, such as having a child who is unwell. I think we should not put too much emphasis on the precise numbers in this report, so the differences between these groups is picking up the unobserved factors that we cannot see. Something like having a child is the kind of thing that might make you more likely to miss your jobcentre appointment, maybe because they are unwell or for some other issue. Those things might also affect your ability to find a job. In this report, all those things appear to be the effect of sanctions. That is my assessment of it.

Q22            Neil Coyle: If there is anything you want to add, feel free.

Dr Arni: I want to add one point. Referring to young claimants being affected by sanctions, again I want to draw a parallel to some international evidence. There is not much on that, but there is some evidence on young welfare recipients and how they react to sanctions. I think that is relevant as this is definitely part of the Universal Credit population here as well.

One concerning additional effect that we see, which I think should be taken into account, is that among them, maybe for reasons that have been mentioned, we see more people leaving the system. They exit the benefit roles without destinations or without known destinations. This is a special issue because then particularly young individuals will no longer be supported in their journey back into the labour market. If we have sanctions that are too strong on younger individuals, also with income effects, there is a risk that they exit the system.

Q23            Neil Coyle: Is there specific research on that—that some of the group of economically inactive young people in the UK are there because they feel that they would be sanctioned or that the regime is too onerous for them? Is that robustly justified?

Dr Arni: In fact, I was referring to a study in Germany. I am not really aware that there is a specific investigation on the UK.

Q24            Neil Coyle: I take your point that it is making a hard-to-reach group even harder to reach because it puts them outside where they cannot be encouraged to do anything. Would you encourage DWP to look at that specific group and the impacts of any future risk?

Dr Arni: Yes. It would be good to have a more thorough investigation.

Q25            Neil Coyle: Thank you. In examining sanctions generally, have you come to conclusions or seen conclusions that you think would work for the UK example around mitigating the negative impact on young people or those with children, for example? Parents, I think we call them.

Dr Arni: Yes, and maybe in a larger context. I do not know if we will discuss that later on. One recommendation that I have is to rethink the size of the sanctions that are imposed in the system. That tendency may be too high and then the risk, particularly for young and lower income individuals, that this has more severe consequences, like exiting the system, is higher.

I think that to optimise the sanctioning system, one could reflect on the system where the use of sanctioning is still more or less present, the monitoring is important, but the size of sanctions and tendency may be reduced. I do not know if it would work practically, but one idea in addition could be to have additional income means tests on the impact of a sanction, so that you would go along and see if we impose that sanction, how that relates to the available income of the household of the people. If that reaches a threshold, a level that is too high, maybe one could reduce the size of the sanction.

Sorry to mention Germany once again, but in that context there has been a constitutional court ruling in 2019 that in fact declared some of the large-sized sanctions as unconstitutional for that type of reason. They said that the size of the sanction is too high in reducing people below the minimum poverty level of surviving and they capped the maximum size of the deduction after that ruling. That also goes a bit in the same direction. Maybe one could induce more means testing in this sanction. I know there are some components like hardship payments in the UK system, but this could probably be strengthened.

Q26            Neil Coyle: That is just taking from one bit of the system and pushing it somewhere else. It is not just a German example, by the way. The Select Committee also recommended caps on sanctions. Sorry, Serena, I know you wanted to come in as well.

Dr Pattaro: Yes. I want to add something mentioned in the report as well, the differences between the European continental countries’ sanctions regimes and the UK’s sanctions regime that places quite a strong emphasis on means-tested provisions. In the case of UK benefit claimants, there is no second tier of assistance, of social assistance, that could come in, for example in the case of exhaustion of benefit sanctions in countries like Germany, which rely more heavily on unemployment insurance and therefore are characterised by more generous benefits and higher replacement rates as well. Therefore, we have to keep that in mind when we are looking at other countries. The UK has particular features that may engender a harsh negative impact on people in the areas of sanctions.

Q27            Neil Coyle: I do not know if you want to add anything, Tom, but the public’s attitude to this is that they believe that there is adviser discretion at the jobcentre and that there will always be exemptions for certain people in certain cases, the parent of a sick child being an example. The Government want to head down a more automated approach to sanctions. You have already outlined some of them but are there any other risks to a more automated approach and what is your view on jobcentre work coach discretion?

Tom Waters: That is a tricky one because on the one hand it allows the possibility of tailoring the approach to the claimant, taking account of something like a sick child, which is obviously appealing. On the other hand it means that people are treated differently through no fault of their own. Someone might get a work coach who is quite tough and someone else gets a work coach who is quite lenient. Having a consistency across the board is appealing. It strikes me as a hard thing to get the balance right. It is probably an area where we do not know very much about the effects of those decisions.

Dr Pattaro: We conducted a descriptive analysis of differences in sanctions rate across Jobcentre Plus catchment areas in Scotland. We looked at the aggregated level data for Scotland, not the rest of the UK. We did not find any evidence there of systemic differences, just by exploring the aggregated trend data over time, of any differences across areas. Therefore, I would not say that there are particular problems. DWP colleagues have conducted similar studies for the UK that reached the same conclusions.

Chair: Thank you. We do need to speed up a bit; we are getting a little bit behind.

Q28            Selaine Saxby: Moving beyond the research and looking at the actual system that we have in place here in the UK, do you think that the correct conditions are being put in place to get people back into work? If not, what changes do you think we need to make?

Tom Waters: A picture is the few tools that Government have to get unemployment claimants back into work. We have been talking about job search conditionality and sanctions but there are also financial incentives, training and subsidised employment.

In the UK the focus is on conditionality and sanctions and to some extent financial incentives. We have quite a bit less on training compared to some other countries and not much on subsidised employment. My sense is that there is pretty good evidence that training programmes, if designed well, can in the long run deliver increases in employment. The same is true for employment subsidies, by improving the work experience, the claimant building up their human capital and delivering these longer-run positive effects.

Compared to other European countries the UK is much more reliant on the conditionality regime rather than things like training, subsidised employment and other things. It is certainly something for the DWP to consider whether that sort of thing can deliver more value for money.

Dr Arni: I definitely agree with what Tom said and I underline the point. One could imagine doing more on the level of installing training programmes. Also what has shown to be quite effective is relatively intense job search assistance programmes. You probably want to target that to certain groups. For example, there has been good experience in job search coaching for older jobseekers—45 or 50-plus. These intense job search assistance programmes are applicable relatively early in the spell, which is also a good advantage.

It was mentioned before that the earlier we can support and help people go back into the labour market the better. If there is more investment in such activating support in the first couple of months or the first year of unemployment, that could be quite effective. I agree in the international comparison that the UK is rather at the lower level of investment in these more activating programmes.

Dr Pattaro: Beyond the labour market mechanism or support services that have been described by my colleagues, the evidence in the UK is something that has not been covered in my review, because one observation is that robust quantitative evidence in the UK is rather scant. However, there is an increasing number of studies that looked at the health outcomes for lone parents and the lone parents’ obligations within the Universal Credit system. One of the authors is Tom. There is a number of studies that apply robust methods and show that there are consistently negative impacts on mental health for parents or lone mothers and also their children.

Although these studies are looking at a period when the level of conditionality for lone parents was relatively lower, one can imagine that increasing these can lead to additional health burdens. Yes, we need to increase an evidence base that uses more rigorous methods and includes bringing wider outcomes into the picture.

Q29            Selaine Saxby: The Government have recently strengthened their sanctions regime here. Do you think that these changes have a positive impact on employment outcomes and how should the effectiveness of this strengthening be measured?

Tom Waters: Two recent changes are to increase the amount of conditionality on parents of children aged one to 12 and to change something called the administrative earnings threshold for couples, which applies conditionality to non-working partners of workers, or at least some non-working partners of workers. The Office for Budget Responsibility does not think that that second one will have an impact, or it has not scored it as having an impact, whereas it thinks that increasing the conditionality on parents will bring an extra 10,000 people into employment. My sense is that at the margin these things would have some sort of effect but we should not be expecting gigantic numbers. Something in the neighbourhood of 10,000 reflects that.

On measures of these effects, this connects to some of the things that we have been talking about. Without access to DWP’s data, it is quite difficult for anyone to assess the effects of these policies; certainly the change to the administrative earnings threshold would not be possible. We would be reliant on either DWP doing a study using its own data or making its data available to outside researchers to study these policies.

Dr Pattaro: I want to add a couple of points on the issue of how we should measure these impacts. I reiterate what Tom has said there but also add that it would be helpful to have consultations to define and identify the characteristics by which benefit sanction statistics should be published to enable monitoring over time.

For example, no information of benefit sanction statistics by age groups is regularly published, or by sex. Ethnicity is another dimension and the health status. These are key characteristics that should be considered in the production of benefit sanctions statistics.

The other issue, touching on what Tom said, is how should we measure the impacts of benefit sanctions. There the issues pertain to the sensitivities that benefit sanctions measures may have on other outcomes. Again, we would need to have a consultation here with experts in the areas, or widen the conversation. There are a lot of areas that could be looked at by combining data across Government Departments. For example Administrative Data Research UK, of which the Scottish Centre for Administrative Data Research is part, is relying on cross-sector administrative data linkage to look at the wider impact.

Dr Arni: I want to make one additional point. There were key points taken up by that. Given the recent strengthening, if we rerun studies on the recent conditions, one in addition that we would need to keep in mind that would complicate the picture is that now we are in a quite particular situation of the labour market in general. We have a labour market that is thriving. It has dried out in quite a few areas. That means that the individuals who are now on Universal Credit who are also longer on the UC is not the same population any more as it was in the report that we were initially discussing in 2016. I suspect that if we ran the same type of study now we would see different results.

In tendency, the negative financial effects could be stronger now because the individuals who are now on the UC are probably people who are more distant from the labour market and who struggle more to get back in the labour market. Among those the risk that sanctioning leads to more detrimental financial impacts is higher. In that sense my prediction is that even without the strengthening we would find a different picture of the sanction effects per se, which makes comparison quite tricky because we have changes in policy and labour market tendencies.

Q30            Steve McCabe: Thank you. Apologies for my lateness; I have been at another meeting elsewhere. I want to pursue the issue of sanctions a little bit. The Secretary of State told the Committee in a letter just the other week that sanctions are fair and effective and that the majority of claimants felt that sanctions made them more compliant with requirements. Therefore, there is a general view that a tougher regime produces better results. Do you think that that is true or could there be an effective way of monitoring compliance in job search without sanctions?

Tom Waters: One question is about monitoring, the meetings with the work coach at the jobcentre and filling in the Universal Credit diary being the key mechanisms for that. The second part is enforcement, which is where sanctions come in. If you were to have a job search conditionality regime but without sanctions—saying that if you are going to claim benefits out of work and you are not unwell we expect you to search for work but there is no consequence if you do not—we would not expect that to have too much of an impact. There is a broader question about do we want to have conditionality at all. However, if we do have conditionality I struggle to see a way that it can be applied without sanctions.

That said, of course, once you decide that you want sanctions there is still a huge range on how tough they are. We were talking earlier about how the UK compares to other countries. However, I think that some level of sanctions is required to make a conditionality regime work.

Q31            Steve McCabe: Thank you. Is that a view shared by all of you?

Dr Arni: I agree with that point. I also do not see how a conditionality system would work without any enforcement. Indeed, one may need to move away from conditionality in the classical sense and just try to support people as well as possible. Imposing some requirements without any enforcement consequences is probably difficult.

As the point was made, where I see potentially a calibration of the system, as we mentioned earlier, is the size of the sanctions versus the frequency of the sanctions. The monitoring per se in tendency has a good impact in the sense of helping individuals to comply more with the rules and probably to search more. However, one could adjust the size of the sanctions in tendency downwards and a bit more targeted to income issues to reduce the potential detrimental long-term financial effects that sanctions can have.

If you just go even further up with monitoring and more sanctions and at the same time keep a certain set of strong sanctions, probably for a large part of the group that may be still okay but the other part of the group will struggle because the financial burden of the sanction would also increase. I would rather see a calibration of these parameters.

Q32            Steve McCabe: We are broadly agreed that some element of conditionality is necessary if you want compliance, particularly with job search and with reporting on time, is that right?

Dr Arni: I would say so.

Dr Pattaro: If the question is whether strengthening of benefit sanctions leads to better outcomes, based on the evidence that I reviewed and the paper I co-authored with my colleagues at Glasgow it does not seem that the international evidence supports that a strengthening of sanctions may lead to better outcomes, at least in the longer term. If you look in the short-term outcomes, yes, there may be some gains there but the evidence also shows that there are persistent, longer-term and far-reaching negative outcomes.

Q33            Steve McCabe: Are they in the types of jobs that people take or on their mental health?

Dr Pattaro: It is a combination of the two elements. There is a bidirectional relationship there. Having a lower-paid, lower-protected job may lead to poor health outcomes and vice versa. Sanctions as one of the mechanisms certainly do not help in reducing the level of anxiety and depression among people. We need to calibrate carefully and to carefully revise the evidence that is available at the international level but also improve the evidence within the UK because we need to understand the mechanism in our own country.

Q34            Steve McCabe: Am I hearing the evidence correctly—that you do believe that some element of conditionality is essential for compliance? There are some concerns about proportionality in the scale of them and the longer-term effects are likely to be detrimental for future employment potential and people’s mental health. Is that an accurate summary of what you are telling me?

Tom Waters: It is fair to say that on the longer-term consequences and on the wider consequences, as reflected in Serena’s paper on this, the quality of the evidence available is quite low. We have much less evidence on distinguishing correlation evidence and the actual causation, what is the true causal effect of sanctions. I agree with Serena that more evidence on this, particularly in the UK, would be very valuable for these longer-term effects.

Q35            Debbie Abrahams: I am going to follow up from my colleague. I absolutely agree with you and I have found the discussion fascinating. We are saying that the quality of the research that we have and the type of evidence that we have is not strong enough to say that there is a causal relationship between sanctioning and improving or worsening health outcomes, but there are indicators that it can have long-term significant consequences not just to the claimants but to the claimants’ children.

There is a lot to unpack there in the research design. There needs to be the access to data, the extension of data that is collected, to make sure that we can build that evidence base. However, I am sure that everyone here is familiar with the precautionary principle that when there is significant evidence, even though it does not establish causation, we should be taking a precautionary approach to that when we think that there may be harm associated with a particular policy. Does the panel agree, given the constraints on time that we have? This is the first of a series of questions.

Tom Waters: In any setting of benefits policy you are facing trade-offs. When you think about a much simpler question of whether benefits should be higher or lower, the basic value should be higher or lower, those things have shown that there can be effects on people’s health and the children of claimants and so on. On the other hand, of course, the Government do not have an infinite budget and they have to make choices and trade-offs and these policies can have effects of people’s work incentives and so on. Accepting the facts about the quality of the evidence on the long-term effects on conditionality, it is part of that big trade-off.

Q36            Debbie Abrahams: If you don’t mind my saying so, it does assume from what you are saying, that the Government believe that there are a lot of people who deliberately try to evade taking up their responsibility, which I have issues with, because it is an assumption that most people are trying to buck the system. I will put that to one side.

Could I have your response on a precautionary approach to a policy that potentially could be doing harm? Would you be recommending that? We are policymakers but we are not the Government and we have to make recommendations to the Government.

Dr Pattaro: Yes, it is a valid approach. I would also like add that on health impacts there is an emerging body of evidence, for example through the work of a colleague of mine, Evan Williams, that was published in the Journal of Social Policy 2021, that shows, using quite robust methods, that there is a detrimental effect of health. There is evidence there and we should certainly adopt a precautionary approach.

Dr Arni: In principle I also support that. That is also challenging in time to produce this high-quality evidence, which is certainly needed. If we are discussing something like that, maybe we go back five years again to revise that evidence. In elements like health I also saw some evidence from Sweden. On the detrimental long-term impacts it is clearly an insufficient base but there are clear indications that there are issues that should probably be taken into account.

Chair: Debbie, it might be worth making the point that came up before you came in, that Dr Pattaro’s team at Glasgow has been talking to DWP for a long time about accessing data to explore the impact of benefit sanctions on health. Last July the Department said that it would not be providing that data, after 10 years of discussion, by the sound of it.

Dr Pattaro: Yes.

Q37            Debbie Abrahams: That is very disappointing indeed. Thinking about the point that Tom made, these are having an effect not just on one Department, apart from the human effect that this is having on people and their lives and their future. We as a Committee, I am sure, will want to comment on that point.

Back in 2015 there was a freedom of information request to the Department. It revealed peer reviews that are undertaken when a benefit claimant dies; it looks at the system and what happens in the system. One in five of those deaths was related to a sanction. Has anybody investigated that? This is with a precautionary approach where we have increasing evidence, if not causal evidence, that is showing that there is a potential harm. This is 2015 data. Do you know of any further data on investigating what the impacts on the potential deaths of claimants may be? Have you come across any?

Dr Pattaro: Stephen Timms mentioned the projects that we are planning to conduct and one of the outcomes that we are interested in is deaths by suicide but also by other causes. It is one key outcome of our investigation. There is some evidence at an international level that shows that an increase of conditionality is linked to an increase of deaths by suicide, but it is looking at aggregated data. I think that it would be beneficial, and that is what we were proposing in our project, to look at individual-level data to discern the mechanisms that are underlying the type of relationship that links sanctions to health outcomes.

Q38            Debbie Abrahams: My next question is on the international evidence. We are focusing on financial sanctions here. Are there any other types of sanctions that could be used? The principle of conditionality has been with us since the inception of the modern welfare state since the second world war. Most people will agree that that is the right approach, but recognising enforcement and what you would do to enforce that conditionality when the vast majority of people do the right thing but a tiny minority may not want to. We talk about the proportionate scaling back of sanctions but we have a subsistence level of support and this is having an impact on the families not just the individual.

What alternatives, from the international evidence, are being used now? You mentioned other carrots within the system, about different interventions around training and so on. Is that an alternative that has been used—“That won’t be available to you because you haven’t done enough in this bit of what you’re meant to be doing”? Do we know of anything around that?

Dr Arni: I have thought about that also in preparation for the session. There is no broad evidence on that. There are a few hints of some mechanisms that could be a bit similar to the operation of a sanction. One element that can be observed in certain studies is that you can also use some other labour market policies, unintentional often, to put some pressure on individuals, because they dislike these policies.

For example, there are certain types of public employment programmes or certain types of disliked training programmes where it has been observed in evidence, where you look at the dynamic effect of these programmes, that there is also a bit of a threat effect from these programmes. People would like to avoid getting to the stage of being more activated and they increase their job search.

Debbie Abrahams: That is an area that you say there is not a lot of evidence about and there is not a lot of mechanisms. That is something that potentially the Department should be looking at and there might be pilots that could test these approaches. That is probably all I want to say, thank you.

Q39            Neil Coyle: We touched on this earlier, but what evidence is there to show that administering the sanctions regime is cost effective?

Tom Waters: It comes back to what you want to achieve from the sanctions regime. The closest evidence that I am aware of on it is we did some work looking at the lone-parent obligation that imposed conditionality for single parents between 2008 and 2012. That was the imposition of conditionality. That is not specifically getting sanctioned but it includes the threat of sanctions.

We found there that it did not save the Government very much money. While it did get some people into work, it tended to be quite low-paid work and part-time so they did not pay much in tax and still got a lot of benefits. Some people switched over to incapacity benefits like ESA, which cost the Government more, because of the imposition of conditionality. That was just for single parents, but we found there that the savings to the Government were pretty close to zero.

Q40            Neil Coyle: Did that study look at accessing other benefits and support as well like PIP?

Tom Waters: Yes, there was a small increase in getting PIP. Yes, and PIP.

Neil Coyle: Net zero and others.

Dr Arni: A key question when I thought about cost effectiveness is what costs do we consider and also which benefits; which weighing and trade-off do we consider? What I have not seen in international evidence is that they would go that far and also try to estimate the cost of the operational system. Then it is again difficult to define it. Should we take the cost of the whole monitoring system, for example, as the cost side? Should we count that in or not? I have not seen studies doing that.

One thing that we have tried to do in our 2013 study on Switzerland is to do the trade-off of the earnings losses of the individuals on the one hand and the reduced benefit payments on the other hand. What we found there with the sanction-imposition effects is that the earnings losses over the two years after unemployment were about 50% of the amount of reduced benefit payments. That gives a bit of the dimension on what the Government could save on benefit payments on the one hand, but on the other hand there is the earnings loss that goes on the account of the individual. It is again the question of whether you want to count that in or not but it gives some of the dimensions of these trade-offs.

Tom Waters: With lower earnings come lower tax and other contributions.

Q41            Neil Coyle: Does the Department for Work and Pensions provide a cost for how much the sanctions regime costs to run? Does any Government provide that estimate for its sanctions regime that you are aware of?

Dr Arni: I am not aware of anything.

Q42            Neil Coyle: Would a full cost analysis have to take into account issues like, Patrick, you mentioned the hardship fund, other bits of Government spend? You touched earlier on GP and health appointments. Would you welcome DWP analysis of where people go when sanctioned, what their next step is and whether there are other governmental costs involved?

Dr Pattaro: Yes, I would. I would welcome it. In fact, we were proposing to help the DWP and provide some analysis for their impact on health outcomes but also use of public sector as well, health services in particular.

Neil Coyle: Is it essential to have a full analysis?

Dr Pattaro: I think so, yes.

Q43            Siobhan Baillie: You have touched on mental health already but I want to go through it again. We know that increasing numbers of welfare claimants are citing mental health conditions—depression and anxiety—not always diagnosed medically. They are citing conditions as barriers to entering the workplace.

You were saying, Serena, that sanctions exacerbate mental health conditions like anxiety and depression but you are not satisfied that there is a sufficient evidence base out there at the moment. However, we also know that having a job, getting up in the morning and having a purpose is good for mental health conditions and general medical health. This is causing huge problems for work coaches as well because they want to be compassionate and they want to make sure that they are helping people.

How should the negatives be weighed against the positive impact of employment? How should DWP be treating this, notwithstanding that you want it to agree to a massive study, which sounds like it could be very helpful but would not necessarily help us immediately? How do we deal with this issue at the moment?

Dr Pattaro: I reiterate what I said earlier: that we need to bring in the wider picture, the wider outcomes. It is difficult to respond a priori when the evidence is there, some evidence, but we would need to conduct more rigorous studies to provide some recommendation.

Siobhan Baillie: We cannot say that it is wholly negative or wholly positive.

Dr Pattaro: Yes, it is an empirical question.

Tom Waters: In our work on the lone-parent obligation we found some evidence. I do not want to put too much weight on it but we did find some evidence that it worsened mental health for those who stayed out of work and were now having to search for jobs but it did improve mental health for those who got into work.

For those who stay out of work you worry about whether this causes a vicious cycle, that the very fact that mental health has got worse makes it harder for them to get into work and so on. To some extent this comes back to the question earlier about work coach discretion. If work coaches had more discretion, would they be able to apportion sanctions and conditionality in a way that is a bit more conducive to mental health? That is one thing for the Department to consider.

Dr Arni: There is one piece of additional evidence from Sweden that is pretty recent where they could do some of these registry connections. They took the social insurance data and added some data, notably on drug prescriptions. On the drugs, mostly they could distinguish cardiovascular diseases and prescriptions for mental health issues like anxiety and depression and they could connect sickness absence data. What they found for the sanctioned is that in the months before the sanction was imposed the prescriptions related to mental health problems increased. That is in the month before, so establishing a causality is difficult or in that framework probably not possible, but it shows that there is something going on.

The other thing that they found as a short-run effect of sanctions was that there was an increase in sickness absence in the short period after the sanctioning, the couple of months after the sanction. Therefore, there is some evidence there. It is not causal but it would show, exactly as you say, that there is a possibility to connect that data. I think that it is available. You could use the national insurance number to connect it to get a broader picture of that.

Q44            Siobhan Baillie: Am I right in saying—this may not be your area—that if you remove the sanctions element of this there is sufficient data and there is a sufficient evidence base that links positive benefit for people who have mental health conditions getting into work? Ignoring the sanctions element, is that established evidence or are we not there yet?

Dr Pattaro: It is a debated issue. In the literature there is still a question mark there. Evidence is needed to appropriately establish this. Do my colleagues have anything to add?

Tom Waters: I am not sure.

Q45            Siobhan Baillie: What evidence is there that benefit sanctions have a negative impact on people’s trust in jobcentres and work coaches and DWP? Tom, you said earlier that the inconsistency of approach may cause problems and could also provide a benefit with description. Do you think that is also damaging trust across the whole system?

Tom Waters: That is a fair point with respect to discretion if you happen to get unlucky and you have a work coach who is a bit tougher. That is the kind of thing that could, as you say, undermine trust in the system. I am not aware of any evidence on the issue but it seems plausible.

Dr Arni: There is only something a bit indirect that I found on that for reciprocity. There is a bit of research in Germany about the reciprocity effect between the jobseeker and the caseworker. They found there that if a caseworker does stricter sanctioning, some jobseekers may reciprocate by not collaborating as much as they would otherwise. However, that is quite restricted. I do not know all the details of that study but they say that it depends on the personality traits of the jobseeker. Not everyone would reciprocate like this but there are some groups of people who may react in that way. It is a little piece of evidence that may be relevant, and the attitude of the caseworker and the strictness of the caseworker.

Q46            Chair: Dr Arni, the study that you told us about in Sweden where people are getting prescriptions before a sanction is imposed, what is that about? Is there any suggestion? Is that because they know that a sanction is coming up?

Dr Arni: That is the point that they made in that study. That was their main hypothesis. They were also a bit puzzled with that and explicitly pointed out that there could be a reverse causality, that depression can cause sanctions and sanctions can cause depression. They hinted to that point. Usually the sanctions are announced, similar to the UK. There is a process that you get an announcement and you may respond to that and maybe appeal it. Therefore, usually people may be aware quite a bit earlier that they potentially may get the sanction, and that could cover that period.

Q47            Steve McCabe: Following on from that, how do you think that this country compares with other similar countries on the approach that we adopt? Are there any obvious lessons, good or bad, that we should draw from experiences elsewhere?

Dr Pattaro: That is something that I touched upon earlier. There is a clear distinction between the UK versus continental European countries where we have a dual tier of support within a sanctions regime that is heavily relying on employment insurance and assistance. That is a combined element, whereas the level of support is much more scant within the UK context.

The evidence that shows detrimental effects in the long term has been, interestingly, in our studies, focused on English-speaking countries, coming from the US context and the UK context, which are characterised by means-tested provisions on benefits in the social security systems. It is interesting that there we see the adverse impacts of sanctions, so one could draw the conclusion that—we see detrimental impacts or wider impacts for a number of outcomes in a number of countries that offer a very low level of social assistance and means-tested provisions compared to other countries where perhaps we have—I cannot say because the evidence was mainly covering the US and UK context and Australia as well.

The evidence is missing more at the international level for other countries, European countries, which cover the labour market outcomes primarily. Therefore, it is difficult to draw a comparison when there is such an imbalance in the evidence but I would say that it is linked to the type of regime that is in place that may have detrimental effects, certainly.

Q48            Neil Coyle: Do you want to add anything, Dr Arni?

Dr Arni: Two thoughts. One is referring back to a discussion we had right at the beginning about the deterrence effect that had not been investigated already in the report that we were discussing. At the place where it was possible to investigate the deterrence effects, there is some evidence from Switzerland and also from the Netherlands, interesting additional evidence, that the part that comes from the deterrence effect, the impact from there, is a bit less detrimental on the earnings. The negative amount of the earnings effects is smaller in the deterrence part than in the imposition part.

The conclusion, at least conceptually, is then to have a system where the deterrence effect is mostly triggered rather than the imposition. That goes back to what I mentioned earlier. Probably that is to some degree more possible with sanctions of a smaller size but maybe still keeping a quite intense monitoring regime at the same time. Maybe these two elements may help to shift the impact of the total system a bit more towards deterrence and a bit less towards long-term negative imposition effects.

That is one takeaway that I will mention. Another one, which may be a bigger leap, is we also had some discussions about who should do the monitoring and what is the role of the caseworkers, the work coaches, there. One could imagine more extreme reform of decoupling the two functions, the monitoring on the one hand and the support giving by the work coaches on the other hand. I am aware that that would be a bigger organisational reshuffle but it would to some degree address these issues about caseworkers, work coaches, having a different leeway or more or less strictness that may lead to a certain inequality of treatment.

The other advantage that is linked to a bit of evidence from Sweden is that the caseworker’s initial assessment of non-compliance is a challenging job for the work coaches to do. Particularly if the sanctions are large, what they observed in Sweden is that caseworkers would hesitate to flag the sanctions or refer it to the decision-maker. Maybe that issue would be solved if these two jobs were split into different functions, the supportive part with work coaches and the monitoring and controlling part with a separate institution. To some degree there is a bit of a step in that in the Netherlands where they tried to digitise the controlling part of unemployment insurance quite a bit so that they would try to go online with the more repetitive monitoring tasks. Then you can decouple it more obviously from the work coaches.

There is no systematic evidence that that would make a difference in the behaviour of the work coaches and the resulting sanction strictness, but some of these hints could be an interesting point to think about, whether a reorganisation in that direction would solve some of the issues.

Q49            Steve McCabe: Are there any international models at all where they put the emphasis on rewards and support rather than sanctions, that have a better outcome than we have here? Is there anything that you could point to and say that this is a model we must look at?

Tom Waters: Benefit regimes for people at work who are not unwell do tend to have some sort of conditionality and that is enforced by sanctions. One thing that you could point to, which could be here in the UK, is that historically we had income support that was meant for different groups but for the latter part of its life it was mostly for single parents. You could be out of work and get support. You could get training if you wanted to but there was no obligation so it was not a conditionality regime until the lone-parent obligation came in. That was for a specific group, for single parents, but it was another way of doing it. We said, “We don’t require you to look for work. We will provide help if you want to look for work, if you want to get training”. That was obviously a very different way of doing it.

Q50            Steve McCabe: Would that provoke a different debate about treating single parents differently, which I seem to recall we have had in this country already?

Tom Waters: That policy has largely disappeared now. If you have a child aged under one and you are on Universal Credit, you do not get any conditionality at all and only a limited amount for the next couple of years but it used to be all the way until your youngest child was 16. That change, I presume, did not happen for no reason and there was some sort of demand for something along those lines. However, it is notable that there was a long period where single parents were not required to look for work for a long time.

Q51            Steve McCabe: One last thing. I noticed that Germany has quite a tough regime on people under the age of 25. Is there an argument for treating different age groups differently and does this have any positive impact on encouraging a work ethic in Germany?

Dr Arni: I would say no. I mentioned the constitutional court ruling earlier in the session. They were blocked from doing that type of policy in a rather extreme way. The constitutional court forbade going for strong reductions like 60% or 100% of the whole benefit for something like three months or even more. That was particularly put on youth below 25. That essentially has stopped now.

There were also some policy recommendations before the federal employment agency in Germany to change that practice because it could not underline any positive impact of such an extreme policy for the young with positive evidence. From that point of view I do not see an evidence base for why you should be particularly strict for below 25s. Rather there is a bit of a risk of what I mentioned earlier, that for young welfare recipients, who would also be the second tier in Germany, there is an additional risk that they would go out of the system and say, “The finance pressure is too big so I would rather leave the support system entirely”. Then we are at the point where we cannot support these young individuals with help to go back into the labour market. Particularly for the young, that is a very detrimental impact. Therefore, from that point of view I do not see a reason why one should put enormous pressure on below 25s.

Q52            Chair: Thank you very much. Dr Arni, you floated a moment ago the idea of separating the role of work coach on the one hand from benefit decision-maker on the other. You said that the Netherlands have moved a little bit in that direction. Are there other examples of where that separation is well established or is it normal for the work coach and benefit decision-maker to be the same person?

Dr Arni: It is much more normal to be the same people. Also the current UK practice, as I understand it—I do not know whether it has changed—is to have two steps. The first step would be done by the work coach to flag up non-compliance. In some countries you then have a formal warning, a letter, that would be sent to the individual to say, “Can you please justify why you didn’t comply?” Then they have a certain period of time to react to this letter and then it would move, like in the UK, to another decision-maker in another administrative unit. However, replacing the very first stage to flag up potential non-compliance, I am not aware of continental European states that have fully moved into that regime, or not yet maybe.

Q53            Sir Desmond Swayne: It is suggested that the DWP’s research report finds that sanctions lead to a decrease in the rate of exit into higher-paid employment and also that claimants spend less time on Universal Credit. Those are the two conclusions that appear to come from the DWP report. Are those findings borne out by similar research in the rest of Europe?

Dr Arni: This aligns with other findings that we have from other studies in Europe. There is a bit of a difficulty of comparison. My colleagues mentioned earlier that a lot of the evidence in Europe is focused on employment insurance, on this first-tier insurance where the group of people usually is not as long on unemployment benefits. They fulfil higher eligibility requirements, so usually they have less strain on income than in a system like the Universal Credit. In that sense, the amount of the effects that we observe is probably not directly comparable to those studies.

However, the general impacts that are found in this report are reflected—there are some studies on welfare recipients in the European style of notion that show similar reactions. Then it is more about the question of what is the relative weight of people going out of Universal Credit with some earnings or without. That is probably a bit more difficult to compare across countries.

Q54            Sir Desmond Swayne: My prejudice is that we are measuring the caprice of human nature rather than the effectiveness of sanctions. I suggest that the sort of behaviours that would make one liable to sanctions would be the same behaviours that would make it difficult for you to hold down high-paid employment. That is my prejudice.

The study that you did with Professor Gerard van den Berg would challenge that because you are measuring, as I understand it, the effectiveness by caseworker. A caseworker who is a real martinet and imposes lots of sanctions tends to have worse outcomes that one who perhaps might rely more on support. I am sceptical but I am looking for the mechanism. What is the mechanism by which a sanction makes you less able to hold down high-paid work in the longer term?

Dr Arni: Maybe you are referring to our study where we wanted to estimate what we call regime effects. That is what you are referring to, where we were using the variation that you were talking about, about more and less strict caseworker behaviour. We investigated the carrots and the sticks but let’s focus now on the sanctioning, which are the sticks there. The way that I interpret that result is along the lines of the deterrence effect that we were discussing. If you are a jobseeker in an area or are confronted with a work coach who is stricter, that would per se make the deterrence effect stronger so that individuals would comply more with the rules. That would speed up some of the unemployment exits.

On the other hand, and you referred to that, in the longer run we see that this boosts the negative earnings effect as well with the stronger threat effects. That would have been our main interpretation of the caseworker-level effects. I would see that in the context of the discussion on deterrence effects.

Q55            Sir Desmond Swayne: Are we not up against the problem faced by any social science: that we are attempting to make sense of the caprice of human nature—the random caprice of caseworkers as against the random caprice of claimants—and trying to make sense of that by imposing the science of large numbers, statistics? How confident are you in the robustness of the conclusions that you have drawn, given that limitation?

Dr Arni: I would see it like that. Probably what you are referring to as caprice is the variation, we see that caseworker A and caseworker B are not the same and they behave differently. Technically speaking, in that study we used that variation between the caseworkers to identify an effect. In that sense we are not directly saying that we are now comparing just the variations of the caseworkers, but we are estimating, using that variation, the propensity of how caseworkers behave specifically on the sanctioning issue, for example. We are using it as a methodological tool to estimate variation in the sanctioning regime. I am quite confident that this worked out. This is not just noise, or at least we could channel the noise to use it to estimate the sanction effect.

If we go even deeper methodologically, I agree that that study is also not perfectly waterproof. You could criticise that you would need to have further controls for establishing even better comparability. That is one thing; you could go further. However, it passed all the peer reviews as well and this study is published in a well-established statistical journal.

The other point where maybe there is some leeway, if you wish, is the interpretation. That is just the main interpretation that we offer, the deterrence effect interpretation. Maybe you could find alternative ways of interpreting caseworker behaviour and the way you look at that would be a bit different about how these caseworkers behave and how that impacts on the jobseekers.

However, also in the context of other evidence with similar studies, we think that that is clearly the most obvious interpretation of the results that we have there. It is consistent with other evidence, it is consistent with the theory, the so-called job search theory, that can model that as well. There was a Nobel prize from that a couple of years ago. It fits into that picture, so in that sense I am pretty confident that this interpretation makes sense.

Tom Waters: If I can add to that—correct me if I am wrong—the key thing is that claimants are randomly assigned to caseworkers.

Dr Arni: Yes.

Tom Waters: They are randomly assigned so it is something like a drug trial. In a drug trial some people randomly get the real drugs and some people randomly get the placebo. That is what Patrick is trying to do here. Some people randomly get a really tough caseworker and some people randomly get a more lenient one. That is the key thing that he is using. Is that fair?

Dr Arni: Yes.

Chair: Thank you. That concludes our questions to you. Thank you for the very interesting evidence session that you have given us. There is a lot for us to think about. I think that you have made a very strong case that it is in the interests of the Department to allow access to its data for these things to be properly explored and understood. We are very grateful to you. Thank you for joining us. That concludes our meeting.