Science and Technology Committee
Oral evidence: Research integrity, HC 350
Ordered by the House of Commons to be published on 6 March 2018.
Watch the meeting
Members present: Norman Lamb (Chair); Vicky Ford; Stephen Metcalfe; Carol Monaghan; Damien Moore; Graham Stringer.
Questions 532 - 615
Witness
I: Professor Sir Mark Walport, Chief Executive, UK Research and Innovation (UKRI).
Witness: Professor Sir Mark Walport.
Q532 Chair: Welcome, Sir Mark. Thank you very much for attending. You have done this before, so I am sure you know what is in store.
The Science Minister has declined to give evidence this morning and has effectively sent you in his place, although he had initially agreed to attend. Did you advise him to take that approach? We have never experienced this before, at any time, so it is a novel experience for us and one we are very concerned about.
Professor Walport: No, I did not advise him to take that approach. I had a discussion with him about the appearance. The first thing to say is that you will have the opportunity to speak to him about Brexit in an hour’s time. I do not think for a second that he does anything other than take this very seriously; there is no issue with that. I think he felt that, given my expertise and your concern with the details of this, I was probably best placed to answer your questions.
Q533 Chair: Do you agree with what he said in his letter—that how research is delivered is the responsibility of funders, such as UKRI, rather than the Government?
Professor Walport: Ultimately, the Government own the policy on this. Our job is to deliver and to provide advice around the policy.
Q534 Chair: So you are not able to speak on behalf of the Government today.
Professor Walport: In this role, I am a public servant, not a civil servant.
Q535 Chair: So we will not get the Government’s view through this evidence session, in effect.
Professor Walport: I can speak to you as the chief executive designate of UK Research and Innovation.
Q536 Chair: Okay. Obviously, we will put some points to the Minister in due course. Are you comfortable about taking the lead in considering whether the regulatory framework for research integrity needs changing?
Professor Walport: I am comfortable that we can provide good advice, given our intimate relationship with the delivery of research funding on behalf of the taxpayer. I am confident that we can deliver advice. As I said, it is for Government to make the policy decisions.
Q537 Chair: What advice would you give to the Government about whether you are satisfied with the regulatory framework as it is at the moment? Do you think that there is a need for change?
Professor Walport: The broad framework for the assessment and management of research misconduct is appropriate and is in line with most international practice. There are four lines of defence, if I can put it that way. First, this is a matter for employers. Ultimately, they have the sanctions and the ability to investigate research misconduct. Secondly, it is a matter for the terms and conditions of the research councils—what will be UK Research and Innovation.
Q538 Chair: The funders.
Professor Walport: Yes. There is then an advisory level, through UKRIO. Ultimately, we as funders can take sanctions. Indeed, we do so occasionally.
Q539 Chair: Can you remember occasions when you have taken sanctions?
Professor Walport: This is before my time in the research councils, but there have been three occasions in the last five years when the research councils have taken sanctions.
Q540 Chair: Will you write to us to confirm those occasions in some detail?
Professor Walport: Certainly.
Chair: That would be helpful.
Professor Walport: In 2017-18, BBSRC imposed a sanction. In the same year, NERC imposed a sanction. NERC also imposed a sanction in 2015.
Q541 Chair: What sanctions did they impose?
Professor Walport: I do not know the details.
Q542 Chair: If you could confirm those in writing, that would be useful.
Professor Walport: I am sorry, Chair; I think I can give you the details. In the case of MRC and BBSRC, the scientist concerned had a current MRC grant and was responsible for a BBSRC doctoral training fellow. They were dismissed by the university, responsibility for the BBSRC funding was transferred and outstanding funds of the MRC were recovered.
In 2017, a research organisation upheld an allegation that a BBSRC research grant application had been mishandled by an external reviewer. Bans were enacted for the respective periods of time for refereeing of BBSRC applications and membership of all committees.
In 2015, EPSRC found plagiarism in some funding applications. The applications were withdrawn and a temporary ban on future applications was imposed.
Q543 Chair: Thank you for that. The concordat has been in place since 2012. Are you content with the progress in implementing its recommendations?
Professor Walport: Yes, I think so. As you will be aware, I was one of the original signatories of the concordat and was involved in its creation in 2012, when I was at the Wellcome Trust. It was reviewed, as you know, in November 2016. Since then, it has been tightened, and a forum is being initiated. If you look at the evidence on compliance with the conditions of the concordat, the numbers appear to be going up.
Q544 Chair: It is compliance with something that, ultimately, is quite high level. What you have to do to comply can be interpreted in different ways, can’t it? Do you think that there is a need for the concordat to be further strengthened and for elements of it to be more prescriptive in what it requires?
Professor Walport: The principles of it are absolutely clear. It requires the reporting of allegations of misconduct and that institutions in receipt of funds have a website that states what is going on. It is a reasonable set of requirements, actually.
Q545 Chair: Quite a significant proportion of universities do not report annually on their websites, as required. Six years on, that seems rather extraordinary, does it not?
Professor Walport: Let me find my numbers.
Chair: I think it is about a quarter.
Professor Walport: I have it here. The point is that 104 institutions, which represent 94% of the funding, have a webpage dedicated to research integrity. The number of institutions providing at least one annual statement has doubled from 26% in 2016 to 54% by 2018. That is over 80% of the public funding for research.
Q546 Chair: If you have a concordat that people sign up to through Universities UK, is it not fair to expect 100% compliance?
Professor Walport: Yes, although it is important to recognise that some institutions have a very small amount of research activity. The really important issue is to make sure that those institutions that are doing significant amounts of research are reporting. All the evidence is that they are.
Q547 Chair: That still leaves 20%, on the figure you have just given. You said that institutions taking 80% of funding comply.
Professor Walport: No, 94% have a web page dedicated to research integrity. On the annual statement, you are right: that is over 80% of public funding. There is room for improvement; I do not think anyone argues with that.
Q548 Chair: In UKRI’s leadership role, would your message to the research community be, “We need to have 100% compliance with this”?
Professor Walport: We should have 100% compliance with respect to funding that comes from the public purse, through UK Research and Innovation and other public sources of funding.
Q549 Chair: Will you do what you can to ensure that that happens?
Professor Walport: Yes.
Q550 Chair: You mentioned that the concordat had been tightened since it was initiated. When was that? In what way was it tightened?
Professor Walport: It followed the review—the progress report—in 2016. It has been tightened through UUK, for example, which is pressing its members to increase compliance. As I said, the percentage of institutions providing one annual statement has doubled from 26% in 2016 to 54% by 2018. That is a trend in the right direction.
Q551 Chair: Not even 100% of Russell Group universities comply with the requirement to provide a statement of inquiries. Does that not shock you rather?
Professor Walport: The first thing to say is that there is no question but that this is an important activity. It is something that UK Research and Innovation will take very seriously. Bringing together the research councils and Innovate UK under the same umbrella will give us a much clearer opportunity to have a completely unified approach.
Q552 Chair: Do you see the leadership on seeking compliance and, perhaps, looking at how the concordat may be tightened, if that is deemed appropriate, as resting with UKRI or with Universities UK?
Professor Walport: Ultimately, the employer has the vires to supervise its employees and to conduct the inquiries. The lever that UK Research and Innovation has is that we are responsible for taxpayers’ funding. We have that power.
Q553 Chair: You can demand the highest of standards, can you not?
Professor Walport: Absolutely. Our job is to demand the high standards, but we can enforce only where we are the direct employer.
Q554 Chair: On the question of whether the concordat is being tightened, my understanding is that Universities UK has said to us that it believes that that should be done—that it should go through a process of updating and improving the concordat, where it is seen to fall short—but that it has not happened yet. Is it your understanding that the concordat has already been tightened?
Professor Walport: I think so. As you probably know, on 23 November, Professor Paul Boyle wrote to all UUK members about research integrity issues. I should say en passant that I think it is very helpful that you are conducting this inquiry, because it will help to raise the profile of the issue.
Chair: We appreciate that endorsement.
Professor Walport: The more attention this gets, the better. I think that UUK is taking it very seriously, but there is room for improvement, and we will continue to push for it.
Q555 Graham Stringer: What gives science its pre-eminent position in society and keeps it honest is that the experiment is done and somebody else should be able to repeat it. The number of experiments that are repeated is now quite small, isn’t it? Do you have a figure for that?
Professor Walport: I do not have a figure for the precise number of experiments that are repeated. It would be very difficult to get that. You are now taking us into a slightly different area, which is the whole issue of research reproducibility. I gave a talk on that at the American Association for the Advancement of Science two weeks ago. Science advances by approaching a problem from a number of different directions, so you do not need to repeat every experiment in an identical fashion independently. Obviously, when researchers are developing a paper, they should reproduce their own work; otherwise it does not have the rigour that is necessary.
There are all sorts of factors around reproducibility. It is worth saying that that is very different from the sort of research misconduct you are talking about here. That is about how we improve the practice of research overall, which UKRI does and will take very seriously, and which the research councils have taken seriously before. It is about making sure that experiments are adequately powered—that there are sufficient human subjects, for example, if it is a human study, or that sufficient animals take part, if it is an animal study. It is not ethical to conduct underpowered experiments.
We have gone from a problem of small data to one of large data. When you are dealing with very complex systems, you may need large numbers. Of course, what has happened in research in the last 20 years or so is that the numbers have gone up greatly. Before, it may have been possible to do a genetic study in only 100 people, compared with controls. The risk was that that was underpowered. Now, when millions or hundreds of thousands of individuals are potentially being studied, in some cases, it leads to the possibility that, by dredging the data, you can find associations that appear significant, but are not. This is all about the good conduct of research.
Q556 Graham Stringer: I accept that point completely, but reproducibility is one of the ways of checking whether somebody has not reported accurately, either by intention or by accident, what has happened. Without that, if science is going to keep its position of honesty and integrity, do you think we need to look at other ways of checking the literature that there is? For instance, there is the stat check, where a computer programme goes through it all and checks whether there are mistakes. Do you think there should be more of that? Should Government or the academic establishment require that that is done annually?
Professor Walport: That is just one of many tools. It turns on the whole culture of research. It starts with training. It is important in doctoral training programmes, for example—and in undergraduate programmes—that people are adequately trained in statistics and have access to advice. It is about working in teams and about all those involved in the assessment of funding of research looking at it very carefully, to see whether it is adequately powered and well designed. It is about clarity in human clinical studies on what hypotheses are tested and what the end points are. It is about registration of clinical trials, which is happening. It is about the conduct of research, from the beginning right through to the end, and increased rigour in the whole process. I know that that is something research funders around the world are concerned about. It is something UK Research and Innovation will work hard on.
Q557 Graham Stringer: Rigour, particularly statistical rigour, should be part of the training process, clearly, but what we are looking for is where something has gone seriously wrong by mistake or there has been fraud. If you take a quick look through the literature or google it, the estimates seem to be that—without having the hard evidence, apart from in sampling—people believe that 2% to 5% of papers have made-up statistics in them or are in some way dishonest or fraudulent. It is about getting to those areas, rather than just training people to be better when those people have good intentions. How do you see that happening?
Professor Walport: It is quite important to tackle the right problem. The question is, how much of this is wickedness and how much is poor training and incompetence? They are two quite different things.
Chair: Both are important.
Professor Walport: They are both important. Take Retraction Watch’s data. It reckons that, in the UK, 0.75 papers are retracted for every $1 billion spent. You want to use the sledgehammer for the right nut, as it were. The right nut is to make sure that you deal with the conduct in the vast majority of papers where there is non-reproducibility. That is likely to be due to things like error in design and error in statistical analysis, rather than actual wickedness.
Q558 Graham Stringer: A famous American study of psychologists says that, when they were asked, 2% of participants admitted to making up figures in their experiments. That is one indication that fraudulence in science is higher than you might want it to be—certainly higher than the figure you have just given us.
Professor Walport: It may not be appropriate to generalise from one area of science to all others.
Q559 Graham Stringer: That is the whole problem, isn’t it? We are looking for indications, because we do not have the objective take on it.
Professor Walport: To be honest, it is a very difficult thing to work out, which is not to say that it is not very important. The issue in psychology is also one of small numbers, to a significant extent, with the potential for confounding factors. I do not know that it is appropriate to generalise from that to other areas.
Q560 Graham Stringer: Do you think that the research councils or the Government should keep nationally a record of where there has been misbehaviour or mistakes by scientists?
Professor Walport: The answer is that there are records kept at the moment. The research councils have kept records. UK Research and Innovation should certainly keep records.
Q561 Graham Stringer: This is my final point. Probably the most famous case of fraud recently was the Wakefield case, on MMR. We are staying away from the detail of that in this inquiry, because we do not want to repeat it. The significant point from that is that it was not The Lancet or other scientists, but a journalist, who worked out that Wakefield had distorted and fiddled the figures. Do you think that there is anything that we can draw from the fact that it was not science itself, but a journalist, who discovered what Wakefield had been up to?
Professor Walport: With respect, the scientific community picked up the fact that there was something wrong with that paper extremely early. It was subject to criticism from the moment it was published.
Q562 Graham Stringer: Although The Lancet did not withdraw it.
Professor Walport: That is a matter to discuss with The Lancet’s editor. It was eventually withdrawn. The short answer is that there was an uproar—an outcry—from the research community from the moment the paper was published. When it was retracted is another matter; I agree that that took too long. I do not think that you can fault the scientific community for picking up the fact that the work was deeply flawed—wrong.
Q563 Carol Monaghan: Another study that was done—and one I have spoken about in this place—was the PACE trial for ME. One of the issues with that is that we are still struggling to get the data released. If we are talking about research integrity, do you think that, as a rule, all data should be made available to all people who need to see them?
Professor Walport: Open science, which we strongly support, requires that data are made available. Therefore, we expect data to be made available.
Q564 Carol Monaghan: This is another case where The Lancet has not withdrawn the paper, but the researchers are refusing to release data. In fact, I think that they have said that a lot of the data have been lost. Do you agree that that does not do UK research integrity any good?
Professor Walport: We will be happy to write to you about the PACE trial, if you would like something more specific. I have some briefing on it. I am aware that a number of aspects of the trial have been criticised, including methodology. The trial team made it clear that there were some changes in the statistical analysis plan, but that decision was taken before data were examined. This was rectified by the trial steering committee.
Q565 Carol Monaghan: If you are happy to write to me or to the Committee, the information I am looking for is about release of data from the team.
Professor Walport: I am afraid that I will need to write to you about that.
Chair: We would appreciate it if you wrote to us specifically addressing the issue of transparency.
Professor Walport: Okay.
Q566 Chair: In response to Graham’s question, you said that UKRI will keep records. Are you also willing to commit to publishing the information, again focusing on the importance of openness and transparency?
Professor Walport: There are two separate questions there. Open science is very important. I spoke about it at the soft launch of UK Research and Innovation at the Institution of Civil Engineers. It is part of the whole drive towards intelligent transparency. We will publish within what the law allows us to do.
Q567 Chair: Going back to the concordat point, there appears to be quite varied practice from one institution or university to another in whether to involve an independent person in its process of inquiry or investigation and in the panel that hears any allegation. Of course, there is concern about universities marking their own homework. Do you take the view that there ought to be an independent person, certainly in the more significant inquiries that universities conduct into issues of research integrity?
Professor Walport: UKRIO will provide guidance on that. I am sorry, but I do not know what its guidance is in respect of that. One has to take a proportionate view, but I can certainly see a case for significant cases of misconduct having some independent membership.
Q568 Chair: Do you think that that may be one area where the concordat can be tightened up—again, focusing on the importance of demonstrating that we are demanding the highest possible standards?
Professor Walport: I would be happy to look at that.
Chair: If you could write back to us on that, I would be very grateful.
Q569 Stephen Metcalfe: Good morning, Sir Mark. As you know, the Committee has taken quite a wide interest in the network of departmental chief scientific advisers, as custodians of all things scientific within individual Departments. Not unsurprisingly, the Chair wrote to all the chief scientific advisers asking for their views on research integrity inside their Departments. We will publish the letters today. Have you seen them?
Professor Walport: I have. I am not sure that I would pass a detailed examination on them now, but I have seen the letters.
Stephen Metcalfe: Fair enough.
Professor Walport: There is quite a volume of them.
Q570 Stephen Metcalfe: We are pleased that there is a volume of them, because that means that there is a chief scientific adviser in almost every Department. In what capacity did you see those letters? You are now chief executive of UKRI, not designate.
Professor Walport: I saw them because I was shown them as background information. But you are right; it is not my current responsibility.
Q571 Stephen Metcalfe: So you will not have any oversight of Government research.
Professor Walport: Absolutely not. That is a matter for my successor as GCSA.
Q572 Chair: Is this a favour to the old boss?
Professor Walport: Clearly, one of the important things that UK Research and Innovation will do is talk to Government Departments. Of course, the publication of statements of research needs is an important step in that direction. But, no, I am now the chief executive of UK Research and Innovation, not the chief scientist.
Q573 Stephen Metcalfe: You were previously the Government’s chief scientific adviser. When you were in that role, were you ever concerned about the integrity of research that was undertaken in Government? From your brief look at the letters, were you surprised by any of the responses that we received? They were quite varied.
Professor Walport: The short answer is no. Obviously, the integrity of Government research is as important as the integrity of any other research, so it is an important issue, but I did not have any specific concerns.
Q574 Stephen Metcalfe: So it would have been a nil return for Government, under the concordat, during your tenure as CSA.
Professor Walport: I think so, although, did I look at the returns from individual research organisations? No, I did not.
Q575 Stephen Metcalfe: More broadly, do you think that the Government should produce their own concordat-style guidance narrative to reassure people that research that is undertaken in the name of the Government complies with the same integrity standards as everyone else’s?
Professor Walport: This is a concordat that can include Government-funded research, as well as any other research. It does not seem to me that we need lots of different concordats; it seems to me that the general principles of this concordat are generalisable. It is interesting. I have looked at the equivalent Australian paperwork, and it is very similar. When you look around the world, they all say essentially the same thing on principles.
Q576 Chair: Should Government Departments specifically sign up to the concordat?
Professor Walport: That is not a question that I can answer. It is a matter for my successor.
Q577 Chair: But you would welcome it, presumably.
Professor Walport: I can certainly see an argument for it, but I do not believe you would want me to bind the hands of my successor.
Q578 Stephen Metcalfe: No. However, as you have done the role very successfully for some time, we could ask whether you think that you would sign up if you were still in that role.
Professor Walport: I would look at it very seriously.
Q579 Stephen Metcalfe: Very good.
As I said, we take an interest in the network of departmental chief scientific advisers. The Science Minister told us that “the standards for research funded and undertaken by Government departments is the responsibility of departmental Chief Scientific Advisors.” That means that there has to be one in every Department that undertakes some form of research. At the moment, the Department for Education, which I think is quite an important Department to have a chief scientific adviser, has had a vacancy for some months. We are told that it is reviewing that. I know that it has a chief analyst, but it is considering whether to get a chief scientific adviser. Does that worry you?
Professor Walport: Again, this is really a matter for either the interim GCSA, Professor Chris Whitty, or my successor, who will start in less than a month’s time.
Q580 Stephen Metcalfe: Will you give us your personal view on the importance of the network of departmental chief scientific advisers?
Professor Walport: It is extremely important, given that a role for UK Research and Innovation is to talk to Government Departments about the public policy research needs, that we have counterparties to talk to. I am very pleased by the progress that has been made on the publication of statements of research needs. I am looking for a good counterparty and I look forward to working very closely with my definitive successor as Government chief scientist. Indeed, I have worked with Chris Whitty during the last few months.
Q581 Chair: Given that the Minister is clearly saying that it is the responsibility of the chief scientific adviser in each Department to maintain the Department’s integrity of research, does that not imply that every Government Department needs to have one, because if they do not no one is responsible?
Professor Walport: It depends on the scale of the research that is undertaken as part of the Department.
Q582 Chair: But surely if a Government Department is undertaking any research someone has to be responsible for its integrity, do they not?
Professor Walport: I am sure that is true, but a large amount of the research in Government Departments is undertaken by the analytical community. The analytical community is not in general answerable to the individual chief scientist. Again, it is not really for me to answer that question. The general principle, if you like, must be that the employer is responsible for the integrity of the behaviour of its staff, and should be responsible for providing the training and oversight. It is a matter for each employing Department. Again, it is quite difficult for me in this position to answer that question.
Chair: It reinforces the point about why the Minister needs to be here.
Q583 Damien Moore: At present, research misconduct proceedings are handled by the employer under employment law. Would you consider an Australian model, creating a court of appeal to review how universities respond to allegations of misconduct?
Professor Walport: The Australian model is only marginally different from the UK model. The principles are the same. They have their code of conduct. As you say, the employers do the investigation. What they have done is created a committee—the Australian Research Integrity Committee—which started in February 2011. I did look at a number of cases that they have looked at.
In 2013 they took action on three matters. In 2014 there were four matters. In 2015 there were two matters, and in 2016-17, where five matters were still being investigated, there was one. It is not really a court of appeal; it is an independent means for examining whether institutional process has been well followed.
Q584 Chair: Do you think that having it there, in a sense, focuses the minds and acts as a preventive measure to ensure that it is done properly?
Professor Walport: It is an interesting question. Of course, the Americans have a similar system in health, though not in other areas. Again, if you look at the number of cases, it is very small. Whether we should go there or just evolve what has happened in RCUK—the research councils—I have an open mind at this stage. It seems to me that there does need to be a very clear signal from above. Given we have responsibility for taxpayers’ money, I can assure you that UK Research and Innovation will send a very strong signal. We will collect data. Whether something like the Research Integrity Committee needs to be duplicated, I am not sure, at this stage.
Q585 Chair: But it is worth consideration.
Professor Walport: It is always worth looking at international best practice. Again, it has been helpful to me that you have had this inquiry because it has put this quite high on my agenda and the agenda of UKRI right at the beginning. We have looked at international practice. I would say that we are not more than one standard deviation from the mean of international best practice.
Q586 Damien Moore: How will you monitor the state of research integrity from your position in UKRI to assess whether the regulatory framework remains fit for purpose?
Professor Walport: At the end of the day we can only look at it through the lens of the institutions that employ scientists. As Graham Stringer has questioned, it comes back to some mechanism that could automatically scrutinise papers and find out what was right and what was wrong. Of course, if such a method really did exist, that would probably be the most powerful policing force you could have, but I think it is rather harder to do than one might think.
Q587 Damien Moore: At one level of poor research integrity, would you go to the Minister with options for changing the regulatory system?
Professor Walport: If we thought that the Government policy needed changing, we would certainly provide advice. But, as I say, I do not see any evidence to support the need for a change at this stage.
Q588 Damien Moore: What can a whistleblower currently do if they are not satisfied with the quality of a university investigation process? Would UKRI intervene at any point in that?
Professor Walport: I am not sure that I can answer that question because whistleblowing procedures ought to have built into them—if they are whistleblowing at the university—a mechanism for investigation of that independently; otherwise, it is not an effective whistleblowing procedure.
Q589 Damien Moore: Are you satisfied that that is being carried out properly?
Professor Walport: I do not know the answer to that question. I am not sure how easy it would be to find out the answer, actually.
Q590 Chair: I want to pursue the question about whether the framework is sufficient at the moment. We are conscious, as we have heard, of the fact that a quarter of universities are not filing annual reports. We appreciate it is a smaller percentage in terms of where the money goes—20%, I think. We are also conscious that among those that are filing reports very many of them are filing nil reports. Yet at the same time there was the statistic that Graham mentioned about the numbers who may be involved in dodgy activity. The work of the Nuffield Council on Bioethics in 2014 revealed that 58% of respondents to their survey reported that they were aware of scientists feeling tempted or under pressure to compromise on research integrity and standards, although evidence was not collected on any outcomes; and 26% of respondents had themselves felt tempted or under pressure to compromise on research integrity and standards. The figure was highest among those under 35 years. It was 33% lower in those over 35 years at 21%.
In other words, the impression one gets is that there is quite a significant issue below the surface, yet all these universities are filing nil returns—year by year there is never a problem. Does that not leave you feeling uneasy that we are not getting to grips with where it is happening?
Professor Walport: I am uneasy overall about issues of reproducibility, but I would reiterate that it is very difficult to extrapolate from one area to another. When you look at the retractions, which are just one measure, they are a very small percentage in relation to the amount of public funds diverted. Temptation is not the same as acting, but it cannot be a good situation that people are tempted to misbehave, although in all walks of life people are tempted to misbehave.
Q591 Chair: But one imagines that a percentage of those who are tempted succumb to that temptation.
Professor Walport: Again, we do not know the answer to the question.
Chair: We do not.
Professor Walport: What is much more important is that this then turns on what we need to do to increase the reliability of research. In spite of everything we have discussed, research is the gold standard method to find out the nature of the university and to ask important questions. It has never been bettered by anything else.
Q592 Chair: Of course, and in a sense that is why it is so important to maintain those high standards, is it not?
Professor Walport: Indeed, which is why I think we need to focus on conduct of research in general. I would rather focus on it through the lens of making research as reliable as possible and providing the right incentives to researchers. For example—and this is something for all parts of the system—we need to make sure that when we fund research it is adequately powered and adequately designed. Universities as employers, and research institutes, need to make sure that the necessary advice is available around statistics for that. We need to move away from the bias against reporting negative results because that does not help either.
Q593 Chair: Do you think there is an issue? The data make the point that younger researchers feel more pressured or tempted to cut corners. Is there a concern about pressure to publish?
Professor Walport: Again, I think that comes down to looking very carefully at how we support researchers at an early stage in their career. In other words, where possible we should give them long enough to allow them to tackle important questions. Of course, if you are under time pressure, that provides pressure potentially to take shortcuts.
To be honest, I think these are the important questions. Looking at all of this through the lens of misconduct potentially misses the really big issue, which is how do we fund research in a way that makes it as reliable as possible, but recognising that we are making extraordinary advances in research all the time.
If you look at two rather salutary examples, we have the discovery of gravitational waves—the most extraordinary and exquisite measurement done as part of a worldwide collaboration, discovering the most extraordinary things about the universe—and the episode of neutrinos that appeared to be travelling faster than the speed of light. That was not wickedness. It was the fact that a cable was not connected properly and a clock was not oscillating at the right rate.
Q594 Chair: A cock-up, not conspiracy.
Professor Walport: Yes. That was not wickedness. It caused quite a furore because you do not question easily the speed of light—or particles going faster than it. That was an example of research that was tidied up fairly quickly. It was a simple error.
What we need to encourage, rather than looking for blame all the time, is the creation of conditions where the maximum likelihood is that the research will be done well. Where I believe we ought to be putting most of our focus in UK Research and Innovation is not in setting up elaborate additional regulatory mechanisms around research misconduct. It is more about carrot than it is about stick—that is, carrot in how you provide the right sort of funding in the right quantum and at the right length. It is about working with journals. It is about working with peer review panels. It is about really focusing on what it is to have a good research career and how you do research well. It is not simply about blindly repeating every single experiment. That would not be the best use of the public purse.
Q595 Chair: I can completely see the case for that focus being where you say it is, but presumably you would say that at the same time there have to be effective mechanisms to root out where there is misconduct.
Professor Walport: Absolutely. We have to take it seriously where it occurs, but, as I say, I think we need to go where the money is, and most of the money is around making sure that research is done in the best possible way. As I say, I am not convinced there is any clear evidence that shows that this is a worsening problem. The volume of research is going up. How much of the new reporting is just simply better we do not know.
Q596 Vicky Ford: That leads beautifully to my questions, which are all about carrots, sticks and training.
RCUK told us it has never withdrawn funding due to non-compliance because it prefers carrots to sticks. Do you think that UKRI should take a firmer approach and that maybe there need to be some sticks for non-compliance with the concordat?
Professor Walport: Yes, proportionate to the scale of the crime. I am sure that is right. I could certainly envisage circumstances in which a grant should be stopped.
Q597 Vicky Ford: Do you suggest a single assurance process for verifying compliance with the concordat, or do there need to be many different processes involving research councils and Research England in different places?
Professor Walport: No; I think that is one of the reasons that UK Research and Innovation was created, so that we should have one assurance process across the organisation. On Research England working with the devolved Administrations, there are seven research councils in the UK reserved for the whole of the UK. Research England is England- only; it does not work with HEFCW, SHEFC or the Northern Ireland Executive.
Q598 Vicky Ford: I have enjoyed the mentions of gravitational waves in the past and neutrinos, but I want to focus for a bit on medical research. That is one area where this Committee has heard in evidence the suggestion that sometimes medical research is incentivised to publish the trials that have a positive outcome but not to publish trials or results where there is no evidence of a positive impact of the drug, for example. It has been suggested that researchers should be forced to publish even if they have not come up with any positive outcome; and that researchers who do not publish should be prevented from getting funding in the future. What is your view on that?
Professor Walport: It is a clear input. Unlike gravitational waves, there is substantial potential for medical research to cause direct harm to humans or, indeed in the case of animal research to cause harm to animals, or indeed in environmental research to cause harm to the environment.
Quite a lot of work has been done in the medical field. Going back to 2010, a systematic review demonstrated that about 30% of clinical trial results had not been published. The MRC has taken this extremely seriously since then, as has the Department of Health. When they reviewed awards from 2011 to 2016, they found that of 107 trials 94% were publicly registered in the clinical trials registry. Of course, that is the key thing because you can know if a trial has not been published only if you know it happened in the first place. The MRC is following up with those that are not registered. The first step is to register the trial.
Of 40 completed trials, 82% reported at least one publication, although only half included the main trial result, whereas 8%—three trials—had been complete for over two years without publishing results.
The short answer is that this is a serious matter. The Medical Research Council has been taking it very seriously. Since 2013 the MRC has required the public registration of all MRC-funded clinical trials and to promote the publication of findings within 12 months of trial completion. That policy was extended to public health interventions as well as clinical trials in 2016. The MRC monitors trial registration and publication of clinical trials and it has an ongoing annual monitoring programme, where it is contacting researchers where registration or publication information is missing.
I think you are right that this is an important area, but also it is an area that the MRC has been taking extremely seriously and which we will continue to take extremely seriously as part of UK Research and Innovation.
Q599 Vicky Ford: Quite a lot of numbers came out quite quickly there, but what I think I heard you say is that the trials need to be registered—
Professor Walport: They should all be registered; 94%, yes.
Q600 Vicky Ford: About 80% of those are reporting but only half of those that are reporting gave the main trial outcome. It seems to me to be still quite a potential gap.
Professor Walport: Yes. The review is going to be repeated this year and the report of that study is publicly available. You are right to raise it as an issue. It is a well-known issue in medical research. It is one that has been taken on, and of course trial registration is very important. We should be heading towards 100%, although there will be circumstances when a trial fails, for reasons such as that it does not recruit. There are reasons for trials failing that are neither negative nor positive. They have not recruited; they are a nil return. We should be heading towards making sure that there is maximum transparency of the results so that one can intelligently work out what has happened.
Q601 Vicky Ford: One of the suggestions that have been made to us is that, if a trial does not report, the people who ran that trial should find that as an obstacle towards receiving funds in the future. I am wondering whether you think that is an appropriate stick, or whether you can think of other carrots that might be helpful
Professor Walport: One would have to look at the circumstances for why it was not reported. As I say, if they did not recruit, that would raise another concern, of course. That is a concern that a peer review committee would be likely to look at when assessing whether to fund a clinical trial—the track record of the investigators in terms of what they have done in the past. As I say, if for some conscientious reason they were not able to recruit, that should not bar them from further studies.
Q602 Vicky Ford: We have been told during the inquiry that there were cases of researchers being let go quietly by their institutions when misconduct was uncovered, only for the individual to go to another university and repeat the situation. Do you think there should be some sort of a blacklist of researchers? Is that too simplistic?
Professor Walport: One would obviously need to be able to do that within the constraints of what the law allows in terms of data handling. Subject to it being legal, I can see a good argument for doing it.
Q603 Vicky Ford: You described earlier on in this session that for some trials it is just the wrong facts or the wrong evidence or human error.
Professor Walport: There are good reasons why trials fail.
Q604 Vicky Ford: You also used the word “wickedness.” A small number of people giving evidence have suggested that that wickedness or wilful manipulation of a scientific trial should be a criminal offence. What is your view?
Professor Walport: Obviously, if it were a criminal offence, it would be much easier to come up with a blacklist, as you call it, because by definition this would be in the public domain. I think that is one of the issues with coming up with lists: one would need to have a very robust process to ensure that it was proper to put someone on that list. It is quite hard to think of anything short of a legal process that would enable you to do that, which is why coming up with a blacklist does not seem a trivial thing to do.
Q605 Chair: But do you think there is a case for introducing a criminal offence, as Vicky suggests?
Professor Walport: I think that is a policy matter for the Government. I can see the arguments in every direction.
Q606 Vicky Ford: On training, the Association of Medical Research Charities suggested that there needed to be centralised training rather than lots of different fragmented efforts. Would you agree?
Professor Walport: No, I would not completely agree. I think that training needs to be distributed. It is a bit like health and safety. You would not have centralised health and safety. It is a responsibility for all researchers. It starts with undergraduate students. If you are going to understand the power of science, you need to understand how important it is that it is done well. Graduate training programmes should include training on research integrity, but also on the best way to do research, so statistical matters, study design and things like that.
The Council for Science and Technology produced in about 2007 the idea of a universal code of ethics for researchers. I think that is worth another look because it very clearly captured the key issues around research and research integrity. It is not something where you should centralise the training; it should be ubiquitous.
Q607 Vicky Ford: The Committee has had another suggestion that there needs to be better training on statistics and to have statistics training embedded in the earlier years curriculum.
Professor Walport: That is undoubtedly true and has always been the case. I think it is also about recognising that with very large studies, where the numbers are very large, this requires substantial statistical expertise and may require proper statistical collaborators—people who are part of the study and are expert in statistical analysis. Increasingly, this is about multidisciplinary and interdisciplinary science. Again, it is one of the reasons why UK Research and Innovation was created. It is about building the teams that can tackle these questions in the most robust and reliable fashion.
Q608 Vicky Ford: I just wonder whether the statistics modules of maths A- level have kept up with the rapid change in big data analysis.
Professor Walport: I would be surprised, but I have not looked at the A-level curriculum so I should not comment.
Q609 Vicky Ford: We get lots of complaints in my family about the level of interest in the statistics modules at maths A-level from those who are currently sitting it. That is just a thought.
Are we giving PhD students enough training in research integrity, and are you content with the RCUK? It said it is a dip-sticking approach to this.
Professor Walport: Yes. The REF is another place for looking at this in terms of institutional policies. It is an important part of reviewing doctoral training programmes as well. One of the advantages of doctoral training programmes is that they actually provide the opportunity for additional education as part of the PhD programme.
Q610 Vicky Ford: On a serious point and going back to statistics, it would be very interesting to understand whether other countries have the same issues. We have researchers from all over the world here. Is there better statistics training—
Professor Walport: Again, if you look through the lens of Retraction Watch, both the UK and the US have about half the rate of the world at large. That probably reflects similar research cultures, and part of that is having proper training. Certainly, when papers are reviewed, a statistical review is part of that. How expertly it is done depends on which reviewers are chosen.
All these things require multiple checkpoints. One of the things that should be reviewed is the power of the study to answer the question and whether the primary hypotheses are clearly set out.
Q611 Graham Stringer: You accept the case for full publication of results. Do you think that the Government should use its purchasing power to insist that that process is followed for private research in drug companies when there is not full publication? We have mainly been talking about the integrity of research in the publicly funded area.
Professor Walport: I think that probably the right lens to look at that is through the lens of the regulators of the products, who need to be assured that the data on which they are regulating are robust. I think it is a matter for them. I am not sure that if research is privately conducted there is the opportunity for any Government intervention. It only comes at the point where there is a regulated product as a consequence. That would be the route by which one could look at a trial registration. Indeed, regulators are increasingly looking at the whole question of trial registration. I think that is the lever.
Q612 Carol Monaghan: At a previous session we heard from the UK Research Integrity Office. They were at pains to tell us that they are not a regulator and that they do not investigate misconduct, but they also explained to us that they do not have an official status and that the only funding they receive is provided by subscribers.
Professor Walport: Correct.
Q613 Carol Monaghan: Do you think there is a role for Government or for UKRI in funding UKRIO or promoting this to researchers?
Professor Walport: UKRIO, as you know, provides advice to employers on how to conduct investigations. If they want to come to talk to UK Research and Innovation about subscription, we would happily talk to them about that. In terms of promoting it, I think it is a well-recognised part of the system and the universities are extremely well aware of UKRIO.
Q614 Carol Monaghan: They explained that on the academic side they had a good level of subscriptions, but they did tell us they only had one private subscriber, which clearly is an issue. A lot of UKRI funding, and especially from Innovate UK, is going to the private research sector. Is there an opportunity for UKRI to push subscription for private companies to UKRIO?
Professor Walport: There are an awful lot of companies. The question is how many of them they would need subscriptions from. The question is actually whether they are able to do their job effectively with the funds they have available at the moment. As I say, we can have leverage through funds that are applied through the public purse—so Innovate UK as well as the research councils, and that is one of the advantages of bringing Innovate UK into the same family that includes research councils. We will certainly look at Innovate UK.
Q615 Carol Monaghan: So you can see that there will be a role for UKRI in promoting engagement—
Professor Walport: I think our job is to promote engagement and make clear that public funds should be used with integrity wherever possible to ensure that research is done that is reproducible and reliable.
Carol Monaghan: I think we will look forward to seeing whether the private company subscribers—
Professor Walport: That is another matter. As I say, I am not sure whether subscribing to UKRIO is a test of whether you take the matter seriously. I am not sure I would view it as a useful benchmark for the private sector.
Chair: We have completed all of our questions, so thank you very much indeed for attending. We appreciate your time.