Written Evidence Submitted by Dr Paul Marchant
I made a submission (RES0042) http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/science-and-technology-committee/research-integrity/written/77124.html to the Research Integrity inquiry which reported in 2018. This concerned openness and transparency that are key to the scientific endeavour. RES0042 links to my earlier submission on a wider range of matters, RIN0083, made to the inquiry of the previous Parliament.
1. the reproducibility crisis is serious across a wide range of disciplines, particularly the softer sciences such as criminology / crime science, transport.
2. The issues in academia that have led to the reproducibility crisis are the pressure to publish, the shortage of time allowed, the lack of awareness of subtle statistical issues by those doing research and their masters, the lack of statistical support and insufficient scrutiny through thorough review. There is a lack of rigorous protocols to constrain data-dredging/p-hacking and swapping outcomes in the quest to obtain a ‘pleasing result’. Also, the lack of transparency and openness because of competition and insecure employment.
3. Seek and value statistical advice from properly qualified statisticians such as Chartered Statisticians of the Royal Statistical Society so that statistical issues may be resolved in the design and a tight protocol produced.
4. Have an easy to use public system where by protocols can be uploaded and time-stamped to show what is intended at the outset and how the research will proceed. At the end of the research, ensure that methods used are transparent and there is open data, so that others may check, having in their turn lodged a protocol to say that this is only what they are going to do. The research data could be analysed in different ways but again a protocol needs to be produced first. It is important also that independent researchers outside the formal academic system, such as retired statisticians, can easily use such a system and contribute to science. There is some discussion of these matters in my previous Research Integrity submissions.
5. A national committee on research integrity under UKRI should champion ways (as above) of producing trustworthy science.