Written evidence submitted by Mike Hearn (C190122)

Abstract. This document extends “Summary of methodological issues in epidemiology” [Hearn2020] to include Omicron modelling and classification, additional reform proposals and reflections of a general nature.

 

Omicron. The failure of SAGE modelling to correctly predict the course of Omicron was anticipated within an hour of the paper driving initial press coverage being uploaded1. The relevant report from LSHTM2 stated that “Due to a lack of data, we assume Omicron has the same severity as Delta”, however it was already known that data did exist and showed much lower severity3. As such the unnecessarily aggressive reaction of the Government could have been avoided by fact checking the assumptions on which SPI-M was basing its recommendations.

 

Utility of peer review. Incorrect statements have appeared repeatedly in the peer reviewed COVID literature. For example, a paper from SAGE members published by Nature Scientific Reports in August 2021 contained an incorrect claim about Swedish COVID mortality data in the first sentence4. Neither peer reviewers nor editors noticed and the paper was never retracted.

 

The formal peer review process wasn’t used for SAGE Omicron modelling, nonetheless, it is unlikely peer review would have detected the problems. Many academics contributed to the underlying papers.

Additionally, peer review is a sufficiently weak process that numerous cases have occurred in the past two years of journals publishing computer-generated gibberish. In August, a journal published by Springer Group retracted dozens of papers because the content of this article is nonsensical5. The journal in question has 45 people in editorial positions. Earlier in the year Springer had to retract hundreds of articles from the Arabian Journal of Geosciences, for exactly the same reason6. These fake papers appear to be designed to boost the careers of researchers who buy them. The problem is widespread throughout the scientific literature7, and can frequently surface in the appearance of nonsensical thesaurus-driven phrase replacements (e.g. “colossal information” instead of “big data”)8.

 

Omicron classification. Omicron has many mutations and causes different clinical outcomes to earlier SARS-CoV-2 variants, raising the question of why it’s classified as a SARS-CoV-2 variant at all. Viruses are classified as SARS-CoV-2 variants based on whether they can be slotted into a phylogenetic tree, not based on clinical outcomes or similarity to the original Wuhan strain. Thus, a virus being classified as a SARS- CoV-2 variant communicates no useful information about what any response could be.

 

The question of whether Omicron should really be considered a SARS-CoV-2 variant was raised immediately by a Stanford virologist after the request to classify it was made9. In private correspondence this virologist stated, “There is also strong political pressure not to talk about strains and serotypes, even if it is not explicitly formulated. Currently the ‘strategy’ for dealing with the pandemic is vaccination. And nothing else. Which is doomed to failure, one major reason for that being the continuous appearance of new serotypes and strains”.

 

Ideological bias. This type of ad-hoc approach can be seen frequently in the COVID literature and appears to originate in a form of motivated reasoning, in which ideologically desirable conclusions are selected and then scientific-sounding approaches are created to justify them. Researchers may not realize their models are ideologically biased, for example, weeks after the publication of the LSHTM report Prof Medley justified their approach by saying that “We model the scenarios that are useful to decisions” and “Decision-makers don’t have to decide if nothing happens”, apparently without understanding that this encodes a strongly ideological perspective (i.e. that government intervention is taken for granted). This may be due to the lack of ideological diversity in academia10.

 

Proposals for reform. The types of problems experienced with Omicron prediction are not restricted to epidemiology or modelling, they are widespread in academic research11. There are no objective criteria for a field to be classified as science, beyond whether or not it can get published. In particular there are no mechanisms to reward the use of the scientific method (or penalize pseudo-science) beyond peer review, which is unable to arrest a general field-wide decline in standards. This can lead to unexpected outcomes,

e.g. peer reviewed published papers claiming that people can see into the future12.

 

Reform efforts should therefore focus on external checkpoints like the granting process, e.g. by tying the payment of grant money to whether the resulting research passes various externally applied quality tests e.g. replicability, model validity, code quality review, statistical consistency tests13 and so on.

 

 

February 2022


1 https://dailysceptic.org/2021/12/11/lshtm-modelling-on-omicron-ignores-all-evidence-of-lower-severity- among-numerous-problems/

2 https://cmmid.github.io/topics/covid19/reports/omicron_england/report_11_dec_2021.pdf

3 The pattern of milder disease in Pretoria is corroborated by data for the whole of Gauteng province. 8% of Covid-positive hospital patients are being treated in intensive care units, down from 23% throughout the Delta wave, and just 2% are on ventilators, down from 11%.”, Financial Times, December 7th 2021.

4 The U.K. and Sweden have among the worst per-capita Covid mortality in Europe”, Mishra et al (ICL), August 2021, Nature, https://doi.org/10.1038/s41598-021-95699-9. At the time of paper submission Sweden had better per-capita mortality than the EU average and over the course of the review period, this gap continued to widen in Sweden’s favour.

5 https://link.springer.com/article/10.1007/s00779-021-01656-8

6 https://dailysceptic.org/2021/10/03/436-randomly-generated-peer-reviewed-papers-published-by-springer- nature/

7 “The fight against fake-paper factories that churn out sham science”, https://www.nature.com/articles/ d41586-021-00733-5

8 “Tortured phrases: A dubious writing style emerging in science”, Cabanac, Labbé, Magazinov, July 2021. https://arxiv.org/pdf/2107.06751.pdf

9 "Would we be even calling this a "variant" if we were not in the middle of the pandemic? If all you have is one sequence of SARS-CoV-2 and one sequence of this, sampled from bats, you probably call them "strains" and give them different designations…”, https://github.com/cov-lineages/pango-designation/issues/ 343#issuecomment-976989249

10 Lackademia: why do academics lean left?”, Carl 2017

11 https://www.vox.com/future-perfect/21504366/science-replication-crisis-peer-review-statistics

12 Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect”, Bem, 2011, Journal of Personality and Social Psychology.

13 The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology”, Brown & Heathers, May 2016