Written Evidence Submitted by Richard Keirnan, Pelory Limited

(RRE0084)

This is a proposed solution for the UK government to improve the availability and quality of science for ministers and UK officials, it has been compiled by Pelory limited by Richard Keirnan with the kind assistance of Azimuddin Siddiqi and Dr Asim Beg MBBS,PhD,FRCP(Edin).

Driving forces in reproducibility

Reproducibility has been a major issue for several years in scientific research. The Covid 19 Pandemic has merely highlighted its significant with devastating consequences with huge implications for human lives.

Scientific rigour depends upon consistent and uniform study designs. Selection bias and opportunistic data analysis result in outcomes which may not be truly representative of which data is significant. Such studies are very difficult to replicate. A major issue is the academic institutional monopoly of research and without sponsor mediated agendas. Academia’s ‘publish or perish’ philosophy has resulted in mediocre science and research which may not have any significant role in critical issues pertaining to relevant clinical scenarios. The ownership of data driven decisions by an axis of academia and funding agencies can result in politicised or deliberately advantageous inferences, a fact that was made apparent during the pandemic crisis eg. studies into the application of HCQ and Ivermectin.

Proposed Solution - credibility profiling:

Collaborative research, it is imperative that operational standards be in place that would award reproducible collaborative research with a higher credibility ranking.

Research needs to be awarded a credibility rank score, a maximum rank would be set which at the best practice of current standards of technology, as standards improve the maximum ranking of a given area would increment automatically reranked findings based on older less acceptable or poorer protocols, research which has been challenged successfully would be be more heavily  reranked, driving the ranking lower than than would be acceptable for reference in new work. 

New research with poorly traceable provenance might be indicative that the content required further meta analysis, review or is subject to unanswered criticism.  A very low value would be indicative of extreme bias or fraudulent claims.
 

Replication standards and culture need to be developed, that credible research should be replicated by a team operating independently preferably from another organisation using the same verified processes and data at which point the conclusions can be critically examined.

Methods of analysis and definitions require standardisation in order that transparency can be achieved, and ensuring malpractice and fraudulent intent cannot be obscured.

Registration, validation and sharing of practice protocols

A common protocol to check the validity of documents is to employ ‘Hashing’ a process of creating a key based on the contents of the document, the slightest change creates a totally different key

Formal registration of data sources, software, tools, and analytical processes to facilitate the verification to a fine degree data sources all of which carry a hash key, Data is ‘hash keyed’ when produced to prevent unethical manipulation of data sets, and cited via the hash and the source to ensure data is complete on subsequent research.

With fine granularity of verifiable resources, design of research processes needs to be shared, this will facilitate gauging the validity of practices and peer review can improve on processes assumed to be robust.

Statistical methods require evaluation and bench marking and validating as appropriate, and a ranking maintained

Block Chain

Block chain technology should be used to prove the completeness of the entire chain of research that no part can be manipulated or erroneously compiled.

The block chain would hold the credentials to improve peer review and reporting, require that the persons undertaking review are in fact a registered qualified authority and their personal provenance is acceptable, that they in fact did validate to the degree claimed.

This would be a blockchain of events is created when peer review takes place and is attached when quoted or is integral in the ranking process.

The continual refinement of study design standards held within a blockchain.

Training and education

A comprehensive education program is required for evaluation and statistical literacy, aimed at decision makers, training should be given to understand the appropriate statistical methods and the ranking system in order that high ranking officials can objectively judge materials presented to them.

(September 2021)