Written evidence submitted by the Academy of Medical Sciences (ALG0055)

Summary

• Algorithms based on machine learning and artificial intelligence (AI) have the potential to transform different processes across the healthcare system, from biomedical research and drug development through to healthcare delivery.
• Algorithms are already used in medical research and healthcare services, and their use in these areas is certain to increase in the future. For example, they allow clinicians to work more efficiently and better handle complex information through informing decision-making processes, and also help to empower patients to take a greater role in health management through facilitating shared decision-making between patient and clinician. Despite supporting and enabling clinical decisions, they will not replace the need for clinicians.
• Although the extent of potential benefits of using algorithms is not yet known, they are predicted to have a sizeable health and economic impact and it is important to fully understand this impact and the efficiencies afforded in research and development processes. The effectiveness of different types of algorithms and their relative utility should be the subject of evaluation.
• The development of algorithms for use in healthcare and research and development (R&D) should be prioritised and informed by robust evidence, the source of which must be open to scrutiny.
• Decision-aids must be developed based on robust evidence to ensure reliability and accuracy. We support the National Institute of Health and Care Excellence (NICE), in discussion with NHS Choices, coordinating the development and review of such decision-aids.
• Robust machine learning algorithms are dependent on large, high-quality datasets. Discussions around sharing and use of health data are therefore integral to the development of algorithms. Data quality and inputs need to be standardised across the healthcare system.
• Regulation of algorithms needs to be responsive and dynamic for future developments so that regulation does not stifle new innovations but provides adequate safeguards against any risks of algorithms including probability of errors and data privacy.

Introduction

1. The Academy of Medical Sciences promotes advances in medical science and supports efforts to ensure that these are translated into healthcare benefits for society. Our elected Fellowship comprises some of the UK’s foremost experts in medical science, drawn from a diverse range of research areas, from basic research through clinical application to commercialisation and healthcare delivery.
2. We welcome the opportunity to respond to this inquiry into the use of algorithms for decision-making. The Academy is monitoring the developments in, and applications of, machine learning and AI in medical science and decision-making, and is seeking opportunities to contribute to this debate. We are engaging with the research community around the use of algorithms through various workstreams, including projects on: improving the use of evidence to judge the benefits and harms of medicines; health apps; real world evidence; multimorbidity; and regulation and governance of health research.[1],[2],[3],[4]
3. Algorithms are widely used in R&D to guide drug development and study design. They are also employed extensively in healthcare delivery to guide decision-making and interpret complex data. Recently, increases in availability and collection of data matched by advances in computing power are driving the development of more sophisticated algorithms. Key to this is machine learning – using such algorithms – to support and supplement research and development and care pathways.
4. Machine learning is an algorithm that uses a set of training data to learn how to spot patterns in very complex data sets. Training data comprises data which has already been analysed by a human, for example a magnetic resonance imaging (MRI) scan which has been determined to show a tumour. The algorithm uses this training data to learn how to interpret the data, and the algorithm is subsequently applied to a new set of data – the ‘test data’. The performance of the algorithm can then be compared to a human and it can be further modified and refined to make it more robust.
5. Machine learning algorithms are more precise and sensitive when learning from a large, high-quality set of training data, which allows them to be successfully applied to therapeutic areas where there are large, high-quality datasets linked to defined clinical outcomes. It is therefore important that data collected across the health system is homogeneous and cross-compatible to allow it to feed in to algorithm development.

Current and potential use of algorithms in research and development

6. Algorithms are commonly used to guide R&D processes such as computer-assisted drug design, clinical trial data interpretation and clinical trial simulations such as pharmacological modelling.
7. Randomised clinical trials (RCTs) are often used to generate information on the safety, efficacy and effectiveness of medicines. However, they use specific controlled patient populations to generate accurate results by limiting the impact of confounding variables, and so it can be challenging to generalise results to the wider ‘real’ patient population which is more complex than those typically used in RCTs. Algorithms can be applied to simulate drug effects in more diverse populations or those in which drugs cannot easily be tested such as children.[5] Simulations could also be used for license expansions or drug repurposing without the need for expensive and lengthy Phase III trials.
8. Additionally, algorithms can further our understanding of biology and pathology by detecting patterns or features that have previously been missed by researchers.

Examples of algorithms used in research and development

• Physiologically-based pharmacokinetic (PBPK) simulations are becoming increasingly important methods of clinical trial simulation. For example, for simulating dose-response and drug-drug interactions to aid study design, or modelling drug effects in patient populations that may be difficult to trial.[6]
• Algorithms are being used in primary research to uncover new biology. An example of this is a Stanford-developed algorithm that, using histological images, uncovered new morphological features of breast cancer that hadn’t previously been identified by clinicians using the same images.[7]

Current and potential future uses of algorithms in healthcare delivery

9. Algorithms are becoming increasingly commonplace in healthcare, routinely applied to calculate risk, aid diagnosis, generate medical images and support decision-making. These tools guide the clinician through the diagnosis and decision-making process.
10. Simple algorithms based on clinical pathways and questionnaires are usually modelled on NICE guidance and as new, improved tools are developed they are reviewed and reflected in NICE guidelines. The Health Research Authority’s (HRA) Research Ethics Committees could help to review the use of these tools but guidance and training may be needed to support effective appraisal.
11. One of the key strengths of algorithms is to find patterns and correlations in complex data which may be too complex for human interpretation, such as genetic information. An algorithm may be able to find links that suggest a current or potential health problem and the appropriate course of action to take. This strength allows algorithms to be applied, in particular, to three areas where complexity of data limits human interpretation: prediction of future health problems; improved decision-making tools that can help to treat multimorbidity; and diagnosis aids for complex or rare diseases.
12. NICE, in discussion with NHS Choices, should coordinate the development of decision-aids based on this robust evidence generated by these algorithms. The effectiveness of different types of algorithms and their relative utility should be the subject of research evaluation and supported by funders including NIHR.
13. NICE and the NHS should take an active role in outlining the unmet need that algorithms could fulfil to guide development of tools towards the areas with the most impact.

Clinical decision-support tools

14. Decision-support tools are key to supporting clinicians and patients to make informed decisions about diagnosis and disease management and can stimulate the patient to take a more active role in decision-making. This is particularly important for areas such as complex disease management and calculation of risk. Moreover, as average GP consultations often last less than ten minutes, it can support a more informed decision about the best route of care.
15. Choosing the correct patient pathway for managing multimorbidities is complicated. For example, patients may be prescribed drugs that contraindicate those given for a different

condition, leading to adverse drug reactions, which account for 1 in 15 hospitalisations.[8] Algorithms may help clinicians to better diagnose or manage patients with complex medical histories, by warning of medical errors, highlighting other aspects of the patient’s treatment, or allowing calculation of precision dosing based on a patient’s health record. There are also opportunities to better diagnose rare diseases, which are often mistaken for other conditions.

16. Algorithms and machine learning can aid diagnosis by automating (and speeding up) the interpretation of medical images and are, in some cases, more accurate than human interpretation.
17. Whilst some patients may be comfortable with a more automated system of diagnosis and decision-making, involvement in decision-making remains important to patients and the clinician-patient relationship will still form an integral part of the care pathway.[9] There will remain situations where a clinician is best placed to optimise care based on knowledge and clinical experience. As a result, these algorithms are expected to complement clinical care but not replace clinicians in the full clinical decision-making process.

Examples of algorithms used in healthcare delivery

• Simple algorithms used in healthcare can consist of score questionnaires and clinical practice pathways which guide clinicians on diagnosis and proposed disease management. For example, hypertension is assessed using the ‘Baseline assessment tool’ accompanied by clinical audit tools of ‘diagnosing hypertension’ and ‘drug treatment’.[10]
• An example of an emerging diagnosis aid is a University of Washington School of Medicine study that used 100,000 optical coherence tomography images to train an algorithm to detect age-related macular degeneration. The algorithm achieved sensitivities and specificities of over 90% and could therefore be used for automated screening of patients.[11]
• In addition to imaging, machine learning is finding uses in many other areas. For example, researchers have developed an algorithm which was trained using speech audio to allow hearing aids to enhance spoken words over background noise.[12]
• New apps, such as Evergreen Life, allow patients to combine their digital health record with their own self-recorded health and fitness data to receive tailored medical information.[13]

Future uses of algorithms for predictive and preventative care

18. Algorithms have the potential to predict the risk of developing a condition that would normally only be detected through symptoms by using data sources such as electronic health records, genetic screening and long-term health monitoring. This could then direct the correct route for preventative care or refer the patient to a clinician for further discussion.
19. Digitisation of primary care will help create opportunities for the use of algorithms through the availability of a wider variety of data. Long-term health monitoring can involve self-assessment and self-monitoring through the use of smartphone apps and wearables, monitoring metrics such as heart rate or distance walked, or home based-devices such as blood pressure monitors. These devices could be linked to send data to the GP surgery for use in care. In addition, patient access to digital health records could be complemented by algorithm-driven services such as adherence support or reward schemes, potentially adding value and improving health outcomes.

Regulation and governance of algorithms

Transparency of algorithms

20. Although algorithms actively employed in healthcare are often well-tested, it is important to ensure transparency where possible and the robustness of data generated.
21. Proprietary algorithms are already widely used in healthcare, particularly in medical imaging. For example, algorithms used in MRI convert data recorded by the machine to a visual 2D or 3D image that a human can interpret. This calculation would not be possible or practical for a human to carry out. These algorithms and associated software are owned by the manufacturer of the machine and are therefore proprietary and protected by intellectual property. Consequently, it is essential that these algorithms are thoroughly tested and found to be extremely robust.
22. The same analogy can be applied to new uses of algorithms. It is likely that in many cases, the algorithms underpinning decision aids will be developed commercially and thus the exact methods by which they work may not be publicly available. However, if the algorithm is proven to be robust by establishing its clinical advantage, accuracy and reliability over the alternative system, it could then be used. Despite this, commercial algorithms that use health records or other health data are likely to be put under enhanced scrutiny due to their use of patient data and their implications for human health.
23. The propensity for commercial algorithms to make mistakes and impart bias is measurable using test data and the results of these tests must be made available to regulators. In addition, dialogue between software developers and regulators should occur throughout the design process to ensure that the software fulfils regulatory requirements and to allow thorough and timely appraisal.
24. Software updates or modifications to an algorithm may alter how the algorithm works and how effective it is. Therefore new versions of an algorithm must be tested to ensure it is as robust as its predecessor – this cannot be assumed. Manufacturers should inform regulators of the software changes and regulation should be able to accommodate such iterative changes. The accuracy and reliability of very sensitive algorithms, such as PBPK modelling, is highly dependent on the input parameters used. This must be considered in the light of potentially significant changes to the results/outcomes of an algorithm compared to when an earlier version was used, and how this might affect previous decisions such as aspects of medicine licenses.[14]

Guidelines for transparency of algorithms

For publicly funded or published research, a series of guidelines exist to ensure that the information is interpretable and transparent. The REporting of studies Conducted using Observational Routinely-collected Data (RECORD) checklist gives guidelines that include describing computational or statistical methods, the types of data used and the outcome methods.[15] The guidelines from the Enhancing the QUAlity and Transparency Of health Research (Equator) Network similarly outline the criteria for transparency of published research for healthcare use.[16]

Regulatory safeguards

25. Algorithms are not usually 100% accurate. However, the frequency and types of mistakes can be objectively quantified and the associated risk deduced. This risk assessment should form a key part of the appraisal of an algorithm and it is widely agreed that any algorithm put into clinical practice should undergo the same scrutiny as any new guideline or tool, which includes efficacy and risk analysis.
26. One risk of a lapse in accuracy occurs due to ‘calibration drift’. This is when an algorithm is developed for a particular set of data, however, the relevance of that data may decline over time with the result that the existing algorithm, and any other algorithms based off this, may no longer be tailored to the representative dataset. Therefore, frequent tool reassessment is essential to ensure relevance and efficacy of the tools being used.
27. It is widely felt that in cases where an algorithm has a real risk of misdiagnosis or otherwise misinterpretation, a clinician should thoroughly review the algorithm’s decision. In the case of algorithms that calculate risks or probabilities, the algorithm is used to support an informed decision rather than to make the decision on a clinician’s behalf. Similarly, algorithms could have mechanisms by which they refer the analysis to a clinician if the data is not clear enough for certainty on interpretation.
28. However, the regulation of commercial in-house algorithms used in R&D may not be tightly regulated as companies are free to use whichever algorithms they wish as part of some R&D processes, as long as they are not having a direct impact on health or using publicly produced or owned data. In addition, algorithms may be more closely reviewed by regulatory authorities if they have directed alternative study designs or contributed towards/impact licensing stipulations.

An example of calibration drift

An example of calibration drift is that of the EuroSCORE model for cardiac surgery. Over time, the model became poorly calibrated due to changes in the population including average age of surgery, decrease in mortality and changes to the types of surgeries being performed.[17]

Data sharing and privacy

29. The accuracy and robustness of algorithms is dependent on the quality of, and access to, both the data used to build and test the algorithm and the data inputted into the model.
30. Therefore enabling access to high quality data sources is key. It is important that patients, clinicians and other key stakeholders understand the value of health data and how it is used by algorithms for research and healthcare decision-making, to help them make informed decisions about contributing and sharing data. Further to this, it is vital to note the importance of data quality, as well as quantity, to ensure high quality data collection. The sharing of data, particularly with commercial companies, can be contentious and so there needs to be clarity and transparency around where, how and why this data is shared for this purpose.
31. It is important to acknowledge that no mechanism of data anonymisation – particularly pseudo-anonymisation – will be entirely risk-free, but that the likelihood of de-anonymisation is low and steps can be taken to minimise this risk. Appropriate safeguards that promote accountability and best practice in use of data, and appropriate sanctions for breaching data privacy, will help to reduce the risk. In addition, good data governance practices are essential for those handling data, and these are supported by various guidance and legislation including the Information Commissioner’s Office, the National Data Guardian’s Review of data security, consent and opt-outs,[18] and the new EU General Data Protection Regulation 2016, which comes into law in May 2018.
32. Historically, patients consent for any aspect of their health data to be shared for a specific use. If the terms of use change, re-consent is usually required to ensure the patient remains informed about the use of their data. However, new consent models may be required to maximise the potential of algorithms by allowing new algorithms to access existing data sets. Consent models across the health system should be homogeneous and standardised to ensure that patients are informed and developers understand what data is available to them, and this will be supported through implementation of a UK-wide consent model as recommended in the National Data Guardian’s recent Review.[19]

Examples of data sharing systems

• The 100,000 Genomes Project is a successful exemplar that demonstrates how a large, data driven project can be carried out in the NHS and has identified technical solutions for storing, analysing and sharing data that can be applied more widely.[20]
• An example of a successful data sharing tool is Babylon, which has recently been adopted by the NHS Digital tool library.[21] Patients willingly share data with Babylon through multiple choice questions in return for health advice based upon the answers. Babylon can then accrue the data that patients input to increase the accuracy of the advice.
• An example of a broad consent model is that used by the UK Biobank in which patients consent to ‘long-term storage and use of this and other information about [the patient], for health-related research purposes’.[22] This approach allows the use of data for new applications without requiring patient re-consent.

Current and potential regulation

33. Any software or algorithm that is to be used for a diagnostic, prognostic or therapeutic purpose is considered to be a medical device under EU Directive 2007/47 and must undergo the same validation.[23] The Medicines and Healthcare products Regulatory Authority (MHRA) has a key role in regulating the use of such algorithms for medicines development, for example where they may influence medicines licensing conditions such as their use for different indications or populations. The MHRA has guidelines for the requirements of digital medical devices such as apps and implants and the laws that cover their use.[24] However, these guidelines do not specify the process of validation and this may be the extent of UK regulators influence over home-based healthcare programs such as apps, which may provide informal health information rather than that contained in patient health records. It should be noted that there are other possible mechanisms for the implementation and evaluation of new digital devices such as through the Paperless 2020 initiative, and at a local level through Academic Health Science Networks (AHSNs), as proposed by the Accelerated Access Review.[25],[26]

Examples of data security and governance

• NHS Digital currently implements Clinical Risk Management training for organisations that are manufacturing or developing IT products for use in healthcare alongside the safety standards SCCI0129 and SCCI0160 and it is likely that this will continue to apply to future IT developments.[27]
• One form of data protection or regulation could be fingerprinting such as Google’s ‘Verifiable Data Audit’ (VDA) incorporated into databases.[28] VDA is designed so that every time a data record is accessed, shared or edited a unique fingerprint is added to show how and when the data was used. This would allow algorithm use across data to be recorded and the reasons for its uses critiqued with wider potential for data protection and access.

April 2017