Written evidence submitted by Dr Felix Greaves, Dr Chrysanthi Papoutsi, Dr Claire Reidy, Dr Anthony Laverty, Prof John Powell, Dr Bernard Gudgin, Miss Sukriti KC and Ms Salina Tewolde (DHS0020)

This evidence submission is informed by an evaluation of the roll out of the NHS App being conducted by Imperial College London and Oxford University. This work is funded by the National Institute for Health and Care Research (NIHR). Appendix A contains more detail on the project.

A. Was the commitment met overall? or (in the case of a commitment whose deadline has not yet been reached) Is the commitment on track to be met?

1. Does the commitment have a clear and fixed deadline for implementation?

The commitment for the first stage of NHS App roll-out (68% of adults until March 2023) has a clear and fixed deadline, and appears feasible given current rate of registrations, whereas the deadline for the second stage remains more vague (75% of adults by 2024). The commitment focuses on user registrations – a significant goal in terms of establishing authentication infrastructure through the NHS log-in (which is required for patients to register on the app). Yet, there is less emphasis on the extent to which it is anticipated the NHS App would be used routinely following registration to deliver service and patient outcomes.

2. Are there any mitigating factors or conflicting policy decisions that may have led to the commitment not being met or not being on track to be met? How significant are these? Was appropriate action taken to account for any mitigating factors?

The NHS App has been introduced into an already well-established and crowded ecosystem of patient portals and smartphone applications with similar functionality from major IT system providers and an established user base (following previous policy decisions in this area). These systems have been used for years by GP practices and their patients, therefore the rationale and incentives for switching to the NHS App does not always appear to be clear for users, and the impact on market and competition dynamics has been challenging. Awareness and perceived usefulness of the NHS App between our participants was low initially, as patients and practices were not familiar with the app and its functionality (and in some cases this continues to date). There was particular confusion between the NHS App and the NHS COVID-19 app despite public awareness campaigns.

3. To what extent has the NHS’s Covid-19 response affected progress on targets?

The pandemic and Covid-19 response had both positive and negative impacts on the roll-out of the NHS App. Awareness and familiarity with the NHS App increased significantly with integration of the Covid pass, which drove a significant rise in user registrations. It remains to be seen whether this will translate into sustained use for health-related purposes. At the same time GP appointment booking, one of the core functionalities of the NHS App, has been negatively affected by the shift to triage models in the context of the Covid-19 response, meaning GP practices had to disable direct appointment booking on the app. 

4. How has this commitment been interpreted in practice at trust/patient level?

Many GP practices publicise the NHS App on their websites and in interactions with patients (e.g. in relation to ordering prescriptions) although sometimes the NHS App is offered as one option among a number of patient portals/apps. There has been mention of patient and public representatives supporting other patients registering with and using the NHS App, but we have found little evidence in terms of this being current practice (also see response 2 above). 

5. Does data show achievement against the target (if applicable)?

Results of our quantitative analysis using data from January 2019 up to May 2021 shows that there has been strong adoption of the NHS App, which has been driven by COVID-19 related events. From January 2019 to May 2021, there were a total of 8,524,882 NHS App downloads and 4,449,869 registrations.

The highest number of downloads during the study period occurred after the announcement and launch of the COVID-19 Pass in May 2021, with a total of 2,668,535 downloads for that month. During this month, 2,099,234 users registered for the App. The number of GP registered patients in England at the end of May 2021 were 51,956,423. Of these registered patients, 8.56% of the population aged 13+ were registered for the NHS App.

Our evaluation did not include data after May 2021. Although the analysis is ongoing, preliminary results suggest high rates of download of the NHS App, with a potential to increase the registration rates. The NHS App team at NHS England and NHS Digital will have more up to date data. It will be important to monitor the ongoing relationship between downloads, registrations, and active use of the various functions the App provides.

B. Was the commitment effectively funded (or resourced)?

Our evaluation did not include a health economics component. Yet in our qualitative research we identified that there has been little dedicated funding and support (apart from generic ‘digital health’ budgets at CCG level) for GP practices to incorporate aspects of the NHS App into their administrative practices (e.g. appointment booking, ordering prescriptions) and engage with patient requests for support with the registration process, access to their records etc. 

C. Did the commitment achieve a positive impact for patients and service users?

1. What was the impact on equity of outcome different groups?

From a quantitative perspective, using data up to February 2021:

Compared to GP practices in the least deprived areas, practices in more deprived areas had fewer NHS App registrations (see Table 1 in Appendix B). Practices in the most deprived areas (IMD quintile 5) had a 25% lower registration rate (p<0.001) compared to the practices in the least deprived areas (IMD quintile 1). 

Practices with a higher proportion of white patients had higher App registration rates. For example, compared to practices with the lowest percentage of white patients, those with the highest percentage white patients had a 36% higher App registration rate (p<0.001). Similarly, practices with the highest percentage of the youngest age group (i.e., 15–34-year-olds), had a 23% higher registration rate compared to the practices with the lowest percentage of 15-34-year-olds.  

Practices with more GP registered patients (i.e., larger practice size) had higher NHS App registration rates. For example, practices with the largest practice size had a 44% higher App registration rate (p<0.001) compared to practices in the smallest practice size group.

Also, compared to the practices with fewer registered male patients, those with a higher percentage of registered males had fewer NHS App registrations overall. Practices with the highest percentage of males had a 13% lower App registration rate (p<0.001) compared to the practices with the lowest percent males.

Similarly, practices that had more patients with long term chronic illnesses or disability had lower App registration overall (p<0.001), although the size of the difference was comparatively small, with a difference of 2% between the practices with the highest proportion of people with greater health care needs compared to those with the fewest.

Overall, practices with higher NHS App registration rates were more likely to be less deprived, have a higher proportion of younger people, have a higher proportion of white patients and be larger in size.

Of note, these analyses were conducted at the practice, not individual, level.

From a qualitative perspective:

Given the NHS App is only available in English and requires a certain level of IT literacy and confidence, as well as relevant equipment, our participants have commented on specific social groups currently being excluded and the risks this would carry if the NHS App becomes a mainstream mode of engaging with the NHS.

To maintain equity GP practices needed to set up processes and safeguards so they did not disadvantage non-users, such as only making some appointments available on the NHS app and reserving slots for those who could only reach the practice by phone or in-person.

Core functionality of the NHS App remains the same across England. Yet, depending on commissioning decisions in different areas and availability of specific systems in different GP practices, additional functionality is available for some users but not others (e.g. integration with solutions provided by private companies such as eConsult, Patients Know Best, Accurx).

2. Has (or will) there been (or be) a meaningful improvement in measurable outcomes, reasonably attributable to the commitment?

Our research shows there is some potential for the NHS App to contribute to improvement in outcomes such as administrative efficiency (e.g. ordering prescriptions), if a critical mass of users engage in sustained and appropriate use, and administrative and clinical processes at practice level are streamlined to support this use. Yet, as with all technological solutions, any efficiency savings in one area usually translate into additional work in other areas. Healthcare work is underpinned by vast complexity, therefore many of the tasks assumed to be transactional (such as appointment booking) often require discussion, background work and negotiation to be carried out effectively, e.g. for patients to be able to choose the right appointment with the right clinician, access their health records with due consideration for safeguarding (especially consultations before automatic access to prospective consultations is enabled in November 2022), be provided with proxy access for minors or carers etc. It is unlikely changes in patient outcomes can be reasonably attributable to this commitment, although patient access to own records has the potential to influence safety in case significant errors are identified.

3. Will (or have) patients and service users benefit(ted) directly, indirectly or both?

There are benefits of the NHS App pertaining to supporting the navigation of complex, changeable, ongoing and/or ambiguous health conditions, whereby patients having access to letters, test results and appointment notes enables them to monitor and track different elements of health and to chart progress and change. This access also has the potential for patients to perceive a level of control over managing their health. In addition, Patients describe the ability to request prescriptions electronically through the app quick, easy and convenient.

For patients with more acute care needs, the app currently has limited capacity to support healthcare in those circumstances, however, having access to allergies, test results, medications, and consultations (for one’s own health, or others through proxy access) enables the potential to avoid emergency admissions or to ease worry around safety concerns. Not all patients or carers that we spoke to could access these records though, which were reliant on practices enabling full access.

In our research we have not come across an ‘enhanced range of services from local pharmacies’ other than functionality related to ordering prescriptions on the app and being able to collect from the local pharmacy. We have also come across little evidence that the NHS App is used routinely for correspondence with GPs (instead e-Consult or Accurx were used for this purpose where available). Functionality related to managing hospital appointments, viewing referral letters, as well as test results and patient records at secondary care level was limited and primarily linked to integration of a commercial PHR platform (Patients Know best) with the NHS App.

4. What category of patients and service users have benefitted? And why?

Please see response C1 on equity above.

5. Have (some) patients and service users been hindered by the commitment and its implementation?

Please see response C1 on equity above.

D. Was it an appropriate commitment?

1. Was (or is) the commitment likely to achieve meaningful improvement for service users, healthcare staff and/or the healthcare system as a whole?

Please see response C2 above.

2. Is the commitment wide enough in scope? Does it cover interoperability?

The commitment focuses only on the proportion of adults registered for the NHS App. This does not take in account whether the app is actually being used by patients and does not support routine app use to achieve measurable improvements. It is well known that mobile apps have relatively short retention rates across all categories including healthcare, with most users abandoning apps 30 days after installation. Broader commitment is necessary to ensure the app continues to provide added value to users, including by working more closely with healthcare providers (e.g. GP surgeries) as a key, but so far relatively neglected, group creating value for the app (e.g. by making appointments available etc.). Young and healthy participants in our sample particularly commented they did not see value in the NHS App beyond the Covid pass, as they did not need to have frequent interactions with the service.

3. Is the commitment specific enough?

The commitment is specific, measurable and necessary, but not sufficient (please see responses A1 and D2).

4. Has the commitment had any unintended consequences?

It is likely sole emphasis on user registrations detracted attention from developing the NHS App in a way that can provide sustained value for both direct/indirect users and those affected across the system (e.g. see response C1 on equity). Although users registered on the app, it has not always been easy for them to understand what functionality was available (especially as this has shifted over time and is also controlled by their GP practice).

5. Was the level of ambition as expressed by the commitment reasonable?

Given the Covid pass integration has already driven significant increase in patient registrations on the app, the ambition appears reasonable (but see responses A1, C1 and D2).

6. Is the target contained in the commitment an effective measure of policy success (if applicable)?

As explained above, success in the user registration rate on the NHS App is highly commendable and creates a foundation for authentication infrastructure and further digital solutions. Yet, registration does not necessarily lead to (repeat) use in a way that would add value for patients, healthcare staff and the service as a whole. Additional commitment, resource and implementation planning is needed to harness the benefits of digitisation through the NHS App. As it runs on legacy technical and classification systems (e.g. GP systems) patient information through the app remains largely system- rather than patient-facing (in terms of terminology, structure, availability and presentation).

7. Was the commitment addressing an identified need and relevant to the problem?

It is not clear from our data that needs assessment preceded initiation of app development, although a significant amount of ‘user research’ has taken place since to support ease of use and positive user experience. There has been less emphasis on user groups beyond patients and service users, such as GP practices across the country having to switch from similar patient access solutions to facilitate use of the NHS App, without the relative advantage always becoming clear.

9. Did the system have the relevant tools to support the change?

Beyond an initial roll-out stage where specific Beacon sites were supported with the implementation of the NHS App, there seems to be little concrete and ongoing support for GP practices to increase awareness among their staff and provide support to patients, especially considering groups that may be at risk of exclusion. There has been significant emphasis on user research involving patient groups directly, without necessarily considering the mediating role of healthcare organisations and how they may be considered as direct and indirect users affected by the shift to the NHS App in their practices.

Appendix

A. Project background and methods

Findings in this evidence submission are informed by the academic research project “Evaluating the national rollout of the NHS App in England”. This project is funded by the NIHR’s Health and Social Care Delivery Research Programme. Reference: NIHR128285

Abstract:

Aims and Objectives: This project aims to identify and understand the use and acceptability of the NHS App, measure the extent to which it improves patient experience and influences health service access, and understand patterns of early take-up and participation.

Background: The National Health Service (NHS) in England has introduced a new smartphone app for patients: the NHS App. This has been available to everyone in England from July 2019. Initial functions include online GP appointment booking, access to medical records and ordering repeat prescriptions.

Methods: The study has two workstreams. A qualitative element uses comparative case studies in 5 general practices to explore experiences and views on the acceptability of the App through interviews and focus groups with patients, GPs, practice staff, commissioners and policymakers. Iterative thematic analysis  allow us to develop a theory of change for the NHS App. Our theoretical approach is based on the Non-Adoption, Abandonment, Scale-up, Sustainability and Spread (NASSS) framework. A quantitative element looks at patterns of uptake and adoption of the app, and its various functions. This involves analysis of NHS App usage, together with practice level data linkage with the Patient Online Management Information (POMI) and demographic data. Primary outcomes are use of online appointment booking, access to medical records online and online repeat prescriptions, comparing changes before and after the introduction of the NHS App. Patient participation is done through a specific PPI group for this project, who are engaged in the design, delivery and interpretation of the research, including as co-researchers in the analyses. This work is being done over a 24 month period starting in November 2020.

Anticipated impact and dissemination: The study is being done in collaboration with the team at NHS England and NHS Digital delivering the App. The research feeds back findings to the delivery team and policymakers to inform their work through regular structured meetings between the research and delivery teams. Dissemination includes conventional academic approaches, but also conferences and online material specifically targeted at patients, GPs, and practice managers.

Some of the initial quantitative findings used to inform this response are available in this preprint scientific paper: An observational study of uptake and adoption of the NHS App in England. Tewolde et al. 2022.

medRxiv 2022.03.16.22272200; https://www.medrxiv.org/content/10.1101/2022.03.16.22272200v1. This paper is currently under peer review at a medical journal.

Qualitative findings are still being analysed and initial findings have already been presented at several conferences. Initial themes and findings are presented in our response for your information.

B. Tables [NB - These results have not yet been subject to formal peer review - but are in the process of being submitted to a medical journal]

Table 1. Negative binomial regression table showing unit change in the NHS app registration rate per 1000 GP registered population, with other variables held constant. For Index of Multiple Deprivation (IMD), the percentage difference represents change across the IMD quintiles, in comparison to the reference group IMD Q1 (i.e., least deprived practices). For all other variables, the percentage difference represents change across the variable quartiles, in comparison to the reference group 1 (i.e., practices with the lowest population percentage for the given variable).  Data up to Feb 2021.

*Percentage difference calculated using Incident Rate Ratios (IRR) obtained from the Negative Binomial Regression model.

Submission prepared by the project core team: Chrysanthi Papoutsi, Claire Reidy, Sukriti KC, Anthony Laverty, Bernard Gudgin, John Powell and Felix Greaves

We are grateful to the teams at NHS England and NHS Digital for providing us with access to data on NHS App usage and for their ongoing willing collaboration and engagement on this project.

Nov 2022