Was the commitment met overall? or (in the case of a commitment whose deadline has not yet been reached) Is the commitment on track to be met?

Was the commitment effectively funded (or resourced)?

Did the commitment achieve a positive impact for patients and service users?

Was it an appropriate commitment?

1. Does the commitment have a clear and fixed deadline for implementation?

1. Were specific funding arrangements made to support the implementation of the commitment? If not, why? If so, what were these, when and where were they made?

1. What was the impact on equity of outcome for different groups?

1. Was (or is) the commitment likely to achieve meaningful improvement for service users, healthcare staff and/or the healthcare system as a whole?

Yes

Not to my knowledge

There is not as yet data relating directly to the use of the NHS App on equity of outcomes and in general data is lacking on how health outcomes are affected. Evidence is available on how access is affected and we can assume that without access health outcomes are likely to be compromised.

Those in deprived groups tend to be left behind when services are digitised and deprivation is a key factor in digital inequality. For example, with online appointment booking there is a steep deprivation gradient with those in the most deprived areas least likely to be aware of or use online appointment booking (1). For online services in general practice more widely, patients registered in practices in areas of low deprivation are more likely to be aware of and use these services(3). The use of the NHS App is likely to follow a similar pattern as it is one of the platforms used by general practice for access to appointments, prescriptions and medical records. 

The commitment is focused on registration only. Registration with the NHS App is not a good marker for use of the app and so is meaningless in terms of potential improvement for service users, healthcare staff and/or the healthcare system as a whole.

This has been demonstrated via other online services in general practice, for example online booking, which has relatively low registration rates (as of today 42.23% of patients are registered) and even lower usage rates, with the 2022 General Practice Patient Survey showing that just 13.2% of patients booked online(2). This highlights the gap between a registration and a usage.

2. Are there any mitigating factors or conflicting policy decisions that may have led to the commitment not being met or not being on track to be met? How significant are these? Was appropriate action taken to account for any mitigating factors?

2. Who was involved in determining the funding arrangements? Who was ultimately responsible for this decision?

2. Has (or will) there been (or be) a meaningful improvement in measurable outcomes, reasonably attributable to the commitment?

2. Is the commitment wide enough in scope? Does it cover interoperability?

Not to my knowledge

Not able to assess

I do not believe it is possible to assess this. Presently in general practice there are multiple providers of the same functionality offered by the NHS App. Some patients will use the NHS App functionality but other will use other providers such as Patient Access or Accurx. Separating out the specific impact of the NHS App is likely to be very difficult and such data would be limited to those patients using just the one route to online access.

I do not believe the commitment is wide enough in scope. As detailed above, registration is not a good proxy for use, and even use does not necessarily lead to meaningful use. This would be better assessed by looking at patient journeys happening via the NHS App.

3. To what extent has the NHS’s Covid-19 response affected progress on targets?

3. Do healthcare stakeholders view the funding as sufficient?

3. Will (or have) patients and service users benefit(ted) directly, indirectly or both?

3. Is the commitment specific enough?

Covid-19 was of benefit in moving towards the target.

As the NHS App was used for Covid passports this led to a rapid increase in registration rates (12 million registrations from when the Covid pass was made available to Oct 2021). The figures available on how the other services were used (online booking, repeat prescriptions and online record access) demonstrate that registrations have not translated into healthcare use.

I am not able to directly comment on funding. More generally general practice is under resourced and has to find capacity for supporting patients who wish to use online services, over and above their existing commitments.

As the commitment is not specific this is difficult to assess. There is evidence that patients derive benefit from being able to access online services(4) in general practice and the NHS App is one way that this is now facilitated.

No. As described earlier in the grid, registrations do not equate to use, and do not reflect the complexity of getting patients to use and benefit from online services.

4. How has this commitment been interpreted in practice at trust/patient level?

4. Was any financial commitment a ‘new’ resource stream? If not, did reallocation of funds result in any unforeseen consequences/ undesirable ‘work arounds’ at local level?

4. What category of patients and service users have benefitted? And why?

4. Has the commitment had any unintended consequences?

Patients struggle to understand how the NHS App fits within their use of the healthcare system, due to the wide variety of platforms all doing the same thing in general practice.  Patients are familiar with the use of the App for the Covid passport rather than other functionality.

The use of digital services in general practice has required that patients are supported to use them and this has placed demand on reception and administrative staff in general practices, without any extra resource provided to support this.

Evidence demonstrates that patients with long term conditions are more likely to be aware of and using digital services (2, 5, 6) This may be because patients who need to access the general practice frequently are more exposed to digital services, become ‘expert’ and develop confidence. Ongoing research funded by NIHR has demonstrated that accessing repeat prescriptions online acts as a gateway to the use of other digital services(6, 7). This has also been demonstrated in research being conducted presently in Norwegian general practice and funded by the Norwegian research council(8), which shows that patients engaged with online consultations after firstly using an online platform to obtain a repeat prescription.

Ongoing research funded by NIHR indicates that the NHS App is not an important tool for general practice, sitting outside of their control and with them unable to directly help patients to register and use it(8) this has increased workload as reception staff personally attempt to help patients access the App or signpost them to resources on how to use it.

5. Does data show achievement against the target (if applicable)?

5. What factors were considered when funding arrangements were being determined?

5. Have (some) patients and service users been hindered by the commitment and its implementation?

5. Was the level of ambition as expressed by the commitment reasonable?

The target has not been met but thanks to the Covid pass the rates of registration have increased. As mentioned previously, registrations have not necessarily translated into healthcare usage.

Not able to answer

Some patients will have found it harder to access the NHS App than others, thus missing out on the functionality it offers.

At present, digital transformation does not serve those in areas of high deprivation and the challenges facing these areas are multiple and complex. A recent review looking at who misses appointments and why in general practice found that those with low socioeconomic status were more likely to miss appointments(9). Simply placing digital routes of access is not enough when external factors make it difficult for patients to be able to engage with them.

A single minded focus on getting people registered for the App neglects to consider the complexities associated with access to healthcare.

The introduction of digital services for accessing general practice has been demonstrated to be linked to multiple unintended consequences. In relation to NHS App functionality that allows access to general practice, specifically that access can negatively impact on patient understanding of their healthcare, patients not understanding how the services link to their healthcare and extra work for general practice(10, 11)

6. Who made commissioning decisions (local budget allocation)?

6. Is the target contained in the commitment an effective measure of policy success (if applicable)?

Not able to answer

I do not think so. It is too simplistic to provide any true assessment of whether patients are accessing their healthcare digitally.

7. Was the commitment addressing an identified need and relevant to the problem?

No, as above it is a very simplistic target that does not address a specific need or problem. It is focused solely on initial uptake.

8. How has working to those commitments affected other aspects of care?

General practice does not have additional resource to support patients in using digital services and the area that patients struggle with the most is registration(7). The NHS App sits outside of general practice and outside of their control which means they are unable to help patients to use it but are expected to support them when they have concerns. At a time when general practice is very stretched this takes resource away from other areas.