18
Examination of witnesses
Dr Stephen Harden and Dr Bernie Croal.
Q181 The Chair: Good morning, and welcome to this Select Committee on Science and Technology. We are pursuing our inquiry into the potential for innovations in personalised medicine and AI in medical specialisations. We are pleased to have this morning our first two witnesses: Dr Bernie Croal, president of the Royal College of Pathologists, and Dr Stephen Harden, president of the Royal College of Radiologists.
As you know, our inquiry is investigating the potential of innovations in AI and genomic medicine to provide benefits to patients in the NHS, particularly in the context of personalised medicine. In the specialities that you represent—pathology, radiology and oncology—can you give us some insight into how you see the innovative technologies of AI and genomic medicine benefiting patients in the NHS? What are you already seeing and what might be the potential of these technologies?
Dr Stephen Harden: Good morning, and thanks very much for the introduction. The way this will probably work is that I have more to say on AI and Dr Croal more to say on genomics.
However, if I may start with a comment on genomics, the Royal College of Radiologists comprises clinical radiology and clinical oncology. As a result, we are quite well placed to influence cancer waiting times for both diagnosis and treatment. As far as genomics is concerned in terms of cancer, it is quite likely that it will have a huge role to play in risk stratification, risk identification and advanced diagnosis and treatment, particularly with AI.
From a radiology perspective, we have a process known as radiomics, through which we are trying to distil prognostic data out of diagnostic images, getting down to the fine detail to look for clues of where we can have prognostic benefit. Combining radiomics with genomics will be a phenomenal advance. Add AI into that and you can see a pathway forming for the patient, from initial referral all the way through to treatment, which will be much more effective and efficient and bring the latest technology and advances to bear.
As far as the current situation is concerned, we are big supporters of AI in healthcare and in imaging in particular. AI has huge potential to make us more effective and potentially more efficient. We have seen to date that some of the tech is certainly enabling us to become a bit more accurate. The productivity benefits are a bit more limited. I can go into that in a bit more detail if that would be helpful.
Part of my own practice is heart and lung radiology. When I am finished here this morning, I will be going back to do lung cancer screening as part of my practice. AI is actively being used every day to support lung cancer screening, to help us to identify it and potentially to make us more efficient in cancer diagnosis.
My final point is around AI in radiotherapy planning. It is a wonderful example of where AI can genuinely make us more productive. For my colleagues in clinical oncology, who do a lot of the radiotherapy planning and delivery, AI is already being used to identify organs at risk. For example, if a patient needs prostate radiotherapy, the AI will identify the areas not to be targeted by radiation treatment, avoiding the bowel and intestines and concentrating just on the organ in question. We have evidence showing that that generally already speeds up the process.
What will come next is the specific targeting of the radiotherapy plan to arrange for the treatment rather than identifying the organs at risk. That is a little way off yet, but that is a nice example to show that AI is in practice and is being used to make the processes more efficient.
Dr Bernie Croal: Likewise in pathology services, although we are very much behind the curve from where radiology sits, we are incredibly excited about the potential for AI to come in and impact the services that we provide. Pathology, as you know, is a broad church with 17 very different disciplines, and AI has the potential to be used not just in the imaging-based histopathology areas but across the whole process—the whole sample pathway. We can potentially bring productivity gains from when the sample is taken all the way through to when the result is reported and interpreted by the healthcare professionals requesting it.
The real potential of personalised medicine is an area that brings great promise. We know that AI can be used in combination to bring all the data together from genomics, histopathology, clinical evidence and radiology, and really fine-tune and home in on what is important for individual patients. That information can be used to select the best treatment options and, in some cases, the best drugs. AI has huge potential across the whole of pathology. As I say, we are very much behind the curve from where radiology sits now but we are excited about it.
Q182 Lord Winston: Good morning, and thank you so much for coming. Given that you have told us of the potential that these techniques have, do you feel that they are being used anything like enough? If not, what are the major barriers to using this innovation?
Dr Bernie Croal: For pathology, the main barrier is the underlying state of pathology services, especially the maturity of IT systems. Many of our laboratory IT systems—and I can speak for my own system in Aberdeen—are more than 30 years old. There is no functionality to link in with imagery or with AI. Until we get a progression in improvements in those areas across the board, with systems that are standardised and interoperable, there will be a very slow uptake.
The second thing that limits us is workforce. Pathology has a huge workforce crisis, which is going to get a lot worse. That will absolutely limit and inhibit because we do not have that turning space; we do not have the right system IT specialists with the knowledge base to test, implement and maintain AI systems when they potentially come on board.
Lord Winston: Clearly, we will come back to that issue later this morning.
Dr Stephen Harden: I agree that the major problem is workforce shortages. We similarly have significant shortages in radiology: a 29% shortage in radiology and 15% in clinical oncology. Part of the issue is the time allowed to be able to introduce these developments, because they must be medically led. There is so much strain on the system, with the amount of work required and the shortage of workforce, which means that much more time and effort is diverted into direct patient care, quite understandably. Therefore, these developments are much more difficult to implement.
I have two other comments. The first is on IT. You will all be aware that the NHS IT system is suboptimal. One of my messages today will be hope over hype. To get the real benefit from AI, we have to have a reliable, functioning IT system to have the AI then build on top of. The other point is clinical trust in the tools that we are being asked to use. We set standards for patient care and, therefore, to bring in any new technology we need to be confident that it will do as well as what we have at the moment or potentially even better. For reasons we may come on to, we do not yet quite have the trust to be able to implement it fully.
The Chair: We will come on to trust. That is a key point.
Lord Winston: One issue—this is not meant to be in any way rude—is that you are effectively talking up the potential of this technology, as you should do, but, already in some cell biology for example, if you look at the analysis of embryos using AI and the subsequent results in growth and so on, they are not always totally convincing. How are you sure that you can get better validation of these techniques when you are looking at imaging and phenomics together, which seems to me to be quite a difficult problem in many ways?
Dr Bernie Croal: Inevitably, there is always a gap between what can be demonstrated in research—whether it can work—and whether it does work and whether it is worth it. We need to push proper validation of AI tools at the coalface to ensure that what they deliver is what they promise. There is a lot of hype around what AI can bring and we need to ensure that that is accurate.
We are finding in the early stages of planning that writing a business case in an individual trust or pathology network for AI is incredibly difficult because you are having to justify what the benefits are and what the cost saving might be when, in reality, there may not be any cost saving at all. One area we worry about is that the hype about AI offsets the important need for workforce expansion or workforce changes to be able to get to that point. We are always on the fence in terms of making sure that we are justifying AI when it can be justified.
Dr Stephen Harden: I agree with that. It is similar in radiology. Part of it is introducing it slowly and progressively to build trust and confidence on the ground. From my own experience in lung cancer screening, it took me some time to get up to a level where I thought that it was quite helpful. Initially, it took me longer to do each case because I was doing my normal processes and then looking at it. I got to a level where it was about equipoise, and then I noticed it was missing lung nodules. I thought, “This is what it is supposed to be doing”. My colleague had noticed this as well. We investigated and it turned out that a software upgrade had changed the sensitivity without informing any of us at all. With the personal trust issues in introducing this software, we need to be confident that it is doing its job as well as it says it will.
Lord Winston: What about, for example, the comparison of images sequentially? Will that be helpful with MRI, for example, where you are not doing too-invasive methods of examination? Will that be of use?
Dr Stephen Harden: Yes. I picked up from the first part of your question the comparison of previous examinations. That is already happening. It is hugely useful measuring a lung nodule and seeing what it was like before. It is done automatically, identifying areas of change, but that is one significant benefit present already.
Lord Winston: Presumably that could be useful for how pathologists decide whether to change the treatment, for example, and what other genes they might want to look at.
Dr Bernie Croal: You can see the application of that to the prostate, any lumps or any lymph nodes—you can look at it sequentially and see the changes, and have that automation built in that will deliver services faster.
Lord Winston: How far have we got?
Dr Bernie Croal: In pathology, we are pretty limited because of the IT restrictions. If we are thinking about imagery and AI for histopathology, which is the main area of expansion, at the moment only about 23% of pathology tests in histopathology are images. There is a long way to go to digitise pathology services because, without the digital pathology base, you cannot add AI into it. Adding AI into it has significant barriers, mainly around cost. Several pilots have been done. They always get to the end of the pilot and then realise that there is quite a significant chunk of money to pay every month to keep that particular AI service going. It is moving forward, albeit slowly.
Q183 The Chair: Dr Croal, your written submission to our inquiry referred to geographic and demographic inequalities and how work needs to be done to ensure that there are not these inequalities in accessing the benefits of technologies like image analysis and AI. Could you comment a bit further on that?
Dr Bernie Croal: Yes. You may not know that in England, for example, pathology is delivered through 27 networks made up of around four or sometimes five individual trusts whose services have come together. It is up to those individual networks or even individual departments to decide whether they move forward with digital pathology and then on to AI introduction within that service.
As you can imagine, it is very patchy. I said that 23% is the rough figure we have overall, but that is spread out across the country unevenly. Some departments are 100% digital pathology currently, and some departments essentially have none and are just switching it on. There is inequity and that will continue. The challenge is how we get digital pathology implemented across the board as rapidly as we can so that we can then start to build on that.
The Chair: When you say work needs to be done, in what form?
Dr Bernie Croal: Most of it is money and investment, not just in the capital to buy the scanners and equipment but in the storage costs, for example, which sometimes can outweigh the cost of the equipment itself. There are also costs around the relative workforce that we need to understand and maintain those systems.
It is about training as well because, currently, we do not train pathologists in any aspect of digital pathology or AI. We set the curricula. We set the exams. They do not feature in it. We are training pathologists at the moment for service 10 years ago, whereas we need to be training them for what is coming in the future.
Lord Willis of Knaresborough: Why do you not change?
Dr Bernie Croal: We cannot mandate curricula and examinations based on any of that if only a tiny proportion of trainees are experiencing it in their practice. It is a Catch-22. The faster we roll it out and make it all available, the quicker trainees can catch up.
Dr Stephen Harden: I have quite a few things to pick up there. On the training point, in our current curriculum, which is from about four or five years ago, we could foresee the impact of AI in radiology and so we do have it as an emerging technique, but we will be tasked by the GMC over the next two or three years to update the curriculum in light of the Government’s 10-year plan. Also, there will be a bigger role for AI in that.
We have established some initial courses because we realise it is incumbent on us to provide the training, but it will be a curricular requirement. We need to take the lead in providing that training. Therefore, because in many ways radiology is one of the leading technological specialities in medicine, we are doing a lot of work that our colleagues at other colleges are looking at. We are interested to see how theirs will develop in future, too. We are looking at some form of training and educational resource that would span all colleges but potentially all healthcare as well. You can see it is quite a big undertaking, but this is very much what we need for now.
This is moving so fast. I do not need to remind you all of that, but the curricula will struggle to keep up because of the pace of change. We are clear that now this is an absolute requirement.
On the geographic divide, we find a workforce divide too, in that some centres have better workforce provision than others. Particularly in introducing any new form of technology, if your workforce is particularly short, they will not have the time to introduce these new techniques. Addressing the workforce is clearly a crucial part of this.
The Chair: You are both practising at pretty much the opposite ends of the country, in Southampton and in Aberdeen. Do you see that as difficult for your areas in terms of workforce?
Dr Bernie Croal: Yes, different parts of the country have different challenges. Scotland has a difficulty in recruiting generally, and for more outlying places in the north of the country it is particularly problematic. We would have particular issues. I do not know about Southampton, but certainly more remote and rural parts of England as well, the north-west in particular and the south-west, have particular difficulties.
Dr Stephen Harden: It is exactly the same in radiology. North and west Wales, the north of England and the north of Scotland have the greatest challenge. To a large extent, because we between us reflect the whole length of the UK, we have a clear idea of where the pockets of need are.
Q184 Lord Patel: My main question was going to be about workforce, but I will come to that in a minute. Do you both agree that AI genomics in the wider sense, not just testing for cancer, will lead to more personalised medicine, and that that is the future of medicine? Do you agree with that?
Dr Stephen Harden: Yes.
Dr Bernie Croal: Yes, absolutely.
Lord Patel: You commented on IT structures being inadequate, with workforce issues to be resolved. Have either of your colleges, following the 10-year health plan and the 10-year cancer plan, produced a critique of both of those reports, identifying the issues that will need to be addressed, including workforce, to make this vision of AI genomics and personalised medicine a reality?
Dr Bernie Croal: Yes, we have made responses to both of those reports, identifying what we see as the major barriers to their targets and objectives.
Lord Patel: Now we come to workforce planning. I do not want the numbers of radiologists and pathologists. If the obstetricians were there, they would tell us the same thing. Nuance it a bit more. You referred partly to the training, but what kinds of radiologists, pathologists and medical oncologists, which you have not referred to, do we need to deliver on this vision?
Dr Stephen Harden: You will not like this answer, but we need everyone. We are finding that AI will be helpful in terms of efficiency, process and effectiveness but, at each stage, even once we have AI integrated across all patient pathways, we will still need more workforce to be able to supervise it, to make sure it is working efficiently and appropriately and to make sure it is introduced accurately. I cannot foresee a time when we will be sitting here saying that we have all the workforce we need, even in the short to medium term, because it needs medical leadership. It needs to be medically supervised.
Ultimately, of course, we as clinical practitioners are the patient advocates in this situation. We know that patients are interested but cautious about the impact of AI in our specialties. They still have a lot of trust in us and our medical professional ability. Off the back of that, we have to hold true to the fact that we need to be the ones supervising and leading it. Unfortunately, I do not see in radiology—I will let Bernie comment in a minute—in the short term any special interest area that has its full capacity and does not need more help.
Dr Bernie Croal: For us, it is clear that demand for pathology services will undoubtedly increase in future years, largely because of the demographic changes that we will see in the population—the increase in cancer and so on. That will all drive it, but also the availability of new innovations and testing, especially in the genomic area. We realise that you cannot just keep increasing the workforce. We need to look at ways in which we can reform services, using people differently, bringing in automation and bringing in digital and AI to change the way we work.
We absolutely do not believe that AI will replace pathologists, but we believe that pathologists who use AI will replace pathologists who do not. That is quite a difference. Overall, we need to ensure that we have the right people in the right places to understand, deliver and implement the new ways of working that we foresee will come in the future.
Lord Patel: In England, we have been told that we now have the seven genomic hubs that will do all the testing of tumours, DNA, et cetera. Therefore, one can assume that the specialism of doing that will reside in those seven hubs. They will provide all the services related to DNA analysis in the whole of England.
Dr Bernie Croal: Yes. To some extent, I guess it remains controversial because a lot of the local-based testing has been removed and focused on the genomic hubs, which has led to significant increases in turnaround time, which can be vitally important for cancer services. At the moment, it is struggling to keep up with that. Genomics is one area that is expanding way faster than we can afford to keep up with. We need some rationalisation to ensure that the testing that is selected is the most appropriate and the most useful because, at the end of the day, we will not be able to do everything.
Lord Patel: Would that not disadvantage patients?
Dr Bernie Croal: When you look at the new technologies and drugs that are coming on board that will rely heavily on genomic data, together with our information from radiology, histopathology and so on, it will be vitally important that we can deliver those services in a timeframe and window that enables those drugs in a personalised medicine area to be selected and used and patients to benefit from it. We need to ensure all that works.
Lord Patel: We know that the Government are about to, we hope, produce a workforce plan. Have you had both had input into it? You have. Do you expect that it will be fulfilled?
Dr Stephen Harden: We have hope. Again, the hope versus hype issue remains. We expect that the workforce plan will have comments about technology and AI, quite appropriately and along the lines we have discussed already this morning, but I very much hope that the need to increase the workforce will still be recognised because, for the reasons we have already explained, the workforce is under strain and we will be the ones introducing it and quality assuring it. We cannot do it with the group that we have at the moment.
Dr Bernie Croal: I see it as a little bit more challenging, to be honest. The rhetoric coming out of Jim Mackey, Penny Dash and so on is that the NHS is overfunded and overstaffed and, therefore, the workforce plan will not look at significant expansions. We can work with that but, again, for us to benefit from a lot of the new innovations, technologies and testing, we need to have significant investment upfront to get us to that place where we can start to benefit from the real efficiencies and productivity gains that that will bring. Our concern is that we will be left with the promise of what AI and genomics can bring but have no means of progressing it to a point where we can make the best use of it.
Lord Patel: What did you think about the Penny Dash review, particularly focusing on workforce? You can say that you did not like it.
Dr Stephen Harden: It is appreciating that it is a difficult time. We completely understand the current financial state of the country and that, therefore, we are also seen as asking for more money and greater investment. However, we have a clear view of what needs to happen. We are grateful to have the opportunity to explain to you that we can see a future where it all works better, but we are at a stage now where we just need the investment to give us that platform to do it.
Q185 The Chair: I want to ask one more question about your college’s submission, Dr Croal, where you highlighted the problem of timely implementation of genomic tests and that the target of 14 days was not being met, only by a very small percentage. Is that simply a workforce issue or is there more to it?
Dr Bernie Croal: It is a combination of factors but, inevitably, mostly workforce. When we look at the pathology and the histopathology part, the shortage of pathologists impacts the turnaround time for those particular samples. The next stage is the genomics. Again, the genomic hubs are relatively overwhelmed with testing. Their turnaround time for some tests can be nine months or 12 months. It is as long as that. Clearly, that information will not help someone who is sitting with cancer or a disease that could be treated specifically. We need to ensure that we can get those turnaround times down.
The national cancer plan aimed for 10 days for these types of specimens for histopathology. Ideally, as Peter Johnson says, we would want that for genomics as well. We are a long way away from that. Histopathology at the moment across the 27 networks is sitting at 54% achieving the 10-day turnaround time when the target is 98%. To improve that, yes, we need those workforce gaps to be filled but we also need to take advantage of the other things we have already mentioned—automation, digital, the potential AI. That will all speed up the process.
The Chair: Does it follow that you have to do pathology tests first and then, after they are done, genomic tests? Could they not be parallel exercises rather than sequential?
Dr Bernie Croal: It can be in some cases. Some blood-based genomic testing can be done, liquid biopsy and so on, which you might have heard of, but that is not the case for every test. Generally, the histopathology examination and the staining of antibodies that is done at that stage will define what genomic tests you then request. Most cancer pathways have very specific pathways of investigation.
Q186 Lord Willis of Knaresborough: I must confess that most of the sessions we have had on this piece of work have been incredibly exciting about the potential of this. This morning, we are back to reality. I am particularly interested in what happens in the royal colleges and the universities and what happens at ground level. At ground level, it seems to be totally different from what is happening elsewhere. In 2024, you set out recommendations for the improvement of digital and IT infrastructure in the NHS to enable the deployment of AI. Could you tell us briefly what principles you had within that production, why they were important and, in particular, whether you have seen any progress at all on them? You are indicating this morning a fundamental difference between what we want to see and what is actually happening. Be ruthlessly honest.
Dr Bernie Croal: The implementation and rollout of digital pathology, which is the prerequisite for AI to be used in histopathology, has been relatively slow, although a lot faster than in other countries to be honest. As I say, we are currently at probably 24% of specimens being reported digitally now, but it seems to have slowed down. That is largely because implementation of that technology is not cheap and most networks and NHS trusts cannot necessarily afford to do it themselves.
Where we have seen progress has largely been from money that has been utilised from NHS England funds and central money delivered through the NHSE pathology programme. The real benefits have come there, but that money is not a bottomless pot and that is where the barriers have been. When we look at other parts of the UK, like Scotland where I work, we are way behind on digital rollout. There is always that gap, which at the moment is larger due to lack of money to fund the capital, to fund the storage costs, which are huge, and then to fund the ongoing revenue budget associated with that.
The stuff we see coming out of universities and the research and industry in particular linked to all the new pharma drugs is incredibly exciting. We work closely with them to raise the profile, but it is incredibly difficult because implementation requires the digital, requires the workforce and requires the money. If that is not present, the AI product simply cannot be used and paid for. That remains the biggest barrier.
Dr Stephen Harden: I would emphasise that, among the gloom, there is hope. There is hope in the form of tech, without question. From a radiology perspective, we are very much grasping and leading that. We have a two-day global conference on AI medicine at the end of the month at the QEII building, pulling together expertise from across the world on the current state of play and looking forward to seeing how this will benefit it.
To come to your question about the 2024 report, of course there were big long lists of recommendations, including for us, for example about working across regulatory bodies. We talked about curricula and training; we are making progress there. There were also asks for NHS England and the Government, on which there has been some progress, perhaps not quite so much as we would hope.
We particularly picked out regulatory processes, especially around data protection and impact assessment, where something can be used and trialled in one place but the next place has to start all over again. A lot of that comes from NHSE and the Government, with the ability to say that this will be a national process. If an individual hospital follows these rules and shows that this is effective in practice, we can apply it to other parts of the country as well. There is progress, but not the sort of progress we are hoping for.
Q187 Lord Booth: One major concern that has been repeatedly raised in this inquiry is the lack of consistency between NHS trusts and the issue that may be called “pilotitis”. It is relatively easy to get a new innovation at a local level, but then trying to get it to national trusts and to national commissioning is extremely difficult. In a lot of cases, it seems to be impossible. Do you recognise this issue? Would you recommend anything to improve this across the National Health Service?
Dr Stephen Harden: Yes, absolutely. This is one of the big frustrations, alluding to some of the governance issues I mentioned in the previous response. It needs to be facilitated with a regulatory process that is enabling rather than too constrictive. From the point of view of the pilots, it would work much better if the scaling up was planned at the outset. Rather than just an experiment, let us see how it works. Even before the pilot starts, let us have a plan then to make this regionwide or nationwide. There are also problems with funding. Apologies for raising the funding issue again, but often the pilots are very well resourced and the rollout at scale is not.
We also find that individual centres may take on a new bit of software or a new bit of tech as part of the pilot; they get it for free, get used to it, get dependent on it and then, once the trial and the pilot period come to an end, it is no longer free and it has a cost. Financial issues are related to that as well.
I have one final more political point around commissioning. We are rather anxious, with changes to commissioning based in ICBs, about the workforce cuts there. We have a concern that a lot of commissioning expertise will be lost as a result of that process. Thinking about piloting and developing at scale, having the best people who understand how commissioning works would be a critical part of that.
Dr Bernie Croal: Pathology is well behind the curve in terms of implementation of digital and AI but, to be fair, pilots are how we get a foot on the first rung. The pilots are generally heavily subsidised, as Stephen has said, by industry, by the AI companies and so on, and they also need enthusiasts to take them on locally. However, the harsh reality is that once those pilots end, the company will expect the full amount of money. It is a bit like Netflix. They bring you in the door, they show you what is on offer, and then at the end of it you pay the full whack. That is where it is difficult, because writing a business case for AI in pathology is hard when your finance manager is expecting savings. We are trying to offset the cost that you will be facing if you do not do anything, which is a much more difficult argument to make.
As Stephen has said, we need a better approach to ensure that pilots are done on a wider scale and are funded, perhaps centrally, to ensure that that rollover into normal practice and acceptance, if the pilot goes well, can be made because that would make a big difference.
The Chair: You are both referring to the problem of a pilot having more or less free access to an innovation, but the question of scaling is then a different story. Are you referring mainly to AI companies or more generally?
Dr Stephen Harden: In this context AI companies, but it is an issue generally as well. We have learned this over the years from other innovations but, particularly in the context of discussing how we introduce AI at scale, it is much the same.
Q188 Baroness Willis of Summertown: This question is around trust in AI. I have heard before what you said about it taking you a while to get used to or trust the role of AI and how it could improve the timing. My own experience of AI as a scientist is that it sometimes gives you random results. How can we build a system to create trust in AI going forward for both of your specialities?
Dr Bernie Croal: Pathologists in general are not trusting of other systems, especially if they are not getting to look and check. That is part of the problem that I see in some of the pilots that have occurred. For example, one of my colleagues who works in Cambridge and has been doing the pilot in prostate AI always goes back and checks every result, and he finds errors. But it is a case of accepting the errors. You can look at the numbers and you could have a 2% error rate in cancer pick-up. If you are doing 100,000 cases, you are missing 2,000 cancers. That sounds horrific, but the error rate among pathologists is around 2% anyway. It is about what is acceptable and explaining that what we do is not an exact science but a probability. That is part of the game.
With any technology, the more it is used, the more you get used to it and the more you accept the limitations, as long as you have processes built into quality control and checking it. That is how we operate across the whole of pathology services.
Baroness Willis of Summertown: I will come on to oncology in a moment, but I was taken with your comments about training. One of the issues is that we are not good at training people to consider risk. Instead, we are risk averse throughout all aspects of society, but particularly in medicine. Where within the training of pathologists should we get people to understand this risk aversion or managing risk and think about how to look at these new technologies and whether to embrace them or not? From your experience, where should that fit in the system?
Dr Bernie Croal: In pathology we do quite a bit of that already in our curriculum and in our teaching because pathologists across the board—I am a chemical pathologist—from day one in the job can be managerially responsible for the whole department. We have a lot of that management skill built into our training to ensure that we understand all these things around risk and responsibility. We do a fair bit of that.
The introduction of AI ups the game, though, because a lot more things in there are outside your control. You need to build some of the deviations and technical errors that you need to know about into your overall risk profile for any part of that service that is ongoing. That is why the regulation is important. We need to ensure that what we are buying is proven to be safe with a degree of quality. At the moment, regulatory systems are quite immature in being able to achieve that.
Baroness Willis of Summertown: That is my question. Who does the regulation? Where does that sit within the process?
Dr Bernie Croal: There are processes for IVDRs that include AI which overlap with NICE and the MHRA, but they only scratch the surface. We have to bear that in mind.
Baroness Willis of Summertown: Should a separate body be responsible for AI regulation?
Dr Bernie Croal: Yes. Stephen will have more advanced comments on that.
Dr Stephen Harden: Again, I have quite a few things to pick up. On trust and regulation, if I can take that first, it is so important that we get this right. You mentioned the phrase “AI hallucination” that we use at the minute, which is when it comes up with random things, which in the context of patient care is utterly unacceptable.
I would break down the regulatory process into three. The first is that we need on the ground to be clear on the evidence. If I am using an AI tool to look at a chest x-ray, I need to know that it is good at what it does, but also that it is applicable and has been trained in a population similar to what I am looking at. A summary of that evidence needs to be instantly available with a click to check that it is applicable to this patient.
The next aspect is quality control. We have a process in medicine for flagging adverse reactions to drugs. We need to get the same thing for AI and tech. Similarly, in my office or at my workstation, if I see an error, it needs to be literally two clicks to send it off to the MHRA, the central body, to flag this as a problem. I do not have time to fill in a big, complicated form. It needs to be a simple version of QA and identifying error.
The third aspect, which we have not yet touched on, is blame. Whose responsibility is it? We are quite used in hospital practice to innovations and new technologies being sold to the hospital, and the hospital takes the blame. Of course, it is because of a whole raft of issues—it depends on your IT system and so on. We will have to address this, particularly as AI becomes more involved in medical practice and particularly as it starts to have some element of decision-making. We would support a supervised AI process for triaging, et cetera, but we have to be clear whose responsibility that is. That comes back to your question about trust. Individual practitioners on the ground will have much more trust in the process knowing that the liability was clarified and that, from using a particular tool, it would not all come back on them.
Baroness Willis of Summertown: When you talked earlier about the workforce, you were saying we need more people, not less, because of AI. That is also an important point. Would you see this as a separate body that sits outside of your own areas? Maybe you have already answered that.
Dr Stephen Harden: We are working as part of the AI commission. We are looking at a more modern and appropriate regulatory process for AI tools because we need to move with the times. It needs to be fast and enabling rather than too bogged down waiting for the finer details. Our preference would be the MHRA or an equivalent body, but one body needs to be responsible for overall regulation so that it gives confidence to the profession and to patients.
Baroness Willis of Summertown: Of course, there is AI for clinical work, but then there is AI to improve the flow of process and administration. How does that fit into your vision for what we should be doing?
Dr Bernie Croal: That is the bit I get most excited about, to be honest, because within laboratories we have relatively inefficient processes. Yes, we have brought in automation in the blood sciences. We do not quite have it in histopathology, but we could improve it. You can imagine a system whereby AI is able to best control and advise everything from when the blood sample is taken—even whether it is taken—and then follows that flow through transport and the lab process all the way out to when the result is produced and interpreted. You can use AI to help every single part of that journey.
That will bring efficiencies, productivity gains and improvements in turnaround times. Most importantly, it will ensure that we do the right tests on the right patient at the right time. That is the real goal, because part of the equation for us is to keep up with the huge demand that we will face in future and to ensure that we are not being wasteful and are able to reduce unnecessary testing so that we can improve the necessary testing that needs to be done.
Dr Stephen Harden: I share that excitement and optimism around the role of AI in taking an entire patient pathway from referral through to completing treatment. In a sense, perhaps one of the problems at the minute and one of the reasons why we are a bit more cautious is that we have AI tools that will fix a specific problem rather than what we need, which is a process to identify where the AI will help, so that we can go to industry and say, “Do not necessarily give me what you could make or what you think might be helpful, because this would really make the difference on our pathway vision. Can you go away and make that?”
The Chair: You have covered this whole question of trust and quality control. Can I ask about image analysis? How do you validate that the AI is correctly identifying an image? What is the process for that validation?
Dr Stephen Harden: It is a good question. At the moment it is based on comparison with the human eye. This will make it more difficult to demonstrate, when the human eye cannot see it, whether the tech must be wrong. We know that is not the case—the tech will get better and will be able to resolve better than the human eye can. That is why it is important that we link with outcomes rather than just test results. This will take time. This is why we are still at the early stage of this process. To demonstrate that finding A will lead to outcome B in 10 years’ time is where the real benefit will come.
The Chair: Are there examples, albeit a small percentage, where the AI is much worse than the experienced human eye in imaging?
Dr Stephen Harden: I will give you one example hot off the press. A publication from Sheffield in one of the big international radiology journals looked at all the major chest x-ray vendors and analysed 11,000 x-rays. It did pretty well in identifying normal ones, but it identified over 2,000 of those 11,000 that it recommended additional tests on, which on the ground we would not have done. The problem is not so much false negatives. The problem is false positives in a constrained system where we do not have the time to do, as Bernie says, additional unnecessary tests.
Q189 Lord Stern of Brentford: Thank you for your thoughtful contributions. You described what you would look for from AI in terms of following through the whole patient pathway. How do you persuade the AI researchers sitting in companies or universities to work on exactly that problem?
Dr Bernie Croal: It is difficult. We always have a challenging relationship with industry in that it tends to develop products that are in its remit and tend to give us and offer us what it has rather than what we need. It is improving. Our college is working more closely with industry. We have formed a corporate partnership solution that brings together many of the companies from across the whole spectrum of pathology on a regular basis. We now have 12 corporate partners that we work closely with, building education, webinars and research. Industry has an understanding now that it has to work with the profession and with the NHS to face what will be the most important challenges that the NHS faces.
Lord Stern of Brentford: Is there real progress on that?
Dr Bernie Croal: Yes.
Lord Ranger of Northwood: Earlier the Chair was asking about whether AI is better. Is the fundamental point here that, when we are looking at the systems you are using, they are gradually getting better and continuing to evolve? How are you assessing the point at which that improvement is happening? How do you build that into what you are currently doing or what you may do in the future?
Dr Stephen Harden: There is quite a lot of work on that. It is interesting in terms of trying to simplify the benefits. The benefit to the patient might be a more accurate diagnosis—and some evidence shows some improvement there—as opposed to the benefit to the system, which is increasing productivity. We are not seeing the benefits from that yet.
You are absolutely right—and this is why there is hope amidst our sense of gloom—that this is still early days. The problem with the hype is that of course this should be ready and available now. We are working on it. We see it very much as a future. We have to accept that it is early days and it will come in time.
Dr Bernie Croal: To add to that, the big driver inevitably is the realisation that demand on diagnostic services will significantly increase because of the patient demographics we are likely to face, but also because more tests are coming in all the time that are important, especially in the genomic area. We realise that we cannot possibly keep up with that demand using the capacity that we have now. We need to embrace all these technologies and innovations, and AI will be a big part of that. We have to try to make it work.
Dr Stephen Harden: I agree with that. Another thought is that we collaborate among colleges and we collaborate internationally as well. That will be part of our conference in a couple of weeks’ time. We have colleagues coming from across the world with their experience. It is very much an international effort. For example, regarding the point at which AI will be proven to be good enough to be the second reader for mammographic screening, there will probably be an international move in that direction rather than us necessarily waiting to be sure that we have been first or have the evidence to show that. That international collaboration is important.
Q190 Lord Drayson: How has the NHS been affected by the sudden withdrawal last week by the US Government from the leading AI models of Anthropic, Fable 5 and Mythos? In particular, how is it affecting current thinking within the NHS and the colleges about the use of technology that is outside the control of the UK?
Dr Bernie Croal: I am not fully sighted on that particular example but, in general, when you look at the source of new innovations and new products, they are increasingly from specific companies or in the IT sector where they are really the only offers on the table. It is putting all your eggs in one basket. It is a worry that global events impacting those particular companies will have a significant effect. We are already seeing it in some supply chains when the sudden withdrawal of a product or products that are not available leads to big problems further down the chain.
The same will be true for IT and the AI sector. Most of the AI companies, as you probably know, are private companies funded from private equity. They are looking for the pay cheque, basically. If it does not come, they will quite easily pull the service and you have no comeback on that. That is a concern, but it is the way of the world and we cannot do much about it. We cannot generate AI from within our own NHS or even within our own universities to some extent. It needs industry and private money to come in and ensure that it works and gets through. As you say, inevitably, that puts us at risk.
Dr Stephen Harden: I do not have much to add to that. Again, I am not fully sighted across that item that you mentioned, but we are dependent on that international effort. We similarly have problems with supply chains and AI is no different in that respect. We do design and build components here, but it is very much an international effort.
Q191 Lord Duncan of Springbank: In some ways, you have already given us insights into how we as a Committee can focus. However, if you had to try to summarise it down into two or three recommendations that we could place into the report for the Government to act on, to take the report away from being an academic study to being a driver of change, would you emphasise anything to us where you see benefit in our report focusing to move the Government?
Dr Bernie Croal: It is clear that the future needs to embrace AI to bring together all the information from other innovations, especially in genomics, to allow us to make the best use of all that technology. To get there, however, we need to ensure that pathology services and radiology services have the right people working in them, enough people working in them with those skills and the right investment to ensure that the tech and the equipment is there to achieve that. We need that pump primed, because it will not happen otherwise. Our main push would be that we need to sort out investment in that technical area—get the standardisation and the interoperability right—and then build AI on top of it. At the moment, in pathology, we are a long way from that.
Dr Stephen Harden: Similarly to Bernie, I would emphasise that we do see hope. We do see a future. Some of these tools are being used currently in the NHS and it will have a much bigger role in future.
My three points would be, first, on the IT infrastructure. There is something about getting the basics right before we get the shiny, new, expensive tool on top so that it works properly. Similarly to Bernie, I would say we should not forget the workforce, which will be critical in implementation and QA. Workforce expansion remains critical. The final point is regulation. We should have a regulatory process that is not obstructive or blocking but enabling, so that we can assure patient safety. My colleagues—and similarly for pathology as well—do not see this as something where we will have an awful lot of blame when things go wrong. The regulatory process needs to work.
Q192 Lord Winston: We have been talking a bit about trust. Could you address something we meant to address, which was the liability issues that you might see? I could see, for example, that it may be different in pathology because you often have to wait for a post-mortem and even that will not tell you what was wrong. But with radiology, there are images. I have been in one course where different experts widely differed on the clinical applications. It seems to me that, with AI, it will be even more difficult. Is that a reason why people who are using it might be a bit concerned about AI?
Dr Stephen Harden: Yes. We have touched on that and you are absolutely right to emphasise it. The vast majority of cases will be fine, but in a significant minority of cases there will be disagreement or differing views. As you say, it has always been that way and will continue to be. That is not a reason not to explore AI for these more complex, difficult cases, but it emphasises the fact that now we have multidisciplinary team meetings to deal specifically with these more challenging cases, and we will still need those in future. Potentially the AI view will contribute to that, and the MDT then has the opportunity to say, “No, we will say from the clinical view that this is the way forward”.
Lord Winston: In pathology, too?
Dr Bernie Croal: In pathology, as you have alluded to, we are at a much lower starting point. Certainly among a lot of the pilots that are taking place, which are showing promising results, a little bit of fear is creeping in about having systems that simply let the AI diagnoses go straight out without anyone checking them. Few pathologists are willing to do that currently. That, hopefully, will change in the future as they get more used to it. Because we are on that part of the learning curve, it is inevitable that they will not want to take the risk personally. At the moment, they are personally liable if a result is signed out by a department. The clinical director’s name goes on that piece of paper.
The Chair: Thank you both very much. It has been excellent hearing from your two colleges, from the pathologists and the radiologists. We are very grateful. Thank you so much for coming.
