Examination of witnesses
Lorraine Turner, Katherine Monnery, Sara Walton and Steve Brunige.
Q156 The Chairman: Good afternoon, ladies and gentlemen. Thank you very much for coming this afternoon to help us with our inquiry. We are on livestream broadcast, so any conversation gets picked up and drowns out whoever is speaking. I would ask you, as with Committee members, to please refrain from conversation between yourselves.
Before we start, it would be very helpful to get you on record. From my left, would you mind introducing yourselves and, if you wish to say who you represent, feel free so to do? If any of you wants to make a short statement to start with, you are welcome to do so. Otherwise we will press on with the questions.
Lorraine Turner: I am the technical and business development director at the United Kingdom Accreditation Service—UKAS. My responsibilities include technical governance, development of accreditation of new areas, government engagement and stakeholder management. I am also a member of the Forensic Science Regulator’s Forensic Science Advisory Council. I can give a short introduction to UKAS. Would it be helpful to do that now?
The Chairman: If you have already sent it to us in the written evidence, we do not need it.
Lorraine Turner: I would emphasise that UKAS accreditation attests to the competence of organisations and their impartiality. We carry out assessments against international standards. I know that those standards have been the topic of discussion in the Committee, and we would be happy to talk about those further. We have currently issued over 60 accreditation certificates in the area of forensic science. That gives you a bit of background.
Katherine Monnery: I am the forensic accreditation specialist at UKAS. I have worked at UKAS for over 22 years and been involved in the assessment of forensic science organisations for about 20 of those years. I am a member of the Forensic Science Regulator’s Quality Standards Specialist Group and Fingerprint Quality Standards Specialist Group, and a member of the National Police Chiefs’ Council’s performance and standards group and national quality managers group. I was also involved in the international committee that wrote ILAC G19.
Lord Griffiths of Fforestfach: What is that?
Katherine Monnery: It is the document that we use in the forensic context alongside 17025 and 17020.
Sara Walton: I work in standards development for the BSI, which is the UK’s national standards body, and that entails stakeholder engagement and general development activities in the standards domain.
Steve Brunige: I am head of industry and government engagement for the British Standards Institution. My role is to raise awareness of the voluntary standards and the benefits that they bring to the UK economy. I work extensively across government to raise awareness of how standards can be used to assist government in achieving policy objectives, and as an alternative to regulation.
Q157 The Chairman: Thank you. The first question relates to the role of your organisations in standards setting and regulation in forensic science. How do you set the standards, including for ISO 17025? How do you develop these standards, and what are the future challenges in developing standards and accreditation in view of the new forensic science that is developing?
Steve Brunige: It probably makes more sense to start with how we set standards. The British Standards Institution—BSI—is the national standards body for the UK. We operate under a royal charter and in accordance with an MoU with government, specifically with the Department for Business, Energy and Industrial Strategy. I would like to explain our process.
The Chairman: Did I hear you right: that you deal with the Department for Business, Energy and Industrial Strategy, not the Home Office?
Steve Brunige: Our sponsoring department in government is BEIS. We work across government, but our relationship is particularly through BEIS. BEIS signed the MoU with us on behalf of government.
May I explain the general process for developing standards, which is the same process that is used for forensics standards? My colleague Sara can give a bit more detail about forensics standards. Essentially, we facilitate the development of voluntary standards through committees made up of experts and a broad range of stakeholder groups. These include industry associations, government departments, regulators, academics, consumers and public interest groups. BSI simply facilitates the process. We bring the experts together and provide the process and necessary support to enable the development of the standards.
These standards are essentially documents of good practice that are based on the principles of expert involvement, full stakeholder consultation and consensus building. Once finalised, we publish and communicate the standards, which are then reviewed at least every five years and either maintained, withdrawn or updated.
As the UK’s national standards body, BSI also represents UK interests in international standards making. We are the UK member of the international standards body, ISO, the International Organization for Standardization, and the IEC, the electrotechnical equivalent, and the European standards organisations CEN and CENELEC. We facilitate the involvement of UK experts into these groups to ensure that UK interests are fully represented.
The Chairman: Does anybody else want to make any additional comments?
Sara Walton: Only that, that is how the standards are developed. ISO 17025 was developed in that way. I know you have extra questions about the other forensics standards. Those are developed in the same way. We gather together the relevant and expert stakeholders and ensure that everybody has an opportunity to input into the development of the standards.
Lord Hunt of Chesterton: Thank you for your reply. Steve Brunige, you said that standards are reviewed about every five years. I have been involved in wind engineering and the use of British standards, and there seems very little change every five years, despite research. How rigorous is the process? The science is moving much faster in general than the standards are. Do you think this is a general feature, and how do you ensure that we really are pushing forward?
Steve Brunige: We endeavour to keep up to date. The standards are required to be updated at least every five years, but if there are quicker developments, committees will meet, and they meet regularly in any case.
Lord Hunt of Chesterton: Do they think about having a change in standards because of some accident or scandal?
Sara Walton: Not exactly, it could just be because the experts have raised issues or, as you say, the science has moved on and something needs to change. That would be brought to the attention of committees, and people would consider whether something needs to change within a standard.
Lord Hunt of Chesterton: I am afraid to say that I wonder about that.
The Chairman: Do either of the other two witnesses want to come back to my original question?
Lorraine Turner: The role of UKAS mirrors in many ways the role of BSI. Our function is as the national accreditation body, so we, too, operate under a memorandum of understanding with government through BEIS. It is that memorandum of understanding that outlines our role in the standards that we use to assess and accredit organisations. That is the reference point for that. We use those standards, and in the case of forensic science the standards that we use are ISO 17025 for the laboratory-testing activities, and ISO 17020 for generically called for inspection bodies, which is used to assess crime scene examinations; the front end of the forensic process. We take those standards and use technical experts to interpret and apply those criteria to evaluate and assess the competence of organisations.
You asked about the challenges that we face. The challenges fall into two camps. We have operational and technical challenges. The technical challenges are in the interpretation and application of the standards. We utilise technical expertise to do that, both at the coalface when we are carrying out our assessments, by using peers, and in the governance and management of the accreditation programme through our advisory committee structure.
The operational challenges have been quite considerable in recent years because of the requirement for accreditation. The deadlines are set and we are looking at the flow of work, the pipeline of work, as private providers and police forces are coming forward for accreditation. They are challenges that we manage well, and we work with stakeholders to get the engagement and support to deliver the accreditation.
Q158 Baroness Manningham-Buller: That leads on to the experience of people seeking accreditation. You said in your opening remarks that you are there to decide on competence and impartiality. How do people seeking accreditation from you give you that assurance? What is the process?
Lorraine Turner: The process is one where they need to demonstrate that competence. That is through the format of an assessment, which covers looking at their documentation, their policies and processes and the principles that they have in place in their organisations. Our assessment includes the witnessing of activities, so we look at the tests that they perform and the samples that they take.
Baroness Manningham-Buller: You go into the labs and supervise or watch.
Lorraine Turner: We go into laboratories, and we go on-site where there are site activities. We witness the evaluation activities that they are conducting. We look at the records and evidence to show that they have followed their systems and processes. We look at the management controls that they have in place to ensure sustainability of the application of those principles, policies and processes in future.
Baroness Manningham-Buller: You have been largely talking about the competence component. What about the impartiality component? How do you judge that?
Katherine Monnery: Particularly with the forensic aspects, we look at the impartiality of the hypotheses that people put forward regarding what they looked at, what they were given to examine, and how they looked at what they were given to examine, and look perhaps for prosecution bias or an alternative hypothesis to what had been presented to them. We look at their way of weighing up the evidence rather than take a particular view, depending on whether someone had been the customer in that regard.
Baroness Manningham-Buller: Do you choose which examples to look at rather than wait for what they give you?
Katherine Monnery: Absolutely. We would perhaps ask for some to be prepared in advance if we wanted to see records of particular activities, but we would also take the opportunity to see what was going on during the laboratory work or at the crime scene, whatever it may be. We would look to see general work going through the process. We also review a lot of case records, so we go back over cases that had been completed, which we would select at random.
Baroness Manningham-Buller: We have had some evidence that this is quite an overhead for small providers of forensic services. Do you have any way of mitigating the costs for them so that they are not driven out of business?
Katherine Monnery: We have been doing some work on that.
Lorraine Turner: There are two elements to the costs for an organisation going through the accreditation process. The first is the cost of implementing the improvements to address any gaps that it has in its system. Those costs are not really within our control, because they are the measures that it puts in place before it comes forward for accreditation, or certainly before it is granted accreditation.
In terms of the fees they pay to us to cover the costs of conducting the assessment, we are very mindful of the impact that has on small businesses. We have been working with the Forensic Science Regulator and the Chartered Society of Forensic Sciences to look at ways in which we can reduce the amount of time it takes to conduct the assessment and where we can build in efficiencies.
One example of that is the Chartered Society of Forensic Sciences putting a set of standard documentation in place which small businesses can adopt and which we would only need to assess once. That would bring about a reduction in the amount of time it takes to conduct the assessment.
The Chairman: You mentioned inspections. Are these blind inspections, or do you do unannounced inspections?
Katherine Monnery: We can do both. We have a programme of visits. Generally, we would visit annually each organisation once it is accredited, but we can decide to make that more frequent, we can do it in response to a quality incident or a complaint, or we can undertake unannounced visits, if we feel that is necessary.
The Chairman: How many of those unannounced inspections do you do?
Katherine Monnery: I did one not that long ago in response to a quality incident. We do not do them very frequently, it is not routine, but we can do them in response to issues. We did some work about three or four years ago where we did routine unannounced visits to give ourselves confidence in what was being undertaken in the forensic science area.
We found that went very well. People embraced it quite well and it did not raise any significant concerns that we needed to take forward any further.
The Chairman: The processes you inspect are the processes of the providers that provide the service, not individuals working in them.
Lorraine Turner: We look at the competence of the service as a whole, but that inevitably means that we will look at the competence of individuals, and we will look at the systems and processes that the organisation has in place to assure and maintain the competence of individuals.
There needs to be a system where there is evidence that somebody has had the appropriate training and has the right knowledge and experience to perform the duties that are assigned to them. We look at objective evidence of that. We will not necessarily look at every member of staff. We will sample across the different types of activities and we will look at their competence in different ways. We do not attest to the competence of individuals. It is the sum of the parts within that organisation.
Lord Griffiths of Fforestfach: How tough are you? How difficult is it to pass accreditation? When you go to a company and you have certain standards and so on, how often would you typically say, “It is a first-class organisation. We have no comment”? How often would you say, “There are certain remedial actions that need to be taken”, or, “I am afraid we cannot accredit you”? If you had categories like that, what would be the result of the last few years, for example, of what you have done?
Lorraine Turner: It would be very rare that we conducted an assessment of an unaccredited organisation and did not identify any non-conformities. There will pretty much always be non-conformities.
Hopefully, if we have done the preparation with them, and they have had a pre-assessment and taken on board the comments that have been picked up, the initial assessments should be successful, with only a small number of matters that are hopefully minor in nature.
Sometimes we need to conduct further assessment activities. Across the board in UKAS, probably less than 10% or 20% do not succeed first time. We have seen a higher number of revisits with some of the new activities in forensics, because the organisations are perhaps not as prepared in some cases as they might be. That is one of the key aspects.
We were talking about how we keep the costs down. The better prepared the organisation is coming forward, the more likely it will be successful first time. Typically, there are not that many cases where we conduct an initial assessment and cannot make any recommendation at all that does not relate to follow-on work. It would probably be a matter of a quite catastrophic failure.
Lord Hunt of Chesterton: You used the phrase “quality incident”. My question from thinking about this is: do you look at trials afterwards where there were two very different cases and scientific or technical questions emerged from that, and use that data?
Katherine Monnery: There are two things. A quality incident would mean something a little different. We would certainly ask the organisation to review any work that has gone through the courts, and if there has been any significant challenge to raise that itself as non-conforming work within its systems. That would form part of our assessments and be something to look at.
Lord Hunt of Chesterton: I was an expert witness in a case and it was palpable that the science being used was extremely old fashioned and wrong, and I would have thought that was a quality incident.
Katherine Monnery: Absolutely, yes.
Lord Hunt of Chesterton: And I suspect that these developments are happening more rapidly than every five years. That was my point.
Katherine Monnery: If any negative judicial comment were made, or if there were anything negative in the courtroom, we would expect organisations to take cognisance of that, take it into their systems and in effect deal with it as a complaint, and put it through a non-conforming work process to raise issues, investigate and take some corrective action. We would certainly look at that.
We also see what is happening in the courts, and if we see something from an accredited organisation we expect some action to have been taken. A quality incident can take very many different forms. It may be something along those lines, or it could be a contamination incident, an erroneous method has been used, or a piece of equipment has gone outside the calibration and therefore the results generated by it might have some doubt cast on them.
We expect people to take cognisance of that and to do something about it: to investigate and to take some corrective action. They need to inform us. That is also a very important part of the accreditation: if something comes to light in an organisation that is accredited, we expect it to alert us to that straightaway so that we can review it and make sure that we are involved in seeing the investigation and the corrective action that has taken place.
Q159 Lord Thomas of Cwmgiedd: What external steps do you take? Expert witnesses are dishonest from time to time; they colour the evidence, and they are unreliable. How would you check on this externally to ensure that the individuals are fit and proper to be doing their task in each of the bodies that you accredit?
Lorraine Turner: We will look at the staff within that organisation.
Lord Thomas of Cwmgiedd: Do you go to outside people and ask, or do you rely purely internally on what you are told?
Lord Fox: Do you look deeply into their eyes?
Lorraine Turner: We would look at the body of evidence that they have, and that needs to be more than just a statement from the organisation. That would include any professional qualifications they hold, the participation in quality assurance activities in the organisation that might be set externally, proficiency testing, external quality assurance measures.
The individual experts giving evidence in court are not currently part of the assessment and accreditation process. The Forensic Science Regulator is working on an appendix to the codes to elaborate the requirements so that that can be included. Currently, the process takes us up to the point where they issue the report, so their performance in court is not included in our assessments.
Lord Thomas of Cwmgiedd: Their ability to explain forensic science to the jury, to take a very basic level of competence, is not assessed, nor is their integrity.
Katherine Monnery: One of the requirements in the Forensic Science Regulator’s codes of practice is exactly the ability to give evidence, in the sense of trying to explain to a jury in layman’s terms the science they are using and making sure that is possible. We test that out during our visits by sitting down with people and saying, “Explain to me what you have done? How did it work?”, and asking the questions that might come out in a courtroom with someone unfamiliar with the evidence or the science behind it.
Lord Thomas of Cwmgiedd: Do you go to the courts and ask them what this person is like or test an organisation?
Baroness Manningham-Buller: It sounds as though you have had some experience, Lord Thomas.
Lord Thomas of Cwmgiedd: Indeed, a lot of experience of bad experts. I wondered how you got them out of the system. That was my question.
Katherine Monnery: Currently we do not cover that aspect.
Lorraine Turner: We do where there is feedback within the service itself. We look at the complaints and at any judicial feedback that it has received during the process in order to ensure that that is picked up and addressed, but looking at the performance in court is not currently covered.
Katherine Monnery: Also, anybody can come to us at UKAS with any feedback about any accredited organisation. We take that very seriously and follow it up. That might come to be a quality incident. If we received some feedback in relation to experts from an organisation that holds accreditation, we would follow that through.
Q160 Lord Mair: I want to clarify this point and both the questions, the earlier one from Baroness Manningham-Buller and the one just now from Lord Thomas.
In your written evidence you talked about how organisations get accredited in respect of the competence of the people reporting and interpreting the evidence. It is very closely linked to what we have just been talking about.
Katherine Monnery: Yes.
Lord Mair: Can you say a bit more about that? You talked about “further requirements” being necessary. Can you explain what that means? In other words, how do you accredit how good somebody is going to be at doing exactly what Lord Thomas has been talking about: being in court where the test results are taken as given, but there is a question over the way they are explained and the way they are interpreted? How do you accredit that person?
Katherine Monnery: It is important to clarify what opinions and interpretations we currently accredit. Part of the 17025 standard covers the ability to make opinions and interpretations based on the test results. Currently, we are doing a lot of work with the fingerprint comparison bureaux. That is an example of the test itself being to review a known mark and an unknown mark and look for similarities. The opinion would be, “Given that information, is it likely to be Katherine Monnery’s fingerprint?” You are drawing upon expertise of working in that field to make the inference and the opinion above and beyond the test result. That is the kind of opinion and interpretation that we would currently accredit. It is the direct interpretation of the test result.
What we currently do not accredit, as Lorraine explained, is the expert opinion given in court, which might be source and activity; I think you have heard that term previously. We might look at a source-level approach to things: that is to say, whether it was Katherine Monnery’s fingerprint. How it got there, the activity, is something else again, and that is often the expert opinion given in court. Currently, that would not be part of the accreditation process.
In relation to the opinions and interpretation element, we look at quite a number of other parts, as we said: the qualifications, the experience, et cetera, and a little of the knowledge around the edges of the science and technologies—how things work and operate. We ask lots of questions about that in interviews during the assessments themselves, and we look at the impact of variables, the uncertainty of a measurement, the limitations of a technique. It is quite important in some areas to understand that. We also look at legal requirements, such as the Criminal Procedure Rules or the Practice Directions, and ensure people are aware of their responsibilities as experts going into the courtroom.
Lord Mair: When an organisation comes to you to be accredited or to have its accreditation renewed, you look at the various test procedures that it does. I think that is clear. But does it submit the individuals working for the organisation, who are the types of people who might well be experts in court?
Katherine Monnery: We would not necessarily look at that during the application process or define that, but during the assessments we would ask to see the organisational structures—who was working in different areas within an organisation. We would look at the job descriptions of the people in it and any person specifications to do with qualifications or experience that would be determined for each of the different roles that might be in the forensic organisation. We would see that it had staff fulfilling those roles and that those staff had the requisite qualifications and experience, and that they had been trained and had the competency, demonstrated through objective mechanisms. That would be part of our assessment.
Lord Mair: Is the result of the accreditation that certain people in the organisation are given some kind of rating?
Katherine Monnery: No, it would be the organisation that held the accreditation, not the people themselves. It is not transferable in a sense. If somebody leaves an organisation, they do not take that accreditation with them per se, because they have been assessed under that organisation. It is the organisation and its ability to maintain the competence of the staff within it.
Q161 Lord Fox: The converse of that question is how many of the staff would have to leave before it lost the accreditation.
Katherine Monnery: We would look at resources.
Lord Fox: How would you know?
Katherine Monnery: Organisations need to inform us of significant changes to their organisation, and that may well be loss of staff, as an example.
Lord Fox: What is “significant” in that context?
Katherine Monnery: If they were not able to deliver the services for which they had a contract.
Lord Fox: If you switch someone out, you can still do the service but there is an entirely different suite of individuals or people. They may still be using the same process that you have accredited, but they are different people.
Katherine Monnery: If the person is a significant person—the accreditation is incumbent on the knowledge of that person; perhaps they are a technical manager—if that person were to leave, we would need to be informed. We would not need to know every staff movement if it was at the analyst level or just people coming and going, as would happen in any organisation.
Lord Fox: Does that happen?
Katherine Monnery: People come and go in any organisation.
Lord Fox: Do you get companies informing you that this has happened?
Katherine Monnery: Absolutely. That might prompt us to consider whether we need to pay an extra visit to see whether another member of staff is being brought in and whether they have the requisite experience and qualifications to continue with that role. It might not be specific people. It could be numbers of people leaving that means the organisation cannot continue with that service.
The Chairman: While we need to move on, you have created a lot of excitement, so answers need to be exact to the questions. I will start with Lord Griffiths.
Q162 Lord Griffiths of Fforestfach: What percentage of the organisations that you investigate or review are public sector organisations, such as the police, and what percentage are private sector organisations, such as private companies?
Katherine Monnery: Do you mean in the forensics sector?
Lord Griffiths of Fforestfach: Yes, just in forensics.
Katherine Monnery: We have some numbers, but I am not sure we have those exact numbers. We could provide you with those.
Lord Griffiths of Fforestfach: The second question would have been: are there notable differences between commercial organisations and non-commercial organisations in the problems that they throw up to you, or perhaps the questions?
Lorraine Turner: It is hard to make a sweeping differentiation between the two. We have been assessing for many years the large private sector providers that have been accredited, so we have a lot of history and information about their performance. For some of the police forces—not all of them by any means; the Metropolitan Police, for example, have been accredited for a long time—for some of the newer organisations, we are still in that process of building up that information. There are different challenges for the public sector and the private sector.
Lord Griffiths of Fforestfach: I am very used to regulation in the financial sector. To take the point Lord Thomas made, there are bad apples in organisations, and when they are discovered you may find that they have done a lot of damage. Whether the same would be true in a public sector organisation, I do not know. My hunch would be that if you are looking at private sector organisations, you need to have somewhat more intense supervision than if you are looking at a public sector organisation. That may be totally wrong.
Katherine Monnery: There are challenges for both. It would be wrong to say that one had different challenges in this regard than the other. There are challenges from budgetary constraints in both sectors and that is possibly where people feel pressured to make wrong decisions. That is common in both areas currently.
The Chairman: I ask slightly tangentially, without mentioning the court case going through just now or the organisations involved, whether your accreditation processes would have picked up the problem that occurred. I think you know what I am referring to.
Lorraine Turner: In general terms, there is a mechanism that we have in place with all the forensic laboratories whereby they are required to report to us any significant quality incident. That is a requirement of accreditation. Any organisation identifying major issues or deviations from its processes is required to report those as quality incidents. Interestingly, we tend to get a greater number of quality incidents reported from the organisations that have been involved in accreditation the longest, because they are used to identifying issues and reporting and repairing those issues.
With regard to the particular case that you are talking about, in general where significant incidents are reported, we have to investigate and we have to understand the issue. In cases where there are skilled individuals who are intent on conducting a criminal activity or an activity that is not appropriate, and they are very clever in covering that up, it is less likely that the internal systems within the organisation, or our assessments, will pick that up.
The key point from all quality incidents that get reported is that we learn from them. We look to see whether something needs more scrutiny or a different emphasis. Our assessments are a sampling exercise. We move in on-site for one, two or three days, depending on the scope of the organisation. You cannot look at everything all the time, so you are relying on their systems being in place to maintain their performance.
Q163 Baroness Morgan of Huyton: Is there some sort of process for whistleblowing? Do individuals in an organisation have the capacity to go directly to you?
Secondly, you talked a lot about people being required to tell you things and knowing that they would have to. Where are they told? Is there a pro forma, and, if so, can we have it so we understand exactly what is required of an organisation?
Lorraine Turner: We can certainly cite the relevant requirements they are required to follow. Of course, if we identify that they have not told us something that they should have told us, there are sanctions that can apply.
Baroness Morgan of Huyton: And whistleblowing?
Lorraine Turner: Yes, anyone can come to us about any of our accredited organisations with information and we will investigate it.
Lord Fox: Do they?
Lorraine Turner: Yes, they do.
Lord Borwick: The process you describe is fairly similar to the process used in many other industries for regulating the people in their industry. I am thinking of the CAA for pilots and the FCA for financial services. Is there a metric by which one can judge the efficiency of the regulator? In other words, how much does it cost you to accredit somebody working in forensic science in comparison with similar accreditations in other sectors? Are you more efficient than the CAA? How would you compare yourself with the FCA?
Lorraine Turner: We are not the regulator.
Lord Borwick: I appreciate that, but the process is similar.
Lorraine Turner: We have not done benchmarking against the regulator specifically in our role, perhaps because we are not a regulator, but I can say that accreditation is used by other regulators as a mechanism to demonstrate competence and to support the regulation. The metrics and the costs will be similar, because our pricing model is common across the organisations that we assess and accredit. Perhaps the economics are different in the organisations depending on their size and their scope of business. As has already been identified, we are aware that the costs might be higher, given the proportion of turnover for a small organisation compared to a larger one.
Q164 Baroness Young of Old Scone: One criticism we heard about some organisations was that they are strapped for cash and therefore slow. Do you look at the whole issue of pace and timeousness in your accreditation?
Lorraine Turner: Do you mean in delivering their service?
Baroness Young of Old Scone: Yes.
Lorraine Turner: Yes, we do, because there are requirements in the standards in relation to them meeting their customers’ requirements. While the criteria for accreditation do not specify particular timescales, they would come from the contract and the delivery of the service. If we identify that those criteria, timescales or KPIs are not being met, we will report on that, but the standard itself does not define what those timescales should be. We would look at a number of measures in the organisation—what the contracts require, whether they are delivering, whether they have had any feedback, positive or negative, on what they are delivering to their customers.
Baroness Young of Old Scone: If you find failure on any parameters, and you talked about sanctions, do you apply the sanctions or does the regulator apply them?
Lorraine Turner: We apply sanctions in relation to accreditation. At the very basic level that would be because of a non-conformity, which might not alter its status of accreditation initially but might require it to carry out some corrective action.
We can also apply sanctions to the organisation by suspending part of the scope of the activities from its accreditation, so if there is a problem with a particular test or group of activities, that be can be removed. We can suspend accreditation for the whole organisation if we identify significant failures. The regulator will be informed of the sanctions that have been imposed, but we are not imposing those sanctions on behalf of the regulator. It is in relation to their accreditation and accreditation status.
Q165 Lord Hunt of Chesterton: How closely do you work with the Forensic Science Regulator in setting standards and awarding accreditation? I would like to connect to that the role of working with the science organisations, the EPSRC and the other research councils. Do they connect to the Forensic Science Regulator, or do they set up research projects that are of interest to you in UKAS? Later this afternoon we will be talking to others, but in none of our questions do we have this specific point about how we connect to the whole of the scientific infrastructure.
Lorraine Turner: We do not have a direct relationship with the science research councils, apart from on our advisory group, and by tapping into the various groups that the Forensic Science Regulator has in all the different disciplines and subject areas, we would connect with those scientists.
Lord Hunt of Chesterton: When you apply for a research grant you have to say endlessly how this work is going to be applied, so surely the research councils are reading these documents, all of which say “application, application, application”. Does that not include application to forensic science?
Lorraine Turner: Our role is primarily with assessing and accrediting the forensic science providers who are carrying out routine testing, so the scope of their accreditation relates to their repertoire of tests. They may also conduct research, and if that research relies on testing as part of that, that may fall within the scope of accreditation, but we are not specifically looking at their research activities across the whole range.
Lord Hunt of Chesterton: Does it not help your work?
Lorraine Turner: It helps our work to be connected with developing technologies. It tends to get presented to us when an organisation comes forward using that new technology or methodology as part of their routine work.
Lord Hunt of Chesterton: The role of these private sector companies, for example, is quite important in transmitting science from the research world to the application world. Is that right? That seems to be what you are saying.
Lorraine Turner: Yes.
Katherine Monnery: They would apply the science, I think, and come to us to accredit that activity. It might be novel and innovative and a new area of science or a new application of science and they would come to us to ask for the accreditation.
Lord Hunt of Chesterton: We heard evidence from one of our previous sessions that some 150 research programmes of the research councils were all to do with forensic science and the application of that forensic science. You sound a bit diffident, if I may say so. That witness was very gung-ho about all these applications that were working.
Lorraine Turner: Our role in the process comes later on. Once the new technology has been developed, whether that is in the private sector or by academia, and it gets deployed as routine activity, we look at the body of evidence that demonstrates that that technique or process is valid and fit for purpose.
The Chairman: Does the regulator get involved in your work? That was the original question.
Lorraine Turner: We work closely with the regulator. Obviously, she is a key stakeholder in the delivery of accreditation in this area. The regulator has no role in making any decisions on accreditation, but the various committees and groups that she facilitates and our own advisory group help us to steer our activities.
The Chairman: As the accreditation bodies, who are you accountable to?
Lorraine Turner: We are accountable to a number of different organisations, in the first instance to government through the memorandum of understanding that we have with the Department for Business, Energy and Industrial Strategy.
The Chairman: How does it hold you to account?
Lorraine Turner: It monitors our activities through a number of different mechanisms. It participates in our policy advisory council and forum. It is very well connected to know what is going on. We are monitored and evaluated as an organisation by our peers within the international infrastructure of accreditation. We are a member of the European co-operation for Accreditation. That facilitates mutual recognition of accreditation between countries in Europe through EA and globally through the International Laboratory Accreditation Cooperation.
The Chairman: That is not accountability, is it? It is an association, but you are not accountable to it.
Lorraine Turner: We are peer evaluated against an international standard, so we have to demonstrate that we have robust, independent and impartial processes. We have external stakeholder groups which provide advice in to us. We are also accountable to our member organisations, because although we are a private company we are a company limited by guarantee, and our members include government departments, associations of laboratories and so on.
Q166 Lord Fox: I should probably have asked this question before, but how much of forensic evidence being used in court is accredited and how much is not accredited?
Katherine Monnery: I do not think we would have figures for that.
Lord Fox: Is it roughly 50:50 or only a tiny percentage that is not accredited? What is the broad bulk of the answer to that question?
Katherine Monnery: I suppose the question relates to whether it was scientific testing or examinations undertaken in laboratories. I think you have heard that the main forensic providers hold a large amount of accreditation, so any work going through those providers would most likely be covered by their accreditation.
We find that less so in companies that have not come to us for accreditation. There has been a lot of talk about digital organisations. There will be digital organisations that are not accredited and presenting evidence in court.
Lord Fox: In your experience, how often is the court aware that the evidence is accredited or not accredited?
Katherine Monnery: I think that is becoming more evident and is being asked about more frequently now. Five years ago, that question would never have been asked, potentially, but nowadays, with the role of the Forensic Science Regulator and the deadlines for accreditation, it is being asked much more frequently.
The Chairman: What assessment do you make of the evidence given by people who are not accredited?
Lorraine Turner: We do not.
Katherine Monnery: No.
Lorraine Turner: Our role is to conduct the assessments and accreditations of the organisations that come through our door. It is the goal of the Forensic Science Regulator to have that include a significant proportion of the scientific evidence. The statement of requirements has picked off the different evidence types.
The Chairman: Do you have a view as to whether the evidence presented to the court should only be from accredited organisations?
Lorraine Turner: I suppose coming from an accreditation body we would always respond to that quite positively.
The Chairman: But you have to back it up. Why is that?
Lorraine Turner: It is not really a decision for us. It is for those who use that information to decide whether those quality standards are the right standards to be used for that context. Of course, we would support the view that information from accredited organisations is used.
The Chairman: Quick questions and brief answers, please. Baroness Young and Lord Fox.
Q167 Baroness Young of Old Scone: Whether you are accredited or not when you appear in court is pretty random, so do the sanctions work at all if not all organisations are accredited?
Lorraine Turner: The sanctions that we apply are only applied to organisations within our system.
Baroness Young of Old Scone: If you were to remove accreditation, how much damage would that cause for example to a police in-house service?
Katherine Monnery: We would publish the fact that an organisation had had a sanction applied.
Baroness Young of Old Scone: You would not prevent them from presenting the information to court.
Katherine Monnery: We would not, but currently, as I was saying, the codes of practice require people to declare compliance or non-compliance with the statement in the regulator’s codes of practice. If they had been declaring compliance previously, they would have to declare non-compliance with the codes of practice, which may result in questions being asked about the credibility of the evidence they are putting forward.
Lord Fox: The Lord Chairman asked you whether all evidence-givers should be accredited and you said it was not for you to say, but I think it is for you to say what your limitations are. Are you equally good at certifying a large organisation with lots of scientific process as you are a sole trader who does, let us say, forensic archaeology? Do you cover the full spectrum, or are you much more likely to be useful for that big organisation with lots of scientific process?
Lorraine Turner: The opportunities for accreditation to be developed in other areas of forensic science in addition to those where we are already working are there, and there is scope for us to do that where there is the demand.
Lord Fox: So you are saying that you do not have limitations.
Lorraine Turner: I would not say that there are no limitations at all. We would come back to whether it is accreditable and can be seen as an objective repeatable method.
Lord Fox: A method. Understood.
The Chairman: Lord Fox, would you like to ask your question?
Q168 Lord Fox: Turning to the silent partners at the end of the table, just when you thought you had enough standards, BS EN ISO 21043 or something is creeping up. I understand that you are working on some other standards. The question first is why? What are you seeking to do? Are you merely offering a competitor standard or filling a hole? What is that hole? How did you understand that hole? In other words, how did you develop the scope of what you are doing and with whom did you work on that scope?
Sara Walton: Those standards [ISO 21043 series] are being developed in the international standards organisation space. The UK is inputting expert information and expert content into those standards. They are quite new process standards, so they are standards that are designed to support people who are in this area.
Lord Fox: They do not supersede 17025.
Sara Walton: They do not supersede it at all. If anything, they support what is going on in 17025, but they are not designed to do anything other than support.
Lord Fox: You will have to help me, because I am not a standards expert. What on earth are you seeking to do?
Sara Walton: We are not seeking to do anything, because as the national standards body we facilitate everybody who needs to get involved.
Lord Fox: So a group of people could go into a room and decide that they want a new standard. But why do they decide that?
Sara Walton: They might decide that they want a new standard, but it has to be market relevant and there has to be a need for it. [They might decide that] within the stakeholder community, which includes government departments, regulators, industry and public interest groups, there should be something in that space that would help people.
Lord Fox: Sorry, I am struggling. What are you seeking to cause to happen? In other words, what are you making better? Does it measure hydrochloric acid better? What is it doing?
Sara Walton: It is just guidance. Let me read you the scope of one of those. “Forensic science: Part 1. Terms and definitions”: so it is about the language that people are using. It is about supporting the consistency and repeatability of any activity in this space.
Lord Fox: It is taxonomy.
Sara Walton: Yes, taxonomy. Part 2 of ISO 21043 “The recognition, recording, collecting, transport and storage of material” is another Standard, dealing with the specification of the consumables being used.
Lord Fox: I thought that was already being done in the other ones.
Sara Walton: ISO 17025 is a different type of standard. It is requirements for the organisation, as has been described here, and that organisation is accredited to that standard, which is different from forensic science processes, as in the 21043 standards.
Q169 Lord Mair: Can you say something about the relationship with the European standards organisation CEN? How does it operate?
Sara Walton: It is a similar relationship to ISO. BSI as the UK standards body is a member of CEN and CENELEC, which are the European standards organisations. These standards that we have just discussed are international standards and they are not necessarily being adopted within the European space.
Lord Mair: The driver for updating a standard comes mainly from within the UK.
Sara Walton: It could come from the European space. It could come from another country member of ISO.
Lord Mair: Does that happen in forensic science?
Sara Walton: It happens in all areas. Another country, one of the 165 country members of ISO, may decide that some organisational capability in the country needs support and it may then propose something to help. For example, we have quite a mature system in this country, as you have heard, for accreditation of forensic science and laboratories carrying out those tests, but perhaps in other countries around the world that is not the case. These standards may exist, but whether they are adopted and used within other countries is another matter. These have been adopted in the UK, but I do not know how much they are used. They are supposed to provide guidance and suggested help for processes within an organisation.
Lord Mair: Are there any countries that are prominent influential leaders in forensic science?
Sara Walton: We are pretty prominent.
Lord Mair: I meant apart from the UK.
Sara Walton: I could not say exactly. Would you know of any particular countries, Katherine? Germany is quite strong.
Katherine Monnery: In this area, Australia would be quite proactive in that sort of activity, and other European countries.
Sara Walton: Australia runs the international committee for standards in forensic science.
Lord Fox: You did not help me with regard to the stakeholders, so I will pitch the question in a slightly different way. How did you go about deciding that you were going to get involved in the standard? Did you have a meeting at which people came up to you and said, “We desperately need this standard”, or did you think, “We are a business and we need a product. We do not have a product in this area, so why don’t we do this”? If it is the former, who came up to you and what were they asking for?
Sara Walton: As I say, this would have taken place in the international community, so I am not entirely sure. We would have to check who proposed the standards that we have just talked about.
Lord Fox: I am trying to work out what the demand is for this.
Sara Walton: Exactly; so there could be demand from industries or regulators [for a Standard]….
Lord Fox: It would be helpful if you could let us know offline what the process was for deciding to do this.
Sara Walton: We can certainly give you some background on those standards.
Q170 Baroness Morgan of Huyton: Can I ask about compliance with ISO standards? We have had evidence that compliance at the moment is quite low among some disciplines, and we heard particularly about digital forensic and fingerprints. It is fingerprints that I am most interested in, because I think the quality standard was set in 2014, and here we are, past the deadline, and our understanding is that a lot of organisations have not yet become compliant. What is going on? Why has that happened?
Katherine Monnery: There have been a number of reasons. The main driver has been the secondary legislation that has come into play. I think that has focused a lot of minds on progression and taking things forward. I have to say that we have seen a lot of really good progress in the last six months or so in relation to fingerprint comparison. It is moving in the right direction in terms of compliance.
If organisations do not come to us to ask for extensions to scope for particular areas, we cannot take them forward, so there was perhaps a delay in some of those organisations coming forward. When they come forward and we have done work in the preliminary stages, we may have identified a number of significant gaps in what they were doing to meet the requirements for the standards.
Validation was an area where a lot of people had to do a lot of work in meeting the requirements of the standard. That is quite an extensive area to look at, because people need to decide what they are going to validate, and they need to develop the ground-truth databases and to have the artefacts to put through the validation to demonstrate that the methods are appropriate.
Having identified that there is a gap, there is still a lot work to be done to get it to a position where it is compliant with the standards. That has taken a bit of time to do, but people are now in a position where that is happening, and we are seeing much more positivity in the assessments that we are undertaking and the outcomes from those assessments, so it is looking better.
There are a number of learning points in that area that could be taken forward into other areas to meet other deadlines. From a UKAS perspective, we took an innovative and novel approach to adopting slots for accreditation in the preceding year running up to the deadline to ensure that everybody had an opportunity to have an assessment in time to be compliant with the deadline. Not everybody took us up on those slots or was sufficiently prepared for that slot. The community needs to take that on board.
Baroness Morgan of Huyton: What you are really saying is that until there was a looming deadline, although it was set four years ago, nobody did anything really, so you have had it all in the last six months.
Katherine Monnery: I think it focused the minds of people, particularly the secondary legislation coming in. The regulator’s deadline was always there, as you mention, but there was the prospect of legislation coming in.
Taking the converse as well, the regulator’s deadline in relation to digital forensics has passed, but we have not seen that pick up because there is still no requirement for that to be accredited.
Baroness Morgan of Huyton: Does that suggest that you really do not have much muscle? Frankly, if the deadline passes and they are not too disturbed about it, does that mean that they do not feel that it matters hugely to them?
Lorraine Turner: As the services and the whole system have become more aware of the requirements and of the need to disclose if accreditation is required by the regulator and not held, that in itself will also focus minds, because to continually have to report that this work is not under accreditation highlights that gap.
Q171 Lord Thomas of Cwmgiedd: With regard to fingerprints and digital forensics, the police have a huge responsibility. What is done to bring to the public’s attention the fact that there is a deficiency in accreditation? Why are you not accountable to tell people that there is a shocking state of affairs?
Lorraine Turner: For all the organisations that are accredited, that information and what they are accredited for is in the public domain.
Lord Thomas of Cwmgiedd: That is not the question I asked. What do you do to bring to the attention of the public that certain police forces are not accredited in vital areas such as fingerprinting and digital forensics?
Lorraine Turner: Nothing specifically by ourselves, but we work very closely with the Forensic Science Regulator to look at the reasons why the services have not come forward, the delays, and whether we can do anything to raise awareness and to provide those services with appropriate support and information, because we all want that positive outcome to pull them through the process.
Lord Thomas of Cwmgiedd: Really, it is a question for the police and the Forensic Science Regulator and not for you.
Lorraine Turner: I think more significantly for them, yes.
The Chairman: Lord Mair.
Lord Mair: My questions have been answered already.
The Chairman: Lord Renfrew.
Q172 Lord Renfrew of Kaimsthorn: Concerns have been expressed in written evidence that the 17025 and 17020 standards are not particularly well suited to the forensic science discipline and particularly to digital forensics. Would there be benefits in developing standards that are particularly relevant to forensic science?
Lorraine Turner: The first thing for us to say is that we do not hold the view that standards 17025 and 17020 are not appropriate and fit for purpose for those areas. When the standards are applied in new areas, there is a need sometimes to provide interpretation of those requirements, but those requirements are that the organisation has in place a management system, defined policies and processes, competent staff, suitable equipment, suitable reporting mechanisms and appropriate methods. I find it hard to understand why those requirements would not be seen as appropriate for those areas, but we understand that the application of those generic requirements sometimes needs interpretation.
The process for accreditation is not just the standard; it is the use of peers and other experts to interpret those generic requirements and apply them in order to understand the competence of the organisation. It is not our view that we need more standards to evaluate the competence of organisations carrying out those activities—not at all. In fact, we have a body of evidence and have now conducted over 60 assessments in the area of digital forensics, and with the sorts of findings in relation to lack of validation, staff in the organisation not being aware of the limitations of method and so on—there is a very long list of things—it is hard to understand why that would not be seen as appropriate.
Lord Fox: My experience with quality systems is that they work when you can lock down the process. That is how manufacturing has moved over the years. You have a process, you refine it, and you lock it down and ensure quality through assuring the process. I can see how that works in some cases—DNA testing is a very good example of where it does—but it seems to me that the vast majority of the sort of stuff you seem to be talking about does not fit that process.
You come back to the point that you have to accredit the people who are doing it because you cannot accredit the process as such. That system of 170 whatever it is does not appear to be accrediting people, so how can you ensure the quality through what you are doing? I do not see how it works.
Katherine Monnery: There are a number of things that we can do when we are looking at more interpretative aspects.
The Chairman: You can do or you do?
Katherine Monnery: We do. There are things that we do when we are looking at aspects that are more interpretative or that require slightly more expertise than DNA processing, where it might be more procedural.
Lord Fox: Where it is a process. I understand.
Katherine Monnery: Certainly a procedure would still be required, but we might give a little more flexibility in that procedure to allow for the expertise of the individuals. The rider to that is that the records maintained by the individuals undertaking that examination perhaps need to be a little more detailed to account for what they have done in that examination. In some examinations there might be a procedure that allows for flexibility but records are increased. That will allow the organisation to review what the experts have done and to ensure that their processes, including perhaps peer review of their processes, looks at the suitability of the approach taken by the experts.
Lord Fox: By necessity, that would almost be after they have done it, so it is post-delivery assessment.
Katherine Monnery: We mentioned previously the competence aspects of making sure that people put into positions where they undertake the interpretative aspects have their competence reviewed with that in mind. There are a number of checks and balances in the training and competence assessments to ensure that those people are appropriate to be given that flexibility, if you like, to undertake that activity. There is that check and balance at the beginning of the process to ensure that people are competent.
The process of defining the principles and the process of recordkeeping to indicate what has been done can be audited or reviewed, and we would certainly check all those parts of that process as part of our assessment. We also witness people undertaking activities, and we have a lot of interviews with staff—“What would happen if this were to happen?”—to explore the process itself in different ways.
Q173 Lord Renfrew of Kaimsthorn: I would like to hear a little more about the specific application to digital forensics and what we were just talking about. That would be very helpful.
Katherine Monnery: The competence of the practitioners is an element that we would look at to determine the questions being asked of an examination, the best way to undertake that examination, the combination of software tools that would be used as part of that examination. We would look at the processes associated with that to determine the individual’s competence to undertake that, and to understand the limitations perhaps of some of the tools used in particular extractions or implementation to ensure that they understood those limitations as well and applied them appropriately.
Lord Renfrew of Kaimsthorn: More particularly, are there standards that are particularly applicable to digital forensics that could be further defined and explored?
Lorraine Turner: If the sector itself feels that there is a need for the development of further standards—process standards, procedural standards—as we see in other areas of testing, there might be defined methods that sit in the national or the international standards system.
That is really for the market itself to decide. We do not believe that there is a gap in competence standards for the organisations that undertake digital forensics, whether they are small organisations or large. We have a body of evidence, which we can share with the Committee, on the outcomes from the large number of assessments that we have done that indicate that it is changing things and that it is having an impact and improving quality, which is the Forensic Science Regulator’s aim, and I think all stakeholders would expect that high quality to be there.
Katherine Monnery: For digital organisations that have applied for accreditation, there are additional appendices to the regulator’s codes of practice relating specifically to digital activities that we would apply to those assessments. They might look specifically at things like validation, and we apply those where people have requested that assessment.
Lord Hunt of Chesterton: I want to follow up on Lord Renfrew’s question. We are in a period where some sciences are moving more slowly than other sciences. Clearly, everything to do with digital is moving enormously fast, and I wondered whether the whole methodology that you are using is changing for areas that are digital.
You have given a methodology that applies everywhere, but sometimes there is an extraordinary leap forward. We heard from the Turing Institute about the amazing range of digital approaches applied to psychology, medicine and natural sciences. We are in an extraordinary period. Are you gripping this dramatic change? Public officials should never use the word “revolutionary”. Nothing is too exciting, because it upsets all sorts of people. We are in an extraordinary time here, and I wondered whether you are allowed to reflect that in what you are doing?
Lorraine Turner: We can build it in and we do build it in.
The Chairman: You alluded to this earlier. If you are not currently accrediting for digital forensics and you are just developing this, you only need to say so.
Lorraine Turner: We are accrediting for digital forensics. We have about 40 organisations that are covered by accreditation. It is an area we are working in. They define their scope of activity, so they define the types of evaluations and tests. If they want new ones because technology is moving on, those need to be added on. We can, and do, award flexibility of scope. We look at their ability to access and deploy new technology. It is a challenge for them to do that, but we have examples in fast-moving areas, because although digital is today’s fast-moving area, we have seen it in DNA—
Katherine Monnery: DNA is a fast-moving area.
Lorraine Turner: We have adapted what we are doing. The challenge is greatest for the organisations themselves, because they have to demonstrate their competence to us. They have that in place when they come across our door. It is the work that they have to do before they interact with us that is the bigger challenge, I think.
Q174 Lord Fox: Following on from the fast moving, how will you accredit artificial intelligence systems? There are two reasons why it will be difficult. First, it may well be an algorithm in a black box, which you are not allowed to see, and even if you could see it you may not understand it because no one can. How will you go about that going forward? Also, it will not be long before artificial intelligence systems are providing evidence, so what possible hope do we have of any accreditation for that?
Lorraine Turner: We recognise that it would be a challenge for us. We are seeing AI algorithms being used in data interpretation in quite a lot of areas now in healthcare, so we look at the validation and the ability to demonstrate consistent output. Yes, AI will pose some big challenges for us, so we need to work with all the stakeholders and the right experts to ensure that we have the right approach to assessing that.
In UKAS we have a history of designing and developing accreditation in new areas. We have been doing that for many years, so I am confident that we have a process that will give us the best chance of doing that, but we need to validate it.
The Chairman: A last question from Baroness Young.
Q175 Baroness Young of Old Scone: You have been in a fairly unique position in that you see right across the spectrum of providers, from police forces to the private sector. We have been hearing a lot about cuts. Some of them have been met by increased productivity and efficiency; some of them clearly have not. In your opinion, is the forensic science service improving or regressing at the moment?
Lorraine Turner: It is a mixed bag. There are a lot of improvements where the organisations have come forward for accreditation, and as Katherine mentioned earlier, with the deadlines looming we have seen greater engagement. We have seen shifts in culture towards embracing quality much more, but that still needs to continue.
There are other pressures in the marketplace itself that probably fall outwith our role and are particularly challenging for the sector. We do not have a great ability to influence that other than report back what we are seeing in general terms for the organisations that come forward and are assessed by us.
Baroness Young of Old Scone: Would you say in summary that the market is a good thing or a bad thing?
Katherine Monnery: There has always been a market in forensic science. We had the Forensic Science Service previously, but there was always commercial provision as well. There has been a switch with some of that commercial provision taking over some of the responsibilities which the Forensic Science Service previously had but insourcing into police forces to combat that, as it were, so there is still that mixed economy, as there always has been. Accreditation can apply to all those industries quite well, and we can accredit those facilities.
Baroness Young of Old Scone: I do not think that is the question I was asking. I was asking whether, in your professional and technical opinion, you thought that services had got better or worse over the last three or four years.
Lorraine Turner: We have seen an increased number of organisations gain accreditation, so I would say that, yes, it has got better, because we have seen those improvements. However, we also see the pressures in the market. It is a difficult question for us to answer, because we are providing assurance on the quality and the competence of those organisations rather than the configuration.
What we would say, though, is that there is a need to remain quite steadfast over the standards and the quality, so that however the market is configured you can have confidence in the outcome.
The Chairman: Thank you very much indeed. We have given you a good run for your money or our money, whichever. We are very grateful to you. If there is anything that you might have liked to have said, please feel free to send it in and we will record it as written evidence. Thank you for today.
