Public Administration Select Committee (PASC)
Oral evidence: Preservation of historical files in the Civil Service, HC 550 Tuesday 22 July 2014
Ordered by the House of Commons to be published on 22 July 2014
Members present: Mr Bernard Jenkin (Chair); Mr Nigel Evans; Paul Flynn; Mrs Cheryl Gillan; Greg Mulholland; Lindsay Roy; Mr Andrew Turner
Questions 1-127
Witnesses: Clem Brohier, Acting Chief Executive, The National Archives, Carol Tullo, Director of Information, Policy and Services, The National Archives, Scott McPherson, Director, Law, Rights and International, Ministry of Justice, and Simon Marsh, Head of Information Strategy, Policy and Practice, Ministry of Defence, gave evidence.
Q1 Chair: I welcome our witnesses to our session about knowledge management and Civil Service record keeping, which arises in the wake of the controversy about the 114 Home Office files, although they are not our prime concern today. We are thinking about the wider implications of what we learn from this episode, and we thought it was time to have a broader look at how Whitehall keeps its record. We are particularly interested in the corporate memory of Whitehall, and how that corporate memory is preserved and supported by documentary evidence. May I first ask you each to introduce yourselves for the record, please?
Carol Tullo: Carol Tullo. I am Director of Information, Policy and Services at the National Archives. Within my remit there fall the teams that guide, steer, advise and assess Government Departments on their information and records management.
Clem Brohier: I am Clem Brohier. I am Director of Finance and Corporate Services, and currently the Acting Chief Executive and Keeper of The National Archives. The Archives are the national archives for the UK Government and for England and Wales.
Scott McPherson: I am Scott McPherson. I am Director of Law, Rights and International in the Ministry of Justice where, amongst my responsibilities, I advise the Lord Chancellor on his overall policy and legislative roles in relation to information management.
Simon Marsh: I am Simon Marsh. I am Head of Information Strategy, Policy and Practice and departmental record officer for the Ministry of Defence. On 4 August, I join the Home Office as Deputy Director for Knowledge and Information Management.
Q2 Chair: Home Office Information Management is one of the recognised professions around Whitehall. Is that right?
Simon Marsh: Yes, Knowledge and Information Management is one of the 27 Government professions.
Q3 Chair: We will ask questions as briefly as we can. If you can keep your answers as crisp and short as you can, that would help us. If I feel that you are going on a bit too long, I will pull you up, so can we move through swiftly. Can I just start by asking about accountability at the National Archives? You are established by statute and have been since 1838, updated by the 1958 Act, as modified by 1967. To whom are you accountable?
Clem Brohier: We are accountable to the Lord Chancellor, and the Lord Chancellor has certain responsibilities for the public record system. We are within the MOJ family. We are separately funded from Treasury, which gives us a certain level of objectivity as we work across Government.
Q4 Chair: Do you feel that the status you have under statute gives you a measure of independence that you might not otherwise have?
Clem Brohier: Yes, we do, and certainly that is what we feel when we conduct our business across Government Departments.
Q5 Chair: How day-to-day is your relationship with the MOJ? How regular is that contact and do you feel that you are managed by them? How does the relationship work?
Clem Brohier: It tends not to be day-to-day; it will be probably monthly, but with an update meeting, because I have a dotted line through to our minister, Simon Hughes. I will meet senior civil servants within the MOJ, and that is really by way of update and just the general business of the National Archives. I meet annually with the Lord Chancellor, and that is to give an overview of the work that we conduct on his behalf, but day-to-day we are a pretty antonomous body.
Q6 Chair: Are you actually a non‑ministerial Department?
Clem Brohier: We are.
Q7 Chair: When somebody asks a question in Parliament about the National Archives, how is the answer given?
Clem Brohier: The question will normally be directed to our Minister, and that is Simon Hughes currently. The information to answer that question will come from the National Archives.
Q8 Chair: He will usually answer in person. He would not publish your letter.
Clem Brohier: He will ask questions of our office and we will supply his office with the information to answer the questions.
Q9 Chair: The answer will be given in his name, in his voice.
Clem Brohier: Yes, it will.
Q10 Chair: In terms of your own policy for your own organisation, do you recommend that to be adopted or do you accept what is given to you? How does that work?
Clem Brohier: It is actually a policy that we would advise and develop on behalf of the Lord Chancellor, and it is given in Section 46 of the Code of Practice of the Freedom of Information Act. In relation to the policy, if we feel it needs to be developed, we advise accordingly.
Q11 Chair: What relationship do you have with Government as a whole, in terms of what Departments are doing about their record keeping and their records management?
Clem Brohier: We advise, guide, supervise and co‑ordinate, largely to do with record keeping. In particular, we are interested in good record keeping and making sure that records are kept in good order, so that when they hit the 30‑year rule—which is now changing to 20 years—they are in good order and can transfer to the National Archives for our safekeeping.
Q12 Chair: How good do you think your knowledge is of the different practices and cultures in each Government Department?
Clem Brohier: It is increasingly better as we work more closely with the Government Departments. There are two mains reasons we are working even more closely with Government Departments. One is the transition from 30 to 20 years, and also the increasing volume of digital information, which will increase as we go through the next five years.
Q13 Chair: How different is the character of records management and record keeping between the different Government Departments?
Clem Brohier: It does vary, and of course the work of each Department varies. As we go through the questions, we will probably surface some of the challenges that we face, but it is variable across Government Departments. On the whole, we are making very good progress in transferring records to the National Archives.
Q14 Chair: In your mind’s eye, do you have a hierarchy from the best to the worst? I shall not ask you to name the worst.
Clem Brohier: We can see good practice and we can see areas for development. We do actively encourage. We publish guidance and, also, when we conduct assessments of how record keeping is undertaken, we do publish that good practice.
Q15 Chair: Who audits the quality of record keeping across Departments?
Clem Brohier: It is done in a number of ways. We offer an additional information management assessment service. It is voluntary; we have to be invited into Departments. It is an assessment of how record keeping is undertaken.
Q16 Chair: Is this published?
Clem Brohier: It is published. Departments themselves will have other forms of audit and check.
Q17 Chair: Is there a document I can go to that perhaps shows green, amber and red, in terms of the different performance?
Clem Brohier: Yes.
Q18 Chair: So you can tell me publicly which are the good boys and which are the bad boys.
Carol Tullo: It is more based on getting an overall picture of the capability of the Departments. As Clem says, it is quite risk‑based, because it is looking at good information management, but it is also looking at their technology platform. Do they value information within the Department? Is there a culture of valuing it and appreciating what it does? Every assessment of every Department—we have done over 20 of them in the last six years—is published and then there is a follow‑up programme, because it is not meant to be a one‑off. It is meant to then shape and help and work with them, in that supporting role that we have, to drive up good practice.
Q19 Lindsay Roy: From your experience, how well is the code of conduct adhered to?
Clem Brohier: It is variable, and what we can see is that currently high priority is given to compliance. Record keeping is something that we have to continually promote.
Q20 Lindsay Roy: How frequently does that happen?
Clem Brohier: It is pretty regular. We have a separate team that works very closely with the Government Departments to ensure that there is good practice and it is as consistent as we can make it across Departments.
Q21 Lindsay Roy: There is some sort of action plan to pursue.
Clem Brohier: There are separate action plans for the Departments that we work more closely with, yes.
Q22 Mrs Gillan: At what level within the Departments does the triaging of documents take place to decide whether they are discarded, kept or ultimately transferred to you? Who makes those decisions and who checks those decisions?
Simon Marsh: Within the Ministry of Defence, we have a separate records review team of about 20 people. There is a grade 7 who runs that team and, within that, it is at HEO—higher executive officer—level that the decisions are being taken about whether to destroy or keep a file, although there is a discussion that happens with the information management consultant, who comes from the National Archives, about whether it is appropriate to destroy that or not.
In certain circumstances, particularly where we have perhaps legal holds on types of records, particularly from operational theatres, for example, there will be a separate discussion with the Directorate of Judicial Engagement Policy as well, about whether it is appropriate, say, to get rid of some finance information relating to that theatre, as opposed to some operational information.
Mrs Gillan: Mr McPherson, have you got a comment on that?
Scott McPherson: No. I was just suggesting that Simon was the person best placed to answer it.
Q23 Mrs Gillan: It seems to me that a lot of these decisions are taken at quite a low level within the Civil Service—at HEO. Does it ever go up for arbitration to the permanent secretary on records or is it always kept down at HEO grade-7 level?
Carol Tullo: You have to remember that the process and the approach that is taken is that Government organisations are the best‑placed body to understand their own information. It will be delegated down to the policy teams, specialist agencies and groups to make that initial appraisal over what should be kept and what should not be kept, and then they will follow a selection policy; they will work with the experts, and the National Archives will advise and guide them; but the decisions will be very much for the creator of the record. It might be a director within a policy team, who will have decided, when a transaction is completed—perhaps a report or an annual report is published—what should be kept. Obviously there are ephemeral exchanges for meetings and detail, but the actual record that needs to come to the National Archives is what we are interested in: preserving the history and the audit trail of that decision.
Q24 Mrs Gillan: Is it not true to say that, now, with the technology that is available, there is the capability of capturing what currently may be considered to be ephemeral, but may in the future be considered to be substantive in some way?
Carol Tullo: I agree. It is one of the interesting stages. We are two years into this transition of moving from 30 to 20 years, but we are still largely in a paper world. We are in the paper world of 1985 and 1986 this year, of the records that are coming through to the National Archives. As we go through to 2022, when this 10‑year transition will have finished, we will then be looking at records only 20 years old. In 2002—even then—Governments and Government Departments were only just adopting intranets and storing data digitally and electronically.
I would agree with you that it is much easier. What we find in practice is that people just store everything. They will store it in a local shared drive; they will store it in a collaborative drive. They will then make a decision on what to keep. Every Department will have very clear instructions and they will work through what they should keep, but the corollary of that is what you do not keep: what you dispose of and what you decide is not worthy of keeping for the record. That is a decision that is made within the various sections of a Government organisation.
Q25 Mrs Gillan: I will tell you why I am interested in this. I asked what the annual budget was of the National Archives and, I gather from your 2013‑14 annual report, it was £37.1 million. That means our record keeping, archives and retention of documents is entirely driven by the budget. Do you think that that is right or do you think that it is sufficiently important in this day and age, particularly because we now have the capability of capturing what you term “ephemeral”, which I maintain could actually be important in the future? Do you think that we should be budget‑driven in this area or rather should we let the amount of information that is coming out of each of the Departments drive it?
Carol Tullo: I wouldn’t say we were budget‑driven. Remember, if we go in and assess the overall picture of a Department, we will be looking at a number of issues. The technology platform that they use is really driving the capture of more information now, because there are not the same storage issues. There are different challenges—obsolescence, different platforms—but in fact it is quite intriguing to see that probably more data and more information will be kept in future, because everyone will assume that the tools will allow us to retrieve the data and retrieve the information. It is not budget‑driven in the sense that, actually, the storage of digital data creates opportunities to capture more data than it did when we had to have lots of clerks filing paper and annotating paper.
The National Archives’ budget is much wider. That is not all just for record keeping. It is our primary purpose. Imagine that replicated across, we would say, probably the top 21 transferring Departments, of which the top five transfer probably 60% to 70% of the material that we take in every year. The top 21 transfer almost the whole 100%. If you think about the hundreds and hundreds of record bodies there are and the many Government bodies there are, you can see that we will focus our resources on supporting Departments like the Ministry of Defence, the Ministry of Justice, the Cabinet Office and Treasury, because they are the ones where there is more interest in the records and they are the ones that have the volume. They will invest a lot of money themselves.
Simon Marsh: It is worth adding as well that there has been an exponential growth in the amount of information that the Departments are creating, with the advent of technology. Traditionally the MOD, for example, has transferred about 3% of the records that it creates but, if you scale that up, because we are creating more of it now, then more will be going to the National Archives. If you take a view that you might want to save everything, which would be one approach, then you run into challenges about search and how easy it is to index all this material, and those sorts of issues.
Q26 Mrs Gillan: You have taken me on to my next point, because I was interested; I put in an FOI request to the Department for Transport back in October, and it took them until last month to produce the information. How searchable and how accessible are the records that are held by the National Archives, and which Departments make use of them, or who makes use of them the most?
Simon Marsh: I can talk about what the MOD does. We are recalling files from the National Archives all the time, because we are using them for our current business. We have three of the service historical branches, our defence, science and technology laboratory, and some of the service warfare centres that are always looking back at what we have done in the past as an organisation, and reusing that information. Equally, we are identifying, particularly with our operational information now, what we are going to transfer in 15 years’ time, and we are deliberately saving that now and using that within the Department as well.
Clem Brohier: I would add that probably those who use it the most are, interestingly, the Departments with the historians. It is MOD and the Foreign and Commonwealth Office. We do encourage those civil servants to come over to The National Archives to view the records so that they actually are still in our possession and they are accessible to everybody.
Q27 Mrs Gillan: How do you relate to other archives, for example the Bodleian or the National Library of Wales, for example? Do you de‑duplicate with them or do you know that they are keeping archives that are completely separate from yours?
Clem Brohier: You touched on budgets earlier and, in relation to that, we are all operating within the same environment, focused on our core activities. That has called for even closer collaboration between other archives, museums, libraries and galleries to ensure that what we are doing is our core delivery, but also co‑ordinated right across other archives, so that there is no or limited duplication. We work pretty closely with them. In addition, we offer archive sector leadership to archives across England and Wales.
Carol Tullo: It is true to say that no one else is collecting the official record of Her Majesty’s Government in England and, at present, in Wales. There is PRONI, the Public Record of Northern Ireland, and there is the National Records of Scotland. The archives at Government level work very closely together, because obviously we have a United Kingdom interest, but we will look at local areas. No-one else is charged with our public task, which is to preserve and safeguard the nation’s record. That is unique.
Q28 Mr Evans: When do you go down from 30 to 20 years?
Clem Brohier: We are in the second year, so 2013 was the first year of the 10‑year transition, and we took in 1983 and 1984. This year, it is 1985 and 1986.
Q29 Mr Evans: You are doing it in two years. I was wondering whether there was going to be a 10‑year splurge when people would get it all at once, but it is not happening that way.
Clem Brohier: That 10‑year transition enables Departments to prioritise, review, select, assess and then transfer records to us. It enables us to take it in the centre in a pretty ordered way, phased over the 10‑year period.
Q30 Mr Evans: Even then at the 20‑year period there is still stuff that will not be made public.
Carol Tullo: There will be, because it can be transferred to us open although, in exceptional circumstances, it can be transferred to us closed, usually for national security purposes or personal data purposes. Generally speaking, the amount of material that comes to us would all, eventually, be opened. That is the whole point of preserving it; it will be made accessible.
Q31 Mr Evans: I got this image that, when it was the old 30‑year rule, it would be a load of people heading towards Kew and basically looking through loads of papers, finding out the most interesting titbits, so the Daily Mail could have a field day.
Carol Tullo: Yes, that happens still.
Mr Evans: It still happens that way.
Carol Tullo: Yes, it does. We have two big releases a year. It always used to be 1 January. You would open the newspaper and you would see all the big stories. That still happens, and we do another one around the middle of the year. I can give you a statistic because, last year, 2013, doubling the year, we took in just under 70,000 records. The previous year it was just over 33,000, so you can see that, for transparency, it is opening up sooner a lot more material.
Q32 Mr Evans: When you say “a record”, what does that mean?
Carol Tullo: That is an interesting question as well. How long is a piece of string? It can be a small file but, generally speaking, what we mean by that is a sizable record. We are still talking paper here, but we also do measure meterage as well, but people find it is more compelling for the story to talk about the individual record. Most people have researched a record, created one and looked at one. They can imagine a record box.
Q33 Mr Evans: Looking through all that paperwork must be like trying to find a needle in a haystack. Is everything not being digitised now, so that you can cross‑search things? Even if the paperwork was from the 1980s—I became an MP in 1992—computers were still relatively new in 1992, so I can only imagine the amazing or immense amount of work needed to digitise it all. I assume that is happening.
Clem Brohier: It is a very expensive task and of that £37 million, last year, £7 million was to do with commercial activity. We cannot afford to digitise all our records, so we enter into commercial arrangements with other interested organisations, whereby they will digitise certain record collections for us and we will give them exclusive rights to that information for a limited period. However, it is always available at the National Archives for free.
Q34 Mr Evans: In hard copy?
Clem Brohier: In hard copy or digital.
Q35 Mr Evans: I assume, because you have loads of paperwork at Kew and then, somewhere in a salt mine—is it in Cheshire?
Carol Tullo: Yes, Cheshire.
Q36 Mr Evans: Anywhere else?
Clem Brohier: No, just those two.
Carol Tullo: For various reasons, there are times when it is more appropriate to hold official Government records locally—court records, hospital records, for example—so there are places of deposit that we supervise that will be disaggregated around the United Kingdom.
Q37 Mr Evans: You do not have a place in each country either, under the Foreign and Commonwealth Office, even the overseas territories, where records are kept?
Carol Tullo: No, the Foreign and Commonwealth Office will have local offices. In fact, we have just done a re-assessment of the Foreign and Commonwealth Office’s capability. It finished on 3 July, in fact. Interestingly, in a bit of good practice for the Foreign Office, they really insisted that our assessment team go over to one of the embassies to see what the local record keeping is. It will all come back here; it will all go into the National Archives if selected, but they wanted to see and test the capability of how it is being held locally. Our team actually went out to Bucharest, I think it was. That was unique as well.
Q38 Mr Evans: If I turn up at Kew because I want to find out a bit of information from 1980 about a Government Department, I sit at a desk and you come up with a load of files. How does it work?
Carol Tullo: Search Discovery, the database, perhaps in advance if it is something that is not commonly used, because we might need to retrieve it, but all the common records, all the regular records and even a lot that are not looked at regularly, are kept in the repositories in Kew. You will get it within a very short period of time. It will be delivered personally to a box, and you are free to go into the reading room and look at it.
Q39 Mr Evans: I am just trying to get my head around what size this all is.
Carol Tullo: It is vast.
Q40 Mr Evans: Each year, how many bits of paper are you being given, in this paperless society we now live in?
Carol Tullo: Remember, we are talking about the historic record. They are business assets in 1983 and 1984, but they have only just come to the National Archives and been opened to the public. In 1983 and 1984, we were a paper world; we were a world of clerks. You know those old brown folders? If you opened them and wanted to take a piece of paper out, you had to note that you had removed a paper or made an addition. The world has moved on. I think I know that some of you are going to come and visit but, until you see the scale of it, you cannot really imagine how much the nation stores and safeguards. It is a phenomenal amount.
Q41 Mr Evans: What is the cost to you of somebody walking in and saying, “I want a bit of information”?
Clem Brohier: It varies depending on the type of information they are asking for. If it is some information that is going to be searched and copied, there is a charge that we make under the statutory instruments to recover just the cost of doing that, so it does vary, and part of that will depend on the media that you are requesting, whether it is paper or electronic records. In the main though, we provide information for free.
Q42 Mr Evans: Right, and how often, without going along these missing files, do you say to somebody, “I am terribly sorry; we can’t find that”?
Clem Brohier: Readers will come to the National Archives and they will search the catalogue, so that question would probably be answered by way of search. Just to give it some context, and I am talking very much about averages, we take about 5% of the records relating to Government Departments, on average. That varies from Department to Department. For example, we will take about 40% of the Foreign and Commonwealth Office’s information, because that is information that historians would be interested in. For other Departments, it is a lot less.
Q43 Mrs Gillan: When it comes to you, are you the only repository of that or does the Department still, in some instances, keep those records or copies of those records?
Carol Tullo: The materials should be transferred to us at what was 30‑year, now of course it is the 28‑ and 27‑year, rule. The legislation requires it to come to the National Archives. We do take material in advance. If you think about material, we have to safeguard it. If you think about big inquiries, for example, they are mostly digital, but they have a shelf life generally speaking, so we are taking in digital material now. The Olympics was digital material. We are taking in more recent material that is still required for business purposes, it is open to a Department to approach the Lord Chancellor's advisory committee and get permission from the Lord Chancellor to retain that, but they have to make their case to this committee to hold it for specific purposes. There are a number of retentions that are approved every year, and they are published on the National Archives' online catalogue and the minutes of the advisory council, and they have to make a very good case, because obviously that is almost conflicting with bringing it in under the legislation and making it available to the public.
Q44 Lindsay Roy: Can you give us some idea of the criteria used for retention and rejection or disposal?
Carol Tullo: We can talk about the MOD. Across all Government Departments, remember I said that it is like any organisation: those closest to managing information know it. We have a team of about 30 who work closely in partnership with every Government body, every public records body. We focus, as I said earlier, on the top five and the top 21 for obvious reasons, but we are there to provide advice. Tailored to the needs of each Department or agency will be an operational selection policy, and they are all published. They will follow those, but they will have local requirements that suit their materials that they will follow. That is not something we would see. We would certainly advise and approve that it was meeting good practice but, when you get down to the granular detail of whether a file should be kept or not, our advice would very much be to give proper filenames to the files.
One of the issues that we found was perhaps record keeping has moved and we have got a bit more sophisticated and mature about how we store our information. If you cannot tell what it says on the file—if it is just a series of numbers and slashes—then it is going to be very hard to follow the audit trail behind it. That is one of the things we are very hot on.
Q45 Chair: Basically you are saying it is the Departments that choose what to keep and what to throw away.
Carol Tullo: They do, within very strong parameters.
Q46 Lindsay Roy: Are there overarching criteria?
Carol Tullo: Yes.
Simon Marsh: Perhaps if I can explain what we do in the MOD, that might give you an idea. We produce an appraisal report, which is essentially saying what records we think we should be keeping for permanent preservation at the National Archives. We discussed that with the National Archives to make sure that they are happy with that, and then it has been out on public consultation for a 12‑week period. We got two public comments on that appraisal report, and we are now just finalising it and about to publish it, so we are being transparent about the information that we are intending to preserve historically.
Carol Tullo: If I answer your question very specifically, you asked about criteria. There are two at a very high level. One is that it has the potential to be of interest historically and the second one is that it has a business requirement; the Government should store that. There is no point in recreating the wheel. If you are doing something again, even if it is in 20 years’ time, we want to be able to provide to Government and to the civil servants those records.
Chair: Let us move on to the MOJ.
Q47 Mr Turner: Mr McPherson, what is your assessment of historical record keeping practice in the MOJ?
Scott McPherson: I should be clear that I am not the departmental records officer responsible for it. My role is advising the Lord Chancellor on the overall framework of the Public Records Act and Freedom of Information Act.
Chair: Then you are in a very good position to answer that question.
Scott McPherson: My understanding is that it is similar to other Departments. We, like other Departments, have had an information management assessment done by the National Archives, which highlighted areas of good practice and highlighted areas where we could make improvements. A bit like you have heard from the Ministry of Defence, there are some professional knowledge and information management staff who are responsible for the records management, and then we have a senior information risk owner at board level, who owns the overall framework and makes sure that is taken seriously by officials throughout the Department.
Q48 Mr Turner: Ten are quite good, 10 are not very good and five are pretty excellent, or what are the figures?
Scott McPherson: The figures for what, sorry?
Mr Turner: How well they are rated by you.
Scott McPherson: We had an information management assessment by the National Archives in 2009.
Chair: There are not many green lights on that.
Scott McPherson: It had some greens; it had some amber/greens and some ambers, yes.
Q49 Mr Turner: How many is “some”?
Scott McPherson: I can look up the exact numbers for you.
Carol Tullo: I can tell you that there are 17 out of 24 at the latest action review that have improved and are being worked on. It is interesting, because it is not a zero‑sum game; it is not a tick list. The whole point is about working with them to meet their local challenges and needs, and MOJ, from our position, has a particular challenge, because it is also responsible for courts and prisons, so it has a lot of records that go outside. There are some areas where it is about creating a personal accountability for the record, for each individual official, but it is important that it is driven at a senior level. In MOJ, part of one of the areas for development was that there was not a great awareness, right across the organisation, that this was important. The culture of that was very important for MOJ.
I know that something that was really excellent practice, where a couple of other Departments have copied MOJ, is that they had a history day, fronted by the Minister, where it was hands‑on. People could look at a range of records that we brought from the National Archives, and they could actually see some really quite famous records that you would not normally be able to see. That was trying to instil in people the importance of a mature understanding that these are the business assets of the organisation. Unless you are preserving them now, they will not be available for the future.
Q50 Mr Turner: Yes, I understand that. I am trying to find out from Mr McPherson what the key deficiencies in his Department are.
Scott McPherson: The latest report was from July 2012. Seven areas were rated amber/green; 10 were rated amber; and three were rated amber/red. The amber/red areas related to resourcing, storage and sustainability of digital records. As I said, I am not responsible for the implementation of that. We have an action plan that we have agreed following that report, which some other officials in the Ministry of Justice are responsible for implementing.
I know that, since the report has been done, further work has been done to improve our records management. In particular, our knowledge and information management officials are working more closely with our information technology team, given what Carol was saying earlier about the increasing importance of digital records. We have done some training for the staff around what to keep; we have a guide, referring to some of the earlier questions, that sets out which records should be preserved for the public record. I know our executive committee, which is the senior officials in the Department, regularly discusses our records management. Most recently, earlier this year, they were talking about a replacement for our electronic data and records management system, which was one of the recommendations in the report. Like other Departments, we recognise that it is not perfect, but we are taking quite a lot of action to try to improve things where necessary.
Q51 Mr Turner: To what extent are these deficiencies in the MOJ repeated in other Government Departments?
Scott McPherson: I understand that, as a result of the information management assessment programme that the National Archives is doing, they are currently in the process of pulling together exactly that sort of overview of what lessons can be drawn from all the reports that have been done.
Carol Tullo: There are some key trends that come through. We are always looking at why we are doing this. It is to give us a picture of what record keeping and good effective information management is across Government. We have a responsibility; we want to ensure that the researchers and the civil servants of the future can actually track what happened in the period of the 1980s, the 1990s and more recently.
Some of the trends that come through are ones that, you will appreciate, depend on other demands within the Department. One overall demand that has been a real challenge has been the move to 20 years. A decision was made that that would be good for openness, transparency and the record but, of course, that has doubled the burden on Departments. The negative effect has been a pressure on resourcing in Departments. The benefit has been that we have had to start tracking and monitoring progress much more carefully, so we now have a lot of statistics about what records are held in Departments that we never had before. There is a discipline. We have a report, the record transfer report, that is published twice a year. It is very new, first published in 2012. That has been a great spotlight on Departments’ performance.
Really, we are not here to say who is good and who is bad; we are here to say, transparently, “Here are the statistics.” You can look at that report and you can see how much a Department is transferring, and how much legacy and backlog there is. This new move from 30 to 20 years caused Departments to put a resource into actually identifying what they were storing, what they were doing and where they were holding material information. We have a lot of statistics now that we did not have, so we are very rigorous in being able to record that data, but that does show up deficiencies and it does show up areas where we are driving and working with the senior responsible people in Departments to say, “Please unlock resourcing. Please give us that reassurance, as the National Archive, that we are meeting our public task and safeguarding the record.” There is a balance; they cannot do everything.
Q52 Mr Turner: My last question, in a way, is: we are now transferring it to 20 years. Before 20 years, Mr McPherson’s part of the Department will be relying on the MOJ people making the right decisions there. That is correct, is it?
Scott McPherson: Within the Department, there is a schedule that sets out the time periods that various records should be held for, because obviously the Ministry of Justice has a wide range of records, from court records through to policy records, and so the time periods that are appropriate for those vary. We have a document that sets out the time periods that they should be held for. We also have guidance for officials, as I said earlier, called What to keep, which suggests, depending on the nature of the information, whether it should be preserved. If you were simply organising a meeting, it probably does not need to be. If it is advice to ministers on a new policy, it should be. We give training to officials so that they understand that. We then have a team a bit like Simon’s team, which is full of professional knowledge and information management experts, who take the ultimate decisions about the closure or transfer of the files, as Simon was talking about earlier for the Ministry of Defence.
Q53 Lindsay Roy: Can we move on to Departments approaching you to do assessments of their work? How often does that happen?
Carol Tullo: This is for when we would go in to audit and assess their capability.
Lindsay Roy: When they actually approach you for assessments.
Carol Tullo: They do not approach us. It is interesting: we are invited in, but we do use triggers to look at their work. The reason we reassessed the Foreign Office at this particular time is clearly because of all the issues over the last 18 months about their legacy issues.
Q54 Lindsay Roy: Let me just clarify: you are invited in or you can invite yourself in?
Carol Tullo: It is a nuanced point. If you were asking what would give us a bit more teeth, it would be the ability to just go in; that would be very helpful, but that has not been our role. To be fair to Departments, assessing the risk in managing information is also very substantially the reputational risk, because nobody wants not to run a good ship. We have found it is very easy, in fact, to be able to go through and work with a Department. There are some that have been a little unwilling and it is often that there is never a good time. There is never a good time to have an audit or an assessment, so we have had to push sometimes. We have two big Departments—Health in October and DWP in February next year. By the end of our year 2014‑15, we will have covered all the major Departments bar one, and really that was our opportunity to be able to present a complete overall picture.
Q55 Lindsay Roy: Can you tell us which one you will not have covered?
Carol Tullo: The Home Office.
Q56 Lindsay Roy: Why do you think that is?
Carol Tullo: We piloted the Home Office, and it is vast. It is very big. It was one of our pilots going back into 2006‑07. We then decided that because it is so vast and because it has very discrete areas, we would work with their agencies. We did an information management assessment of the then UK Borders Agency. It has moved on. We have also done some of the other areas of the Home Office.
The central area of the Home Office chose to use a different information maturity model that was tailored, they felt, to what they required, which was outside our rigour and our overview. The benefit of the focus and spotlight on discussions like this is that the Home Office is very keen, and Simon Marsh is moving to the Home Office in August. We look forward to working very closely with Simon to bring them into our system.
Scott McPherson: Carol said that this programme of information management assessments is complete in nearly all of the Government Departments, except the Home Office and a number of the smaller Departments, but all of those Departments are now absolutely clear that they see value in this process, and they all do intend to go through an information management assessment in a time scale that will be agreed with the National Archives, including the Home Office.
Clem Brohier: I would add that, although there is the assessment, it is also part of a rolling programme. The assessment will culminate in an action plan and a set of recommendations. We will then go back and ensure or support the implementation of those recommendations.
Q57 Lindsay Roy: How long ago was the pilot and when do you envisage the Home Office coming into this scheme?
Carol Tullo: It will be quite interesting. I should imagine that within the next 12 months we will cover the Home Office. At the moment, we run about four information management assessments a year, and that is how we plan them. They are intensive, but they range from a really big one like the Department of Health would be. Of course we have looked at the Department of Health, also in 2006, in the pilots, so we will be able to go back and track the differences and the changes.
Q58 Lindsay Roy: We are talking about eight or nine years.
Carol Tullo: Yes but, as Clem said, it is a rolling programme. Once you have an information management assessment, you might do an assessment-lite and do a reassessment. We have just done that with the Foreign Office, but that was two weeks of intensive interviews. We might then decide that it is going very well; you have worked through the action plan; you have worked through the regular reporting and tracking. Remember, this is running in parallel with what our information management specialists are doing day-to-day. Every day, they are in the Department working, perhaps at a more operational level, but there is that constant loop back into what is good practice.
Q59 Lindsay Roy: There are global assessments on a three‑point scale, is that right—good, adequate and room for improvement?
Carol Tullo: Good, satisfactory and development needed.
Q60 Lindsay Roy: Just a little issue about the “development needed”: no matter how good further development could be encouraged and fostered, what evidence do you have that recommendations are acted on?
Carol Tullo: We have evidence, because we know what happens next time we track them. Remember this is all public and published. To what extent it is visible within a Department is something that we constantly work with, and we are very pleased to see, at a very senior level, real interest in using these tools and using this work. Driving awareness in the Department is very different from noting awareness, so I suppose for us making sure of the importance of effective information management underpinning the business of your Department is something we constantly do.
Q61 Lindsay Roy: Effectively, you are monitoring what is happening.
Carol Tullo: We are, within the Department. It is part of our role to do that.
Lindsay Roy: There is a continuing dialogue about improvement.
Carol Tullo: Yes.
Simon Marsh: It is probably worth saying that you monitor the implementation of the action plan that the Department is required to create after an information management assessment. There are checkpoints within that process, where you actually ask for written evidence, their being able to show something that they have done, in order to convince you that they are on track and are implementing your recommendations.
Carol Tullo: We are not always convinced. That is an issue.
Q62 Lindsay Roy: How often have you been asked to provide training for information management systems?
Carol Tullo: A lot.
Lindsay Roy: Is that regular?
Carol Tullo: Yes. We have a number of programmes that we do, whereby we will very much support every Department if they are running training sessions internally. Remember, the tailoring for Departments’ needs is very important. I have spoken at events. I have managers and our specialist advisers always working with Departments. At the centre, what we do also is we bring people in. We bring information managers in four times a year to Kew. There may be 100‑150 there, and we have a theme. Information security and assurance was the last one, recently in the last month. We will have a theme on digital preservation: what should you be doing now to think ahead about what you are doing with your electronic records? We will have external speakers and a selection of our own speakers. There is also a leaders group, which works with the heads of knowledge and information management across Departments.
Q63 Lindsay Roy: Finally, what evidence do you have that best practice is being adopted?
Carol Tullo: I would say we have very little evidence that best practice is being adopted, but a lot of evidence that good practice is. It is very hard to say what is best practice, because of the very distinctive nature of records. An MOJ court record or a prison record, with a lot of personal data, is very different from a Cabinet Office tracking of a major inquiry. You require different skills. What we do is draw together those examples of good practice and we use those, because an assessment is also a dialogue. It is a conversation: “Why aren’t you doing this? Have you talked to DECC? Have you talked to DEFRA?” We keep those examples and we work with them. There are some really excellent bits of initiative, but everybody is running at a slightly different pace, depending on the pressures that they have.
Q64 Lindsay Roy: There are some generic questions, for example, that are posed.
Carol Tullo: Yes. There is a whole methodology with generic questions that are posed.
Q65 Paul Flynn: Is the fact that you have not been invited into the Home Office connected, do you think, to the practice of the Home Office of destroying files without taking a record of their contents? You might have seen the evidence by Mr Mark Sedwill, who said that he had no idea where 114 files had gone. He had no idea of their contents, who removed them or why they were removed. You know the significance of the files; they were part of evidence admitted by our late colleague, Geoffrey Dickens, on allegations about paedophiles in high office, a number of whom he exposed. Is it significant that the Home Office is the only Department you have not investigated?
Carol Tullo: I would say it is not significant, because I would differentiate the rolling process of audit and assessment that we do from the regular day-to-day business‑as‑usual work of experts in the National Archives, and the Home Office dedicated team of three—three people will have a range of Departments, but they work with the Home Office on their practice and their procedures—will be working closely with the Home Office.
The issue that you are referring to is very difficult, is it not, because it is a judgment issue? We have talked about the naming convention for a file. The practice is that, if there is any disposal, there should be an audit trail so that, when questions like this come up, you know what the file contained and when it was destroyed. You might not know exactly what was in it, but you would know the content of it. We are talking about a human judgment. We are talking about a discretion and a skill that should sit with each organisation that is responsible for its records, but we are also trying to remember what happened decades ago, when nobody there can actually recall it.
The best way to give that confidence and reassurance that this is not endemic is to show and point to the disciplines, the rigour, and the skill and knowledge that goes through other Departments. This is a particular example, but we know on our files you can search on the National Archives’ Discovery and you can find records of files. You will find that they have been disposed of, but you will find that they were disposed of at a certain date, so there is this audit trail. Somewhere along the line, the mechanics clearly did not work effectively.
Q66 Paul Flynn: You would take a hard line on the practice of destroying files by unknown people without providing a reason why they were destroying them, would you not?
Carol Tullo: We want a clear audit trail. Remember disposal—destruction sounds really strong—is sound and effective information management. You select and appraise what you keep. We know we want to preserve the important things. All sorts of nuggets are preserved, and thank goodness they were. Do you remember the file about Margaret Thatcher’s ironing board was preserved? There are all sorts of things, but that would have been somebody’s judgment call at the time, and it is very hard to second‑guess that now.
Q67 Paul Flynn: I know that book very well. I did a review of that book about Margaret Thatcher’s ironing board. It is not a trivial matter, unfortunately. We know that there have been bonfires of huge numbers of files by the Metropolitan Police. We know that the murder of Daniel Morgan, 27 years ago, has never been solved, but we do know that he was investigating police corruption. There is evidence of huge collaboration between various groups in London and the police, generally through an unusual body—a brotherhood. There are terrifying reports that are confidential, one was leaked recently to the Independent, showing that there is information that is dynamite, and we have evidence that large amounts of it are being destroyed. What mechanism do you think we could have? The other indication is an incident like Hillsborough, where information was suppressed over the years. Do we need some gold standard; do we need a set of rules that says that certain information has to be preserved and cannot be destroyed without a full and open investigation of what is going on?
Carol Tullo: I see what you are driving at and, whether it is a gold standard, a platinum standard or a silver or bronze standard, those rules are there. The issue with the Met Police is that they are not public records in exactly the same way as Home Office records are. It is very difficult to imagine. You would have to ask them about the content. What we are doing on behalf of the Government, the reason why you have a national archive, is ensuring that we can assess and set standards that reasonable people will follow and that are appropriate. We can never cater for the judgment call that goes on in Departments.
Q68 Chair: I think you are saying something very important, which is that basically the Metropolitan Police is not set the same standard for record keeping as a Government Department.
Carol Tullo: We would not know, but there are discussions about bringing police records into the public record system, partly because of the some of the recent issues that continue. We can speak for the public record, and the public record is very much defined as the records of Government that you need to keep to track what is going on. There are all sorts of public record bodies that you would never imagine were public record bodies.
Q69 Chair: Is the malevolent destruction of records a criminal offence?
Carol Tullo: No, I am not sure it is. We can only speak for the Public Records Act and, within the Public Records Act, disposal of records following review and appraisal by the professional skills within an organisation is perfectly acceptable, but we will always look at that.
Q70 Chair: If somebody goes to a filing cabinet to get rid of some stuff that they do not like and they do not want to continue in existence, and they get rid of it—they put it through the shredder—is that a criminal offence?
Carol Tullo: Then you are in the area of intention, so you do move towards a criminal offence.
Chair: A common law offence?
Carol Tullo: The other issue is the personal responsibility on officials, on civil servants, through the Civil Service Code.
Chair: The Civil Service Code has no teeth, not like statute.
Carol Tullo: I suppose if you take your possibly apocryphal example further, Mr Chairman, you would have to then say that somebody would have to see that individual, somebody would have to be able to report it and somebody would actually be able to track it. What I am saying is that, for the National Archives, we are putting in place the standards to enable that tracking to happen. We cannot say what used to happen decades ago, but we can say what happens now.
Q71 Chair: What we are concerned about is that there has been sleight of hand. “We have to get rid of stuff; it would be quite convenient to get rid of this stuff.” Leaving the police out of it for the moment, do you think that happens in government—that people get rid of inconvenient stuff while they are cleaning out, having their five‑year review or their 25‑year review?
Carol Tullo: If anything, the contrary, because, if that was going to be the case, look at all the Foreign Office migrated archive colonial records that emerged. I spoke to a very senior sensitivity reviewer at the Foreign Office recently.
Chair: I am thinking more about now, when we have so much stuff.
Carol Tullo: Yes, but the sensitivity reviewer said, “I was flabbergasted to hear that we had all these records, because I worked very close to them for many years, doing my job as a sensitivity reviewer, and I used to say, ‘What’s in all those files?’ I was told, ‘They’re some sort of administrative records. We’re not sure.’” Again, in the focus and spotlight of the 30 to 20 years, capturing the data and the statistics, and saying, “We are comparing you now. We are publishing, Department by Department, what those records are,” suddenly lots of records have emerged. That is the counterpoint to your point in that, if people were going to be embarrassed by this, on your analogy, people might go around and just say, “Let’s just lose those, shred them or destroy them.” That is actually not what we have seen. We have seen massive legacy backlogs, which to me are a real positive strength. They are a lot of work for Departments, but they have actually gone through and uncovered a lot of records.
Q72 Chair: That was 30 years ago or 50 years ago. What about now?
Carol Tullo: No, it is current. It is current.
Q73 Mrs Gillan: Would it be fair to say that it is much more plausible that someone would destroy the file at the time, because the further that time elapses and you move away, they move away from those areas of influence and the people who are coming in behind have no idea what those files contained? It is more likely to have happened on a contemporary basis then it is ever on a historical basis.
Carol Tullo: I could not comment on that. That is a point.
Simon Marsh: From an MOD perspective, that is very true, and also because we tend to move the files, the paper files anyway, to a central archive, and they are therefore out of the control of the people who originally created that information.
Q74 Mrs Gillan: Is it not also that the chances are that, when you have some sensitive files like this, there may be other repositories of that information? I am just thinking from my own experience, because I used to be responsible for List 99 in the Department for Education and Employment, which contained police records and an assessment of people involved in the teaching profession, so I would have a full record in the Department for Education. If I decided as the Minister that that person should not be placed on List 99, we would still have that record, but of course that record may not be deemed as important as the ones for those that were placed on List 99, and subsequently then on a sex offenders register. It is quite possible that there are also other sources of this material in other Departments. It is a question of searching to see whether those things may be there.
Carol Tullo: That happens. That does happen. Where you have big events, like for example the Falklands, when all those records were being released, Departments did get together to see if there was duplication and to make sure there was a fuller record and an integrated record, which was made available to the public. You are right: there will be duplication. Those are the little nuggets that the researchers love to find that somebody kept. Remember the judgment call: if somebody kept your List 99 or your non‑List 99, whereas others did not. We are not culling. We are not saying, “Keep a particular document,” but we are recognising that the judgment call will be different. It will be personal. It is a bit like an index for a book. How often do you look at the index for a book and you cannot find what you want, but you know it is there? Somebody else’s mind has assessed, appraised and valued it. It is very different from saying, “This is dynamite; I am going to destroy this.” That is not the business that we are in; we are in making sure that there is a proper trail.
Q75 Greg Mulholland: Just to probe slightly further, you are well aware that there is huge concern among the public and huge suspicion about this. The reality is that all we know is that 114 files are missing. We do not who removed them or destroyed them. Indeed, we do not even know if they were actually destroyed. We do not know who did it and we do not know when that happened. The question that the public want to get an answer to is: how can that possibly have been the case if there was not foul play involved? How can there not have been simple effective processes for when official documentation is destroyed or removed, and how can there not be a proper log of when that was done and by whom?
Carol Tullo: We would like to know that as well. That is why there are those inquiries taking place. We will be as interested. Remember I said earlier that the triggers that we look for to establish what is effective information management will be covered by the output of those inquiries.
Q76 Chair: Is there not some kind of failure in your oversight of this, or do you not have the authority to oversee this properly? Maybe you are making changes today or you have made changes since that time that would prevent that kind of failure occurring again. Which of those is it?
Carol Tullo: As an organisation, we have thought about this quite a lot, especially in the last 18 months to two years, with a number of big issues coming through. It is really difficult to understand and put yourselves in the mindset of the clerk or the filing clerk system of many decades ago. We do concentrate on today’s records. We are looking at the currency of what is being created now to make sure nothing is not kept that should be and, of course, the records of today that are being created in 2014 in 2034 will be the historic record.
The decisions have already been made about those records. It is intriguing to know what the rules were. If anything, I personally am very surprised, because the Grigg system that led to the 1958 Public Records Act was very formalised and very rigid, and that was in operation in Government Departments well into the 1970s and 1980s. It is a little surprising, particularly with a structure in Departments where you probably had quite a lot of business support and administrative support handling filing, that these things did happen. We cannot say; it is the inquiries that will tell us.
Q77 Greg Mulholland: From what you are saying, can I ask you very briefly a simple question? Do you believe that procedures were wholly inadequate or do you believe there was a breach of procedure in this case?
Carol Tullo: It is more likely, from our own experience, to be an error in applying those or perhaps a lack of rigour in meeting the set requirements.
Q78 Greg Mulholland: I am not asking you for the motive. We cannot assess the motivation inherent in this, but do you believe it was a breach of the procedure? We do not know the motivation for that, but was there adequate procedure in place or not, or was it actually that someone, whoever, for reasons about which we can only speculate—and that is not helpful in this context—did not follow proper procedure and breached those procedures that were in place?
Carol Tullo: There will have been appropriate retention, appraisal, selection, disposal, schedules and rules. They will have been applied right across Government.
Q79 Greg Mulholland: I am asking in this specific case—and, with respect, I would like an answer, please—whether you believe that, if, as you seem to be saying, the procedures, the Grigg recommendations, were in place, with these 114 files, for whatever reason, the procedures were not followed and therefore were breached?
Carol Tullo: Clearly the procedure was not followed, because there should have been a track and an audit trail to say who disposed of them and what happened to them. I quite agree that you cannot speculate behind what the content was and what the motivation was. Yes, there is no way there should have been a disposal decision that was not clearly followed, especially if a record had been selected so far within a Department that it had gone through almost the second stage of review and appraisal. It was not discarded just at the end of an inquiry, policy report or a piece of work. It was kept for a sufficiently long time to be on the system, to have a reference. Yes, there should have been an audit trail of that. There would have been a procedure.
Q80 Greg Mulholland: Do you believe it is suspicious?
Carol Tullo: It is not for me to speculate or for the National Archives to speculate on that at all.
Chair: I think we have to wait for the outcome of your inquiry.
Q81 Paul Flynn: The news today that the Home Office is the only Department that you have not been in would add fuel to the fire of the conspiracy theory that these documents were destroyed, possibly by someone in high office who was named in them.
Carol Tullo: It does not help confidence or reassurance.
Q82 Paul Flynn: I am sorry to go on to this. We know the great work you are doing as custodians of the national memory, but it is these breaches that are causing great concern at the moment. It does seem to be unreasonable that those who have the autonomy, the Departments, in the control, collection and destruction of records, are those with the greatest vested interest in guarding their reputations and the ones more likely to destroy them. This is an unhealthy situation, and clearly there needs to be some body or person who has an overriding responsibility to go in and audit what is going on in individual Departments.
Scott McPherson: I just want to be clear that the Home Office is not the only Department that has not had an information management assessment. It is clearly one of the larger Departments. As Carol was saying earlier, a number of Departments have not had one of these assessments yet but are scheduled for one in the coming months. Some of the smaller Departments, like the Scotland, Wales and Northern Ireland Offices, also have not had one of these assessments, but that does not mean that that they have not had a relationship with the National Archives.
Q83 Paul Flynn: Perhaps Ms Tullo could explain. I understood from you that there was some resistance from the Home Office for various reasons.
Carol Tullo: They chose to go a different route in 2012, with their own internal system that they felt was tailored and appropriate to what they required. All the indications are that they are very keen to fall within the same form of overview that we have, to the extent that the way it operates is as we have been able to explain it to you. There is more that could be done, but the process that we manage now will cover some 24 Departments. As Scott said, we did prioritise the key big Departments, but we have done some like the Criminal Records Bureau and HMRC, which are not ministerial Departments, because there is a lot of interest in their material and they have very specific interest for us to be able to get an overall picture of the capability right across the piece. The Home Office is clearly one of them, but they chose to go a different route.
Q84 Paul Flynn: It is just that the atmosphere has changed. I resist conspiracy theories normally, but we now realise that Jimmy Savile was the invited honoured guest at 11 new year’s parties organised by Mrs Thatcher. He was a great and trusted friend of royalty. He was given the keys to two hospitals by two Secretaries of State for Health. It seems extraordinary that somebody could have got away with it for that long. I think we are rightly very suspicious of influence being exercised in high places.
Carol Tullo: May I give you an example related to that? Obviously, we are going to be very dispassionate, because we are not going to get into the subject matter, but some of the public records are hospital records. Hospitals, in exactly the same way as we have described to you, will use and exercise their judgment on how to keep their records. Now, hospital records are quite complex, with closures, mergers and reorganisations, and we have put quite a lot of effort into trying to instil some good practice, but it is still a judgment call. I know that, in some of the inquiries that we have assisted with, in trying to track down some of the corroborating evidence for some of those inquiries, some hospitals chose to keep their visitor logs; others did not. If you are asking me personally, I do not think there is anything suspicious about that. It would have just been a decision at the time.
These are ephemera; these are not actually part of our task of looking after patients and managing their record, but it so happened that a number of the visitor books to hospitals were kept. That has proved very useful, but that is the serendipity of a choice and a judgment that was made early on.
Q85 Paul Flynn: Just a final question: in the book that you mentioned, there is reference to the fact that a previous Secretary of State for Wales put in a report to the Prime Minister saying that Prince Charles was a terrorist, in his view. Clearly it is in the public’s interest that we know that the Secretary of State for Wales was clearly barking mad.
Mrs Gillan: Not this former Secretary.
Paul Flynn: Not this one, no—a less distinguished one than that. Will you encourage transparency on our recent past? I remember I had to go into Congress in America to discover what was being carried on a lorry that was on fire in my constituency, and it was cluster bombs. We have been neurotic, and we should be much more open about publishing these things.
Carol Tullo: Think about those basic rules. If it is going to be of historic interest, if it is ministerial, if it is to do with significant matters, then it will have been kept. The choice of whether or not, if a statement allegedly was made like that, it would ever be open in anybody’s lifetime is another matter, because obviously you have all the freedom of information exemptions that would start to apply as well. There is no reason for us, from where we sit, to think that those sorts of materials would ever be destroyed.
Chair: We must draw to a close very soon.
Q86 Mr Evans: I am just interested in this distinction between files missing, not found or destroyed. I understand destroyed. What is the distinction between missing and not found?
Carol Tullo: I cannot speak for the specific example, but I am guessing it is the logistical issue about storing and having available so many records, probably arranged over lots of different buildings, lots of different places. They have been moved as a result of Ministry or machinery of Government changes. People have imported in just whole lots of files when things have happened. It is not beautifully logical like the National Archives. The National Archives has a state‑of‑the‑art system; it is not going to be replicated everywhere.
Missing, I am guessing, means we do not actually know. The chances are they might be there, but they are not where we first looked for them, they are not in consecutive order or we cannot find them. That is the difference between missing and lost. Lost is a really serious thing if it is a record, because we would be saying, “We are safeguarding these files. We are trusting you to look after them. We need to know where they are.” Files do get lost though; it is a factor. They do get lost. Files that we have released back to Departments, working files, occasionally do go missing; they do get lost. They do get found.
Q87 Chair: It has never happened in my office. This point about the 114 files is that clearly, if they were destroyed, it was not within accepted procedures that were in force at the time. How helpful would it be if destroying files outside proper procedure was actually an offence in law?
Carol Tullo: It is a sanction. It raises the profile of the importance of effective management underpinning your business of operating your organisation. There is always more that you can do. We work as much as we can within the legislative framework and within the collaboration of conversation, support, guide, steer, advise, assess. There are opportunities to give us additional weight to be able to walk in and examine. If a sanction like that was one that you were minded to include, because of some of the wider issues that you have talked about that do not affect us operationally, then clearly that would be additional effort for us and would perhaps make people sit up and listen a little bit more.
Clem Brohier: It certainly raises the prioritisation given to record keeping, and that is certainly what we have seen over the last few weeks.
Q88 Mr Evans: If somebody has asked for a file or a record from the National Archives, have you ever turned around and said, “Sorry, we can’t find it”?
Clem Brohier: Quite often we would say, if we are unable to locate the file, the chances are that the file has not survived, and that is more commonly the answer that we would give.
Q89 Mr Evans: Do you use missing, destroyed and not found?
Clem Brohier: As I say, we only take 5% of Government records, so those answers will arise more commonly.
Carol Tullo: We must do, must we not, but we can let you know that, because actually that is not within my knowledge. We can find out, but files do go astray.
Q90 Chair: I have a few final questions, if I may. In the statement last week given by the Home Secretary to the House of Commons, a former Justice Secretary, Home Secretary and Foreign Secretary, Jack Straw, raised the spectre of what he called “a downgrading of the archiving and record keeping functions of the Home Office”—quite a serious charge from a figure like that. What is your reaction to that comment?
Carol Tullo: We certainly take great interest in moves of officials. Especially in the last four or five years, with so much effort and pressure on efficiency and driving out cost, there have been cases where we have taken issues at senior levels in Departments because we have seen a more senior role, perhaps an information risk owner at a senior level and also the departmental record officer, moved and spread across to a more junior level. We have taken that very seriously.
Actually, if you were to skim through a few of our information management assessment reports, that is often an issue that has come up. Simon is moving from Defence to Home Office. Simon has been an excellent leader of knowledge information management in the Ministry of Defence. We are still waiting and will be following through very clearly who is going to take up his mantle in Defence. It is very important for us to know that there is professional expertise and experience that will follow the standards that we want. We are very aware and we get very involved. Whether it is downgrading or whether it is reallocating resources, that is a departmental matter.
Q91 Chair: In this digital age, and Jack Straw referred to the huge increase in the volume of material that is produced by word processors, we are inevitably throwing a lot more away. Is that a reasonable thing to say?
Carol Tullo: I think we would say we are inevitably keeping more.
Q92 Chair: Departments have to pay for the storage of their stuff. When the cost pressures come round, they think, “What can we cut? Let’s cut the volume of paper that we keep.”
Carol Tullo: “Let’s transfer it faster to the National Archives” is what we find. That creates a burden for us, which is good.
Q93 Chair: One has heard apocryphal stories of whole warehouses of papers just being disposed of because the Department does not have the money to carry on looking after them. Is this true?
Simon Marsh: From my perspective, looking at the record transfer reports that have been produced and published by the National Archives on the 20/30‑year rule change, the scale of backlogs in Departments of paper records, which is primarily what we are talking about here, is quite substantial. The Ministry of Defence has—I cannot remember what the latest figures are—20,000 to 30,000 files that are out of time and are not covered by the Lord Chancellor’s instrument. If we were in the business of just wanting to destroy things, we would have just destroyed them, but we are not doing that.
Q94 Chair: Very often the author of a document, and I am thinking particularly of submissions, will regard them as very important. Are submissions kept as a matter of course?
Carol Tullo: Yes, they are. There are particular rules for private office records because they are so significant, in the same way that all the staff working in private office will be security‑cleared to a very high level. You know that you are dealing with all sorts of material that goes through the minister’s office.
Q95 Chair: Submissions are longer and longer. The word processor has made them very much longer.
Carol Tullo: Ministerial style is actually to keep them very short and have long annexes, in my experience. A submission will automatically be kept, but the private office rules are that, because it is part of a culmination of, perhaps, a consultation, a new piece of policy and a piece of legislation, it goes back to the policy department. The ministerial office will not keep the final one, and you get the convention to say, “This is signed. We are returning it back to you,” but that will be kept.
Q96 Chair: When a decision is made to dispose of a record, to what extent should the author of that record be consulted?
Simon Marsh: That is quite a difficult question, because they may have left Government service; they may have retired by that time.
Chair: We are talking Government here; you usually have their phone number.
Carol Tullo: Their name might not be on it, Chair. The convention may be that all the submissions go in the name of the senior person, so the actual author of that document would not be there. I hesitate to say a submission would never be destroyed. That is not going to happen.
Q97 Chair: If I could produce evidence from an MOD civil servant that he has gone back to find his submissions and he has found they have been destroyed, would you be interested in that?
Simon Marsh: I would be interested in that.
Q98 Chair: I shall follow that up. How big is your backlog? When you talk about a very large backlog at the MOD, how big is that?
Simon Marsh: In terms of our out‑of‑time records that we have not yet reviewed, which is what I was talking about, I think it is somewhere between 20,000 and 30,000 records.
Q99 Chair: How long will it take to clear that backlog?
Simon Marsh: Given that, at this time last year, we had about 60,000‑odd, I think we are working through it quite quickly.
Q100 Chair: How many other Government Departments are in that position?
Carol Tullo: Quite a few. This is the record transfer report that I referred to earlier, which is published. We can certainly make sure you have a link to that.
Q101 Chair: Just one or two other things: presumably it would have been much cheaper to store a lot of these paper records in digital form. Why is this not done?
Clem Brohier: It is done increasingly. What we will find is that, by the time we get to 2023, as we work through this transition, there will be more digital records that are transferred to the National Archives. It is cheaper. There is less space occupied. The technologies we have used have made it a very efficient process.
Q102 Chair: Finally, we had some difficulty; we wondered whether we should have a Minister who is responsible for Government policy on record keeping. That would be the Lord Chancellor, would it not?
Scott McPherson: The Lord Chancellor has the overall responsibility, under the Public Records Act, for appointing the keeper of the public record of the Chief Executive of the National Archives and also for the overall supervision. He also has policy responsibility for freedom of information and data protection.
Chair: It has been a very interesting session. Thank you very much indeed. There are no other questions. We will let you go and we will ask our two next witnesses to come to the stand, please.
Witnesses: Dr Caroline Shenton, Director of Parliamentary Archives, Houses of Parliament, Kim Barata, Head of Information and Records Management Service, Houses of Parliament, gave evidence.
Q103 Chair: I welcome our two new witnesses. Thank you for being so patient, but I hope you found the previous session as interesting as we did. Could I ask each of you to identify yourselves for the record?
Dr Shenton: Yes. My name is Caroline Shenton. I am Director of the Parliamentary Archives. The archive is run as a records management and archive service for the administrations of both Houses.
Kim Barata: I am Kimberley Barata. I am the Head of the Information and Records Service, which is part of the Parliamentary Archives.
Q104 Chair: I suppose you do for Parliament what the National Archives does for Government. Is that a fair analogy?
Dr Shenton: Exactly. We run, as I have said, an archive and records management service. It is a bicameral service, so our funding comes from both Houses: the Commons funds us 40% and the Lords 60%. We are based in the Lords, but we provide services across the administrations, not to Members but obviously Members are supported. For example, we provide services to this Committee, as with all Committees, and thereby you get our services in the course of your duties.
We have three teams: the Information and Records Management Service; the Preservation and Access team, which deals with the back‑room functions around historic records; and the Public Services and Outreach team, which deals with the front‑end services for historic records. It is Kim’s team, the Information and Records Management Service, that I think you are probably interested in today and how we deal with records management.
Q105 Chair: What is your reaction to the evidence you heard earlier?
Dr Shenton: In general, it sounds very familiar. This is something that will be familiar to records managers across all sectors, in the public sector and the private sector: the need for senior management buy‑in; the need for policy; the need to have a constant cycle of communication and promotion to ensure that messages are constantly reinforced across any organisation; the issue of digital storage; the cost of digital storage; the fact that digitisation, which is the scanning of hard copy, really is very expensive and no organisation, no archive or library, almost across the world, has done more than perhaps 5% of their holdings, because it is so expensive and you have to take a cost/benefit view of that. There are issues around reputational risk associated with poor information management. The fact that understanding how and when to dispose of records is really a collaborative process between the professional staff and those who are actually at the sharp end of the business, if you like, is entirely familiar.
Chair: Is there any other reaction from Ms Barata?
Kim Barata: There is also the fact that the way in which information is managed is nuanced slightly depending on the organisation. There are basic standards in place that people look to, but how they might be implemented in an organisation will vary, depending on the nature of that organisation. Also there were comments that record keeping generally is improving. Part of that is down to the impact of technology allowing us to manage things a little bit differently. I think also increasingly—I agreed with Ms Tullo’s comment—it is about good practice, being fit for purpose and making sure that these things reflect working practice.
Q106 Chair: Much of Samuel Pepys’ diary, Samuel Pepys being one of my predecessors as MP for Harwich, was ephemera and yet what makes his diary so fascinating is that it is full of ephemera. Now we have incredible volumes of emails that are full of ephemera. What happens to them all? I can tell you what happens to mine.
Dr Shenton: Private personal records are different from corporate organisational records.
Q107 Chair: I regard our parliamentary e-mail as a public space, not that I expect it to be published in my lifetime, but I do not expect it to remain secret. Certainly I expect people are looking at it. What should happen to all this digital information? Salisbury used to run the empire on 12 posts a day from Hatfield House; now we have instant access around the world to e-mail. That is going to be the source of information for historians, and yet what happens to it?
Dr Shenton: In the last few years, Parliament has introduced an electronic document and records management system for the administrations. It is called SPIRE, and that really follows on from what has been going on across the public sector for the last 10 years or so. We would expect staff in the administrations to be filing into that system, and that would eventually either be disposed of within the system or the output that was regarded as historical would eventually come into our digital repository, in the archives, which has also just gone live.
Q108 Mr Evans: What does that mean? Does that mean that, if the Prime Minister sends an e-mail in his business, in 20 years’ time—?
Dr Shenton: No, we are capturing the records of the administrations, not of the Members, whether they are the Prime Minister or not.
Q109 Mrs Gillan: What happens to my e-mails?
Dr Shenton: Your e-mails as a Member?
Mrs Gillan: Yes.
Dr Shenton: That is up to you.
Q110 Mrs Gillan: If I wanted to delete everything, it would be deleted and it would not be retained anywhere whatsoever.
Dr Shenton: Not within the administration, no.
Mrs Gillan: It would not be on a server somewhere.
Kim Barata: That would be parliamentary ICT.
Q111 Mr Evans: Is what you are saying, Chairman, that this is a historical record? It is just interesting whether they are retained anywhere so that in 20 years’ time we can read these little notes aside, which people are not keeping anymore in the same way that Samuel Pepys did.
Dr Shenton: There is an exact analogy with the past, in that the private papers of politicians are kept in separate collections across libraries and archives throughout the country, completely separate from either the records of Government or the records of Parliament. We do in fact have a small number of political papers. We have the Lloyd George papers, for example, in the archives, but that is not part of the core administrative record; it is an historical accident that we happen to have those.
Q112 Mrs Gillan: In your experience as record keepers and archivists, and having heard the last evidence session, is there anywhere else around the world, in any administration, that actually does it better than we do, either in Parliament or in Government? Is there any shining example we should look at?
Dr Shenton: There are international standards, so the standard will apply across the world, and that is what record keepers across the world will be looking at. The standard is really outcome‑based; it is not a series of prescriptive processes. It is up to each individual organisation to decide how they apply that standard to get the outcome that they need.
Mrs Gillan: There is no shining example you would hold up to us, in your experience.
Kim Barata: It would be very hard to do that, because different industries would manage their information in different ways depending, as Caroline said, on the outcome that they are trying to achieve. You will find that, even within the public sector, what a local authority is trying to achieve versus what a central Government Department and what we here, for the administration of both Houses, are trying to achieve are slightly different. It would be very hard to compare them. There is an international standard on how to manage records. Within various countries there are national standards.
Q113 Chair: Is it actually possible to identify best practice in records keeping?
Kim Barata: It is possible to identify good practice for different industries and record keeping for those. For example, in central Government, as Carol Tullo said, they have standards for what they would consider good practice. You could look in the pharmaceutical industry or other industries and say, “This would be good practice for them.” It would be quite difficult to try to apply that carte blanche across all organisations.
Q114 Chair: What would your recommendations be for the improvement of Civil Service record keeping? What would be the top three things that spring to mind, having listened to this morning’s session?
Dr Shenton: I do not think we can comment directly on Civil Service record keeping.
Chair: Oh yes you can.
Dr Shenton: Generally in terms of records management in any organisation, it is about having really good senior management buy‑in, so that it is demonstrable to staff that this is something that the organisation takes very seriously. By coincidence, the Archives and Records Association of the UK, which is the professional body for archivists and records managers, last week launched a campaign called “Don’t Risk It! Know Your Records”, which was actually about raising awareness amongst senior managers of how important record keeping is in their organisations. I can send the Committee details of that if it is of interest. That is a key thing—having the senior management buy‑in, having the policies in place and understanding the implications of a digital world.
Q115 Chair: I am very interested in your indication of the requirement for strong leadership and clear leadership in this. At the moment, I do not know; are permanent secretaries vested with a particular responsibility, as they are for accounting, for records management? They are accountable to Parliament as accounting officers; should we make them accountable to Parliament for record keeping?
Dr Shenton: I would have thought that good information management is important for any chief executive to be involved in.
Q116 Mrs Gillan: Can I ask an aesthetics question? One of the most amazing things about the archives that we have here is that we can produce the original documentation, signed by the people in question, which are absolutely pieces of magic, as far as I am concerned. Do you think with the digital age that we have entered that we are now going to lose a lot of those wonderful original documents, which people will just scan in and keep as digital records, without actually keeping the originals? Is there a huge danger?
Dr Shenton: Yes. There are records that are born digitally that will never exist outside the digital format. It is absolutely true that, in future, historians of the late 20th, early 21st century and onwards will have a very different experience from, for example, me. I started off as a medieval historian handling parchments from the 14th century. That is a very different experience: it is not better or worse; it is just different.
It is certainly true that digital records change the nature of historical research. This is something that there has been a lot of interest in in digital humanities curricula in universities—about how users respond differently to digital information. Actually, the proliferation of digital information means that, on the one hand, there is a huge amount that historians will have to get through in the future, and it will also change the nature of their research, the actual topics of their research, but it will also add the richness that you could not get in the past by having hard‑copy records.
Q117 Mrs Gillan: There is a danger when you interrogate something digitally that you are looking for the thing that you have in your mind’s eye, but you are not having the advantage of scanning the whole document, as you would if you were reading something on a paper or parchment basis, which would make other ideas spring into your head. It is probably going to narrow the field.
Dr Shenton: It may do. If those psychosocial aspects of historical research are understood more fully, then researchers in the future will be aware of the impact of the digital on them and will take that into account.
Q118 Mrs Gillan: Just by reading across, if we were looking for 114 missing files, if we were interrogating it digitally, we may not just hit on the very phrase or wording when searching that might lead us to a file in question that might actually still be there.
Dr Shenton: It might be. Again, this is back to the issue that came up in the previous session about file titling, and making sure that the meta‑data, the information that you have about that record, is sound.
Q119 Mr Evans: Is it true that you are digitising the whole of Hansard?
Dr Shenton: The historic Hansard has already been digitised.
Q120 Mr Evans: Back to when?
Dr Shenton: Back to its origination, which was 1803. That has been online now for about six or seven years, and that was done by the Commons Library.
Q121 Mr Evans: Prior to that, what is the oldest record that you have?
Dr Shenton: Our oldest record is 1497, which was the point at which the Government clerk in the Chancery decided that he was fed up with sending his records back to the Government; he was going to keep them on site at Westminster. It was at that point that a separation of powers occurred in the administrations of Parliament and Government.
Q122 Mr Evans: I understand there are a lot of documents in Victoria Tower still kept today.
Dr Shenton: That is true.
Q123 Mr Evans: What precautions are now taken to ensure that there would be at least a record of all those documents, which are of great historical importance, should there be another fire?
Dr Shenton: There is very good fire protection in the tower. It is the same with any archive; there is very good fire protection and we do have a programme for digitisation of treasure items and high‑priority items.
Q124 Mr Evans: That is ongoing at this moment in time.
Dr Shenton: That is ongoing but, as we have all indicated, that is extremely costly, and so you have to take a view about what is worth digitising and where you want to generate an interest in particular topics.
Q125 Mr Evans: There would be documents that would be of historic interest and others that also you would say are priceless.
Dr Shenton: Exactly.
Q126 Mr Evans: All the priceless ones are now at least protected?
Dr Shenton: They are.
Q127 Chair: Is there anything else you want to add to the record before we close?
Dr Shenton: Other than to invite the Committee to come and visit the archives if they wish, no.
Chair: I would love to. I will probably take you up on that. We may have recourse to your further advice in respect of our interest in this matter.
Dr Shenton: We can certainly point the Committee towards the sorts of disposal policies and internal documentation that we have on the intranet.
Chair: Thank you very much indeed. It has overall been a very interesting session and you have added to our depth of understanding. Thank you very much indeed.
Oral evidence: Preservation of historical files in the Civil Service, HC 550 2