Getting started with the Findings

Having initiated a preservation planning regime for the collection, and having moved it onto the Windows 10 platform, I’m feeling that the only remaining things I need to do with it are to find it a permanent home and to write up the findings of this lengthy experiment. I took a step forward on the latter activity earlier this week when I had a very interesting phone call with Peter Tolmie, a UK Ethnographer based in the School of Information Systems and New Media at the University of Siegen in Germany. I was given Peter’s name by Richard Harper when I asked if he knew of anyone who is knowledgeable about how professionals manage their documents and who would be interested in working on a wrap-up paper with me. An initial phone call with Peter last Thursday indicated that we have a great many common interests – I found it a very stimulating conversation indeed. I’ve sent Peter some documents describing the collection and we’ve agreed to talk again on 21st March.

Regarding the search for a home for the collection (which is documented in various posts in this Blog going back to 2015), my current efforts lie in conversations I’m having with Dr James Peters, the Archivist of the National Archive for the History of Computing at Manchester University, who has kindly agreed to help me in my search. In a phone call last month, James told me he was waiting for a response from someone he had emailed, but that, if there was no interest from that source, he could issue a note to a relevant mailing list on my behalf. If it is to be the mailing list route, I’m hoping to get James’ advice on what needs to go in the note.

Backup Bolstering

Backing Up has always been an essential part of maintaining my personal document collection; but it was never something I enjoyed – I did it out of a fear of loss. And I have, indeed, experienced loss: in 1996 one of the MO Disks I was using became corrupted and I lost a number of files; in 2004 my laptop was stolen and my whole document collection had to be re-instated from the backups; and in 2017 I had a system crash and, although the repair company was able to recover all my data in that instance, that might not always be the case.

When I was working, I used to take a backup of the more recently created material every month or so, as well as complete versions of the whole collection as it kept growing. This produced multiple copies on many disks which increased my confidence in being able to replace any file that got corrupted or mislaid, but which required managing in its own right as the number of disks grew. As time went by I added other backup mechanisms including storing a copy on another laptop in the househoId, storing a copy on disk in a relative’s house located many miles away, and storing a copy on disk at my son’s house in New Zealand.

After I retired I tried to put the backing up on a more orderly basis and finally fixed on five different types of backup – Cloud, copy on another laptop in the house, local hard disk, remote (in the UK) hard disk, and New Zealand copy on memory stick. I scheduled backups in my iPad calendar for each of these (though, for the Cloud, it was more a matter of checking that it was working and that I could recover from it). However, the iPad calendar doesn’t have a To Do mechanism and I wasn’t looking at the calendar anything like as often as I used to at work. Consequently, I kept missing scheduled backup activities – and, in most cases, didn’t realise I’d missed them; and when I did realise I just kept putting off what was an annoying extra thing to do. One answer would have been to get a To Do app – but I’d had enough of To Dos at work.

The opportunity to come up with an alternative approach, came when I created a Users’ Guide for my document collection in May 2018. I structured the Guide so that it had a Quick Reference Guide to the Collection on the front page, and a Backup Quick Start Guide on the back page. The latter listed the different types of backups to be performed and provided cells to be filled in with a date when that particular backup had been done, as shown below.

This was a definite improvement over dates dotted about a calendar, but unfortunately the schedule was still hidden because the Users’ Guide was tucked away inside an archive storage box.

When I replaced my Windows 7 laptop for a Windows 10 version in December 2018, I decided to review all my backup arrangements again and to try to overcome this lack of visibility. The answer turned out to be really quite simple: I have a display frame for the latest issues of UK postage stamps, on the wall in front of where I sit at my desk. So, I created a table with columns for when backups have been done and when they are due; and this table now resides in the display fame as shown below.

I have a clear view of when the next backups are due every time I sit down at my desk. The next time I miss a backup it’ll be because I just don’t enjoy doing them, not because of blissful ignorance!

From Nottingham to Manchester

Last month I heard back from the keeper of Manuscripts and Special Collections at the University of Nottingham, Mark Dorrington, who said that my collection may not be a good fit with their archives and that, in any case, they were not geared up to deal with such a large digital collection. However he did suggest trying the National Archive for the History of Computing at the University of Manchester and provided a link to its web page.

I have, in fact, already been round the houses with the University of Manchester Library; however, that was not specifically in relation to this particular archive, and it was before I had done any digital preservation work on the collection. So, today I tried making contact with someone specifically concerned with this particular archive and was told that the archivist for this and a number of other special collections is Dr. James Peters. I duly emailed him with the following opening para: ” Dr. Peters, I’m contacting you as the Archivist in charge of the National Archive for the History of Computing (NAHC). I have a collection of documents which reflect the development and application of computers over the last 40 years, and would be grateful for your advice as to whether the collection has any merit and where it could be placed.” I followed this with a description of the background to the collection and of its contents. I’m hoping that my rather indirect approach on this occasion might engender some discussion rather than the outright rejection which I’m becoming used to.

Still looking for a home

Back in 2015 I reported on my efforts to find a permanent home for my document collection. I had no success with any of the organisations I mentioned in that post, and subsequently turned my attention to trying to find a contemporary historian who is interested in the development of computing. I came across one Daniel Wilson (no relation) based at Cambridge University who has a particular interest in the history of science and technology; and I duly contacted him. Despite being interested in hearing about the contents of the collection, he felt unable to help, explaining that “this will require significant work and few people have the budget or the time, given current pressures”. He gave me the name of another contemporary historian at Leicester University who I also tried emailing, but, despite sending a follow-up, I got no response. I’ve concluded that individual academics just have too little time to take on the management of a collection that isn’t absolutely central to a specific piece of work that they are doing.

I am now turning my attention, once more, to institutions, and have just sent an email to the Keeper of Manuscripts and Special Collections (MSC) at the University of Nottingham. I came across this organisation in a JISC email which advised that MSC has just joined the DPC. I was able to mention in the email that, not only have I just completed a digital preservation exercise on the PAWDOC collection using templates which are published in the DPC website; but also that the PAWDOC collection contains much material from the Cosmos project in which the University’s Department of Computer Science took part – perhaps those little extra bits of information might spark an extra bit of interest.


As reported in the Preservation Planning Journey in this Blog, my document collection has just been exported from the Document Management System (DMS) that it has been in for the last 22 years, and now resides in some 16,000 Windows folders. I feel a strong sense of relief that I will no longer have to nurture two complicated systems – the DMS and its underlying SQL database – in order to access the documents.

Over the years I have had to take special measures to ensure the survival of the collection through 5 changes of hardware, one laptop theft and a major system crash. This included:

  • trying to configure and maintain complex systems I had no in-depth knowledge of
  • paying out hundreds of pounds for extra specialist support (despite the software cost and most general support being very kindly provided free because this has always been a research-oriented exercise)
  • engaging with support staff over phone, email, screen sharing and in person for hundreds of hours to overcome problems (it starts to add up over 22 years…)
  • backing-up and protecting large amounts of data (40Gb total) regularly and reliably.

That’s not to say that DMSs are not worth using – they have characteristics which are essential for high usage, multi-user, systems in which regulatory and legal requirements must be met. However, such constraints don’t apply to the individual. The stark conclusion has to be that, for a Personal Information System, using a DMS was serious overkill.

I guess I’d already come to that conclusion back in 2012 when I set up a filing system for my non-work files using an Excel index and a single Windows Folder for all the documents. That has worked pretty well, however it’s slightly different from the way the newly converted work document collection is stored which has a separate Folder for each Ref No as shown below.

Experience so far with the Windows Folder system indicates that it is very easy and quick to find documents by scrolling through the Folders – quicker than it was using the DMS since there is no need to load an application and invoke a series of commands: Windows Explorer is immediately accessible. As for the process of adding new documents, that too seems much simpler and quicker than having to import files into a DMS, because it involves using the same Windows file system within which the digital files reside in the first place.

Its early days yet so it’ll be a while before I have an in-depth feel for how well other aspects of the system, such as backup requirements, are working; watch this space.

Disks and DMS

As part of the digital preservation work (documented elsewhere) that I’m doing on my document collection, I’ve just completed an exercise to organise and index all the associated physical disks.  It turns out that there are 156 disks of which 16 are actually contained in the collection, and the remaining 140 are backup disks (which have been accumulating over the years) of the collection’s computer system and digitised contents. Old backup disks may not be useful to restore a system crash, but I have kept them to provide an audit trail over the 20+ years that the digital system has been in operation.  Over that period documents have been lost, the index has had fields deleted by mistake, files have been corrupted, and no doubt other errors have occurred. Although the number of such occurrences is low, when such problems are identified it is very useful to have the ability to trace back through previous states of the system.

Another activity that has been prompted by the digital preservation work is to establish what future plans the current supplier of FISH (the document management system I use) has for the system. Last time I asked the question in February 2016, I was told that there are no plans to upgrade the product and that current customers who wanted to look at alternatives were being advised to consider a product called File Stream supplied by Filestream Ltd which is based in Berkshire in the UK. I spoke to the Fish supplier, m-hance, again earlier today and was told there had been no change – it is unlikely that Fish will be upgraded and Filestream is still the recommended replacement product. When I contacted Filestream last year I was told that the product would cost £750 to purchase and £250 a year for support including upgrades.

When I was investigating Filestream last year, I also took a quick look at Open Source document management systems and found several – some of them being free to use. However, further investigation would be required to establish what other components (such as the back-end database) would have to be acquired and whether they would also be free.

These and other options to future proof the collection will all be considered in the digital preservation project currently underway.

Digitised and Checked

I reached a milestone today: my document collection is totally digitised, and every Index entry and associated Document Management folder has been checked. It’s been a very laborious process – which is why my last entry here was over four months ago. However, the collection is now in good shape for a digital preservation exercise, and is ready for transfer to a long-term repository if one can be found.

Following the checking exercise, a detailed analysis was performed to derive statistics and rectify problems where possible. The report documenting the analysis serves as a comprehensive status report on the whole collection at the end of May 2016.

Digitisation in progress

Since my last entry I’ve been steadily digitising the remaining paper in my lifetime work document collection. These are documents I want to retain in original form (some of which have a comb binding), documents that need to be scanned in colour, or documents that were too large to go through the scanner. I acquired a better comb binding machine at the end of October, my current scanner has full colour capability, and I’ve found that photographing large items with my modern camera produces a perfectly readable on-screen image. So there’ve been no more obstacles to getting the job done. As each item is digitised and the file inserted into the FISH document management system, I’m checking the index entry and updating the Movement Status field with either OK or XX as described in my last entry.  At the current rate of progress I should finish the digitisation work by the end of January.

Checking the Collection

Two of the remaining things to be done with my lifetime document collection are to:

a) scan the remaining paper (documents not yet scanned because they were labelled as artefacts to be retained in both their paper and electronic form); and

b) go through all the index entries making sure they contain valid information and that there is an equivalent scan in the Document Management System.

For a) some of the paper documents have comb bindings and will require a binding machine if they are to be scanned using a sheet feeder and then reassembled in the comb binding. I acquired a very cheap comb binding machine on ebay some three weeks ago (though, it seems it was false economy – it stopped functioning properly and I had to send it back yesterday…) and have made a start on scanning the remaining paper. I’m addressing b) in parallel, and recording any issues or key points I find using the following notation in the ‘Movement Status’ field:

OK = The Index entry is as complete as possible and there is an equivalent scanned version

XX = There is a serious issue with this item.

Should the index entry and scans be present but there are some points to be recorded about them,  the ‘OK’ notation is qualified within brackets as follows (multiple qualifications can be recorded within the brackets as necessary separated by  a comma):

  • OK(multi): one or more of the equivalent scanned files in the FISH Document Management System are in the form of multiple TIF files – one for each page. FISH obscures the fact that there is a separate file for each page – but that is how the scan is actually stored.
  • OK(n docs): This identifies when there is more than one scanned document associated with this index entry – where n is the number of separate documents (this is a feature of this approach to electronic filing – multiple documents can be stored under a single Index entry).
  • OK(poor): the quality of some or all of the scanned electronic pages is poor.
  • OK(dbl): one or more of the associated scanned files came from documents with double sided pages which have been scanned all of one side first and then the pile turned over and the other side scanned. When this has been done the scanned pages are out of order. This was done with the first two scanners I had which were not able to handle double sided pages.
  • OK( ord): the pages of one or more of the scanned files are out of order for a reason other than the ‘dbl’ reason above.
  • OK(left): the original document was deliberately left at the location of the employer concerned when I moved jobs.
  • OK(A5): one of the scanners I had was not able to handle A5 pages reliably and sometimes recorded a line as an image dragged down the page for an inch or more.

Should an XX notation be applied to an Index entry, the reason it is being noted as such is recorded in brackets with one or more of the following notations:

  • XX(lost): the paper document was lost before a scan could be taken, so the Index entry is the only trace left of this document.
  • XX(ref): The Reference No is duplicated or incorrect in some other way.
  • XX(pap): The document is still only in paper form because its form is such that it has not yet been possible to digitise it effectively.

The fact that such points and issues are present in the collection in noticeable numbers, simply reflects the fact that, when dealing with such large volumes of material in the course of performing busy jobs across many years, it is inevitable that things will go wrong and mistakes will be made. Having been through the whole of the index, I’ll have statistics about the overall prevalence of such issues in this particular collection.

Search Status

In my last entry I said I’d contacted six potential repositories for my lifetime document collection. This is where those communications are up to:

Loughborough University’s Centre for Information Management: My email was forwarded to the University Library which did not respond. I followed this up on 20th September with an email to the Director of Library Services, and am waiting for a reply.

Manchester University’s Computer Science Dept Research Office: My email was forwarded to a researcher with an interest in the history of computing, but that person replied saying that her work in that area had been put on hold. On 20Sep I used the University library general enquiry form to enquire if the library would be interested. The library advised me to contact the Head of School administration  in the School of Computer Science who I duly emailed on 2nd October, and I am awaiting a reply.

City University’s Cass Business School:  My contact said he would pass my email onto his colleagues and I have heard nothing further.

UCL’s Dept of Information Studies: My contact said she would look out for interested people at conferences.

The National Archives: I was advised by my contact to direct my question to the Archive Sector Development team which, while it does not have any direct provision for taking private collections, should be well placed to provide advice. I emailed the Development Team on 16th September and am waiting for a reply.

 The Science Museum Wroughton Library and Archives: The Library asked me a number of questions about the collection but finally responded by sayingThank you for allowing us the time to consider your collection which we have now discussed with the Archive Collections Manager, Science Museum’s Keeper of Technologies and Engineering, and Head of Library and Archives. We have concluded that whilst we find this a most interesting idea, we do not think that the content fits within our current collecting policy criteria. You may have already contacted them, but we suggest that the National Computing Collection might be a more appropriate repository for your collection.”

This is all pretty much as expected: I know its going to be hard to find a repository that’s interested. However, should there be no interest from any of the above organisations, I plan to rely on interest being generated by the publication of my paper on Digital Preservation Planning.