Most types of things can be managed in a digital filing system. Documents, brochures and books can be either scanned or photographed; electronic documents are, of course, already in the right form; and even physical artefacts such as, for example, art works, can be photographed.

Once it’s been decided to establish a filing system, the reasons why the items concerned are being kept, and what is going to be done with them, should be written down. This should help in defining the following three key points: i) how long items are to be kept for; ii) whether any physical originals are to be retained after they have been digitised; and iii) whether every item is to be filed or just a particular subset. Point iii) is important because, if only a subset is required, the amount of time and effort that will be spent on adding items to the filing system, and managing it, will be reduced. Against this time saving must be weighed the extra effort – and potential errors that may be made – in deciding whether each particular object is to be included or not. Conversely, collecting every instance of a type of an item makes life easy and requires no decision making.

If the filing system is to replace or augment an existing collection, a decision must be taken about whether to exclude the existing items, or to undertake ‘backfile conversion’ to include some or all of them in the digital collection. Backfile conversion is often a lengthy and tortuous process and can act as a drag on proceeding with the new system; therefore, it should be avoided at all costs. If, however, it is thought to be needed, the reasons for doing it should be clearly understood before proceeding.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q5. What types of information and media can be managed by an electronic filing system?

2001 Answer: Fully answered: Just about anything can be managed by an electronic filing system including single sheets and stapled sets of hardcopy documents, scientific journals, volumes of bound material, ring binders, books, 35 mm slides, videos, electronic files and e-mail messages.

2019 Answer: Fully answered: Any kind of item can be managed by an EFS – though it may be necessary to also keep the original physical artefact to experience the full impact of certain types of items such as specially folded brochures, or significant documents with a tactile presence. Types of items that I have succeeded in managing in the PAWDOC collection include hardcopy documents, ring binders, books, journal papers, magazines, brochures, overhead slides, flipchart pages, CAD/CAM drawings, photos, videos, web pages, and email messages.

Q6. Can all types of electronic files be managed in an electronic filing system?

2001 Answer: Partially answered: Yes, certainly all types of application files can be. I have not stored executable files yet but have been advised that it can be done and how to do it.

2019 Answer: Fully Answered: Any types of file format – including executables – can be managed in this way. For example, file types in the PAWDOC system include Word, Excel, Powerpoint, PDF, TIFF, JPG, MP4, MPP, ZIP, and HTML. However, the real issue is whether you will be able to open them in future years if the particular application programmes are lost or not kept up to date. For this reason, it may be worthwhile converting some files to more common formats, such as PDF, before storing them. In the future, these so-called ‘Digital Preservation’ issues may get addressed automatically by AI programmes.

Q7. What criteria should be employed when deciding what to file?

2001 Answer: Not started

2019 Answer: Fully Answered: Individuals should file as much or as little information as they think they will need in their jobs – otherwise they will not have the motivation to spend the time introducing new material and managing the filing system. If a robust, all embracing, file facility is available within a particular system (such as in many electronic mail systems) such systems should be used to their full capability (i.e. retain everything up to any limitations set by the system administrator). Inevitably, whatever an individual decides to file, there will always be some material which is never subsequently accessed, and there will always be some required information that can’t be found in the system. This seems to be a regularly occurring phenomenon and should not deter individuals from filing – though they may influence the filing criteria that an individual applies over time.

Q8. Is it worth doing backfile conversion?

2001 Answer: Experience gained: Backfile conversion is certainly not essential, and is probably not worth doing, except for material you definitely want to keep for many years.

2019 Answer: Fully Answered: Backfile conversion usually takes a huge amount of time and effort, so a detailed analysis should be undertaken before deciding whether to do it or not. Alternative approaches to dealing with the old items should be investigated. Backfile conversion should only be carried out if there are compelling reasons for doing so.

Q9. What are the major considerations when carrying out backfile conversion?

2001 Answer: Experience gained: Backfile conversion requires a huge amount of time and effort and is not to be undertaken lightly.

2019 Answer: Fully Answered: Considerations to be taken into account when conducting backfile conversion include:

1. Ensure the physical backfile material does not interfere with any space required by the replacement filing system.
2. Decide if any of the backfiles being converted will need to be retained in their original form and, if so, define clear criteria for selecting which ones to keep.
3. Decide what quality the converted backfiles need to be and then put in place the appropriate equipment, software and procedures to achieve that quality.
4. Decide how users will be informed of which backfiles have been converted and which are yet to be converted.
5. Establish a clear and doable schedule for the conversion process.
6. Put in place motivational aids for sticking to the schedule. For example, setting a target of a certain number of conversions each day; and creating a visible progress sheet on which achievements can be ticked off.

The PAWDOC collection was set up in 1981 to explore the application of office technology. In 2001 a paper was published in the Journal Behaviour & Information Technology called ’20 years in the life of a long term empirical personal electronic filing study’. This described PAWDOC and summarised findings about its use up to that point under the following 15 headings:

• Collection content, size, growth rate and usage patterns (this post)
• Deciding what to file
• Indexing
• Scanning
• Searching
• Use of information
• Archiving
• Hardcopy/electronic mix
• Relationship between work patterns and filing activities
• Technology requirements and problems
• Portability
• Sharing files
• Confidentiality, ownership and intellectual property rights
• Reliability and Longevity
• Costs and benefits

Two extra considerations are ‘Architecture‘ and ‘Requirements and Objectives‘. Now, a further 18 years on, the findings will be further reviewed in this and subsequent posts. The first of these follows below.

Collection content, size, growth rate and usage patterns

The precursor to PAWDOC was a conventional filing system in an upright cabinet using hanging folders and crystal tabs to store documents only (not journals, books etc.). The contents were specified in an index taxonomy with entries of the type, for example, 1.6.8 – Quarterly Progress Reports, which I was constantly adding to. Consequently much space was taken up in the cabinet by folders with crystal tabs housing only a few pages.  This changed after the PAWDOC system came into use: it needed far fewer hanging folders because each folder was filled to capacity with as many of the serially numbered documents that it could take.

The PAWDOC schema was explicitly defined to support the management of multiple sets of different material owned by multiple different owners. This capability was used to manage a variety of sets of my own material in addition to documents, for example, several different journals, 35mm slides (for making presentations), ring binders, and books; and, in the early years of the system, material owned by other people and organisations – though, as time went by, I made fewer and fewer such entries not least because of the uncertainty of being able to access such material.

The system soon became an integral part of my working life, and I used it just about every day. Several hundred new items were being added each year and this steady rate of acquisition soon ran up against the physical limits of the upright cabinet, so it became necessary to archive material in boxes and then to put some of the boxes in store. This went on until I started digitising newly acquired documents in 1996 and disposing of most originals. At this point, I also started to store born digital documents regardless of the applications they were created in.

In 2001 I started a new job in Bid Management which precluded personal storage of its associated highly confidential and fast moving documentation; so my useage of the PAWDOC system did reduce from then onwards. Nevertheless I continued to use it regularly on most days, and was still adding over 200 new items each year up to when I retired in 2012.

The 45 boxes of hardcopy that I had acquired eventually came out of store around 2001 and were stored in my garden shed. Their number was overwhelming and I began to doubt I would ever get them all digitised. However I stuck at the task, sometimes going at it solidly for several days at a time when my wife was away. By the time I retired in 2012 only 4 boxes remained and these had all been scanned by 2014: it was a great relief. The huge amount of physical space taken up by the collection had been reduced to just two archive boxes of significant hardcopy documents; and, after I had conducted a digital preservation exercise on the collection in 2018, the digital footprint of the collection amounted to some 115Gb. I have ended up with a fairly complete digitised archive of all the non-highly confidential materials that I had encountered throughout my working life, including substantial amounts of material from my earlier career from 1972 with Kodak and then CPC. Since retiring I’m continuing to add a few documents (around 40 up to 2019) in three categories: significant articles relating to the work I used to do; documents relating to the digital preservation of the PAWDOC collection; and material relating to work I am doing to investigate and document the findings from the PAWDOC collection’s 38 years of existence.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q1. What are the contents of the collection?

2001 Answer: Fully answered: At the beginning of July 2001, the collection consisted of 14,100 index entries representing approximately 185,000 pages of paper, 50,000 scanned pages, over 30 scientific journals (including Behaviour & Information Technology from 1982), around 30 books and conference proceedings, 3700 MS Word files, 400 MS Excel files, 250 MS PowerPoint files, 150 other electronic files of various types, and 10 CDs.

2019 Answer: Fully answered: The PAWDOC user Guide created in 2018 says: “All types of documents were stored including letters, internal memos, circulars, reports, specifications, minutes, overhead slides, 35mm slides, notes, training materials, brochures, manuals, maps, emails, computer magazines, journal articles, conference proceedings, and videos. As Office Technology became more versatile, electronic documents such as word processor files, spreadsheets, presentations and web sites were also filed.”. In June 2019, the collection consisted of 17,293 Index entries representing 29,610 electronic files in 16,067 Windows folders, and about 340 physical hardcopy documents in two archive boxes. A further 384 old electronic backup files are also stored in a separate folder. The The checking exercise conducted in 2016 identified the following numbers of different types of files in the collection: Word – 6380; Powerpoint – 466; Excel – 625; HTML – 382; Help – 90; Zip – 92;  11 other apps – 88; Scanned documents – 28,418. The collection was primarily a work collection and therefore the number of new items being included reduced to a trickle when I retired in 2012.

Q2. How much space does the collection take up?

2001 Answer: Fully answered: The paper takes up about 4.7 sq. metres of floor space and 1.7 metres of shelf space. The scanned images and electronic files take up 2.9 GB. The scanner, magneto-optical drive and CD Writer take up about 0.25 sq metres of desk space. The Filemaker Pro index is about 8.1 MB in size and the FISH data file is 11.2 MB. The Filemaker Pro, FISH, SQL Anywhere and Easy CD Creator software packages take up approximately 38 Mb.

2019 Answer: Fully answered: The two archive boxes stand one upon the other and take up 0.2 sq m of floor space. The laptop in which the electronic files reside takes up 0.08 sq m of desk space. The electronic files of the main collection take up 45.9 Gb storage space; and the backup files take up 66.6 Gb. The Filemaker Pro software used for the index takes up 336 Mb of file space.

Q3. What is the growth rate of the collection?

2001 Answer: Partially answered: Between 1981 and 1993 an average of 543 index entries were created each year – estimated to consist of an average 29.4 pages per day (Chan 1993:25).Over the whole 20 years life of the system, the growth rate has been an average of 705 entries per year with a range of 210 ± 1202new entries a year. In 1993, it was estimated that the collection was increasing at the rate of 3.8 MB per day.

2019 Answer: Fully answered: The growth rate of the collection is shown in the chart below.

There are three distinct phases: 1981 – 2000; 2001 – 2011 when I was working on highly confidential bids; and 2012 – 2019 when I was retired. The average growth rates per year during these periods were:

• 1981-2000 – 696
•  2001-2011 – 272
• 2012-2019 – 46

Q4. How often are the contents accessed?

2001 Answer: Experience gained: Between 1987 and 1993, an average of 363 records were being accessed each year (Chan 1993: 25)

2019 Answer: Experience gained: The only data that has been collected on this question is in the date last accessed field, and unfortunately that only records the latest date an item was accessed – there may have been any number of earlier accesses. Furthermore, some items may have been accessed without the date last accessed field being updated. Having said that, 4,551 items have an entry in the date last accessed field, implying that 12,742 items have never been looked at for work purposes after they had been included in the collection (the date last accessed field was never updated when items were looked at for the purposes of controlling and writing about the collection). For the period from 2001 onwards, when I moved jobs into Bid Management, there were 751 index items with dates of 2001 or later in the date last accessed field (only 15 of these were in 2012 and only a further 15 of these were from 2013 onwards).

NB. The various references in the texts above to Chan,1993 relate to the following reference at the end of the 2001 paper in Behaviour & Information Technology:

CHAN, S. C. 1993, Feasibility of Paperless Office, Submitted in partial fulfilment of the requirement for the degree of MSc in Information Systems and Technology in The Information Science Department at City University, London (Supervisor: Dr David Bawden) [PAW/DOC/4012/08].

This entry has been jointly authored by Paul Wilson and Peter Tolmie

The PAWDOC filing system was set up in 1981 to try and understand how the newly emerging office technologies of that era might assist individuals to manage their office documents. Over its 35+ years of operation much has been learnt, and it is our intention to try and understand and describe those findings. However, the system was set up to address requirements in the office of the 1980s, and there has been a revolution in the way business operates since then. In order to be able to relate the findings to office work today and in the future, this entry explores the differences in  requirements for personal filing in the office between the early 1980s and 2019.

Perhaps the most significant difference is the transfer of huge amounts of information from paper-based documents to digital files. Note that this is not saying anything about the current volume of paper in the office (though we would suggest that there is probably less paper filed by individuals now than in the 1980s) – just that individuals now have to deal with huge amounts of electronic material in contrast to the early 1980s when they dealt with virtually none. This transition has been facilitated by a huge growth in the use of computer hardware – desktop computers, laptop computers and mobile phones – throughout the world, from a base of zero to near-universality.

The growth in the use of computers has also prompted huge changes in office work. At the beginning of the 1980s, office professionals used support staff – typing pools, secretaries, and admin support staff – to perform their administrative tasks.  However, as office technology became widespread, professionals started to do their own typing and support staff became a luxury which could be cut to reduce budgets. In 2019, only very senior management have secretaries and office workers are expected to be self-sufficient and fully competent in the use of all hardware and software relevant to the realisation of their work.

These changes were accompanied by a revolution in communications. Electronic mail has now almost entirely replaced internal memos and external letters, and has prompted massive increases in the amount and speed of communication. Email also rapidly became the key mechanism for supporting distributed teamwork – nationally and globally – and now underpins a battery of related interests, from the sharing of documents to the organisation of voice conference calls (which are the unsung foundation upon which much of business and government now operates). In more recent years, as mobile phones have permeated throughout the world’s populations, text messaging and chat applications have become an integral element of personal and business relationships. To this highly significant mix of new technologies must be added the recent massive uptake in Social Media. An unfortunate side-effect of the sheer effectiveness and pervasiveness of these mechanisms is high levels of information overload across a large proportion of office workers.

Importantly for the work to derive findings from the long-term operation of the PAWDOC filing system, the changes described above have impacted filing activities in the office. Hot desking and home working have made personal filing cabinets and bookshelves a luxury. The folder systems integral to computer operating systems (primarily from Microsoft and Apple) are now used to store the electronic documents created and received by the individual. At the same time, email systems have their own integral filing systems into which mail can be rapidly sorted and stored indefinitely in the cloud; text messages are stored on users’ mobile phones in the form of text streams by both senders and recipients; and Social Media systems have their own self-contained environments distributed across vast computing networks. The further evolution of cloud-based repositories, such as Dropbox and Google Drive has led to an added utilisation (if not trust) in distributed document stores. Even if users wanted to integrate these different collections, it would be almost impossible for them to do more than just copy selected elements from one to another or to a dedicated filing system: these stores are separate silos and will probably continue to be so for many years to come.

The design of the PAWDOC system in 1981 was based on an understanding of office filing requirements at the time. There was an expectation of how emerging office technology might be used to support those filing requirements, but little appreciation of how the technology itself would change the way business operates. Initially, then, learnings from the development of the PAWDOC system were entirely focused upon what the impact might be of new assumptions about filing built into the construction of computer systems in the early 1980s. Later on, in the middle period of PAWDOC operation, the findings speak to what it was taking to manage a filing system in a changing work environment populated by imperfect but maturing technologies. More recent findings give a somewhat different picture, as many of the troublesome technologies of the middle-era have come to be taken-for-granted resources, giving rise to new kinds of problem, of which information overload is but one potential symptom. What is clear at present is that computer technology and the business world is now changing so rapidly, the presumption present in the early days of PAWDOC – that one could readily identify needs and solutions for the future – now seems somewhat naïve (if still just as pressing).

One thing, however, we believe has remained constant and that is the attitude towards filing across the population. Most people are not motivated to put effort into filing because it is extra work for an indeterminate reward at some undetermined point in the future. A smaller subset of people is willing to put varying degrees of effort into the activity. We believe this has changed little between the early 1980s and the present day. As it happens, the PAWDOC owner was at the more extreme end of this latter group and wanted to file both effectively and comprehensively. Hence the PAWDOC collection contains most of the documents that the owner read and/or believed to be significant in his work; and consequently it should be borne in mind that the learnings derived from his experiences concern almost the worst case requirements of filing load and effort. It should be easier for most of the population. Certainly, it would seem easier, for digital copies are now retained of virtually everything as a matter of course. The extent to which that is oriented to as a personal collection of materials is a different matter, as is the probity of third parties hanging on to everything in that way. These, of course, are burning questions of the moment, and ones to which we shall ourselves return.

Since our initial phone conversation on 28th Feb, Peter Tolmie and I have Skyped twice more – we seem to have got into a pattern of speaking every four weeks or so. In our second conversation, Peter pointed out to me that my PAWDOC filing system was just another manifestation of my inclination to keep things – as amply demonstrated in the various journeys documented in pwofc.com. He asked me what I thought I’d learnt from all these experiences, and I recounted a few things that immediately came to mind. Afterwards, however, I began to think that there were a great many more learnings dotted around the website. So I duly trawled through pwofc.com and recorded in a spreadsheet anything that looked like a finding. For good measure, I used another worksheet in the same spreadsheet to list all the requirements and findings specified in the paper about PAWDOC that was published in Behaviour & Information Technology (BIT) in 2001. I’ve given the spreadsheet to Peter and it will provide a base set of information for our investigations going forward.

My re-assessment of the BIT paper reminded me that one of the things I was thinking about when I wrote it was how one could use the key points in the documents you read to develop one’s knowledge. This idea stemmed from my practice of putting a line next to key points – or nuggets as I termed them – in documents. I remembered that I’d made a start on this work some 17 years ago by recording in a Mind Mapping programme the nuggets I found in books about the Pyramids etc. Peter and I discussed the possibility of my revisiting this material in a ‘Nugget Management’ journey sometime.

In our last Skype call on 25th April, Peter asked if I could keep an auto-ethnographic log of my keeping activities to provide us with more base material to draw on in our analysis activities. I duly created a spreadsheet with the headings listed below and am now recording all instances in which I make a specific effort to store a physical or digital artefact. The word ‘specific’ is used to exclude general keeping of things like email messages in email folders; and the word ‘artefact’ is used to explicitly require that a whole integral item is kept not just information removed from it like the name of a species from a plant label.

• Ref No
• Date
• Item
• How the instance arose
• Reason for keeping
• Initial actions and decisions made
• Actions taken

Peter’s comment on my request for his views on my recording scheme was “This is great. It’s not how I would have done it myself, but that doesn’t matter at all. The main thing is that it works for you. Just different work practices because we come from different backgrounds. Nothing more.”; and I doubt that I, on my own, would have come up with the idea of a generalised keeping log. Herein are clues as to the sheer unique and precious value of collaboration with our fellows.