Despite today’s widespread use of computers and email to create and distribute electronic documents, there has been little, if any, reduction in the use of paper in business. So, office workers  are likely to receive, annotate or even handwrite paper documents for the foreseeable future; and scanning remains an integral part of any filing regime. However, it is a much more straightforward activity than it used to be: scanners are now fast, effective and cheap and capable of digitising most documents. Contrast and brightness settings can deal with faint or over-emboldened documents; and colour scanning is no longer problematic – nearly all scanners can scan colour and the larger file sizes produced are no longer an issue as storage has become cheap and plentiful. Even large document sizes are no longer the issue they used to be because, if they are too large to be scanned on the equipment you have, they can simply be photographed with today’s high resolution cameras in mobile phones to produce easily readable images which can be enlarged at will.

The time it takes to scan a document is highly dependent on the model of scanner that is used and the size of document being scanned, so I can only provide timings for the particular scanner that I use – a 5 year old Canon DR-2020U A4 scanner. It takes roughly 1.5 – 2 minutes for a 1 page colour document; and roughly 2.5 – 3 minutes for a 5 page double-sided colour document. These are overall times which include preparing the paper, specifying the name of the file containing the scanned documents, and saving the file to a particular Windows folder.

Experience has shown that working with scanned documents is perfectly doable – the image quality on today’s screens is good and, in any case, the image can be enlarged if necessary. Of course, it is much easier to do so at a fixed workstation – perhaps with a large screen; having to work with scans on a laptop in meetings or while travelling is inevitably a little less comfortable, especially as it isn’t really possible to fit the whole of an A4 page in portrait mode on a typical laptop screen and be able to read it easily. More often than not users will enlarge the image so that only half a page is showing on the screen and then scroll up and down. Having said that, I have found that the iPad – and presumably other tablets – overcomes this problem. Whole A4 scanned pages can be presented in full on an iPad in portrait mode, and are very clear and fully readable. The iPad is light to hold and it is easy to navigate through multi-page documents; and it can be purchased with sufficient storage as to hold all the digital documents in a collection such as PAWDOC. This would seem to be the best way to put a digitised filing system to use – though great care would have to be taken to look after it as the very characteristics that make it so useable also make it extremely vulnerable to loss or theft.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q15. How long does it take to scan documents?

2001 Answer: Fully answered:

• About 30 seconds for a black & white page. (Chan 1993: 28)
• 40 ± 50 seconds for a black & white page (Wilson 1997: 3)
• 30 ± 50 seconds a page for dual journal pages on a flatbed scanner (Wilson 1996b)

2019 Answer: Fully Answered: Using a Canon DR-2020U scanner purchased in 2013 and a Chillblast i5 computer with 8Gb of RAM and 1Tb SSD drive purchased in 2018, scanning time for a 5 page double-sided colour document is about 70 seconds; and for a single colour page about 25 sec. However, this doesn’t tell the whole story because the scanning process also involves preparing the pages and storing the output. Approximate timings for all of the different process elements are as follows: a) Prepare the paper ie. remove any staples or postITs, get pages  aligned etc.. (5 seconds if there are no problems – but could take much longer) b) Start the scan – place the pages in the scanner, select the appropriate scanning job – Colour or B&W etc. – and press START (10 seconds), c) Perform the scan (70 seconds for 5 double sided colour pages, or 25 seconds for a single colour page), d) Create the file title – paste the Ref No (which had been copied in the indexing process) into the file title box and type in the rest of the file title  (15 -30 seconds), e) Save the file – select the folder into which the document is to be placed and press SAVE (20 – 30 seconds), f) Check the folder – check that the file has been stored in the correct place with the correct title (15 seconds); g) Check the file – open the file to check it has been scanned correctly (only for multi-page documents or for documents where there may be an issue) (15 seconds).

In summary, overall scanning time for a 1 page A4 colour document is roughly 1.5 – 2 minutes; and for a 5 page double-sided A4 colour document about 2.5 – 3 minutes.

Q16. What are the practical problems associated with scanning?

2001 Answer: Partially answered:

• Scanning double-sided pages is time-consuming and error prone (Wilson 1995b: 121, 1995c: 1).
• Paper jams and misfeeds sometime occur when using the scanner sheet feeder (Wilson 1995c: 1).
• Some documents need preparation work prior to scanning, for example, removing staples or guillotining bound documents (Chan 1993: 34 ± 37).

2019 Answer: Fully Answered: Practical scanning problems are of two types – those that are inherent to the scanning activity; and those that are due to shortcomings in the scanner being used. The main inherent scanning problem is the need to ensure that the papers being scanned are free from staples, PostIT notes and any other additional items which will jam as the paper passes through the scanner. Problems due to scanner shortcomings may be due to a lack of capability in the scanner concerned such as no sheet feeder; the inability to scan large pages such as A3 (A3 scanners are available but they’re big and more expensive); and inability to scan double sided pages (a particular problem I had when I started scanning but I’m glad to say that my current scanner scans both sides of a page at the same time as it makes a single pass through the scanner). Scanner shortcomings may also include capabilities in which technology developments are continually making inroads, for example, sheet feeder technology (hugely better in my current scanner – but multiple pages are still pulled through occassionally); and dealing with very faint text (I would hope that future software will automatically adjust brightness and contrast settings to deal with this). Finally there may be shortcomings in a particular model of scanner. I believe this is the case in the software driving my Canon DR-2020U which seems incapable of scanning a full page which has continuous black or very dark areas on the edge of the page – the scan truncates the page to remove those areas. To deal with this I have to cover the relevant edges with white paper. This  may have been resolved in a later release of the software – though I haven’t checked for some time.

Q17. What are the practical problems of scanning documents containing colour?

2001 Answer: Ideas formed:

• Colour scans take more time (Wilson 1997: 2).
• Some colours scan black when using a black and white scanner (Wilson 1995b; 76, 77, 98)
• Colour scans produce files 10 or 20 times the size of a black and white scan (Chan 1993:43, Wilson 1997:2)
• Colour scans (which are made up of red, green and blue) may not work well on printers (which use cyan, magenta, yellow and black) (Chan 1993: 44)

2019 Answer: Fully Answered: Colour scanning no longer presents a particular problem because today’s scanner all have a colour capability, and even though colour scans take up more space, storage is now plentiful and cheap (for comparison purposes, with my current scanner, a single page with 5 colour photos on it scanned at 300dpi B&W to a PDF of 126 Kb; and at 300dpi Colour to a PDF of 801 Kb).

For the most part, colour scans are normally sharp and clear. The only problem I have had with colour scanning seems to be specific to the software used by my current 5 year old Canon DR-2020U scanner. I’m very pleased with the performance of this scanner except for the fact that it truncates those parts of the edges of pages which have black or very dark colour. To deal with this, I have to place white paper just over the edges of the page – which can be difficult for larger pages which go up to the edge of the scanner platen. I have never experienced this problem before and hope that it has been fixed in a software upgrade.

Q18. Can poor originals be successfully scanned?

2001 Answer: Experience gained: Modifying the scanner settings can dramatically improve the quality of scans of poor originals.

2019 Answer: Partially Answered: Originals can be considered poor for a variety of reasons including skewed images, holes punched in the sides for binding purposes, and faint or heavily smudged images. My current scanner has on/off features to deal with skew (works well) and punched holes (I haven’t really tried to use this); and has settings to adjust contrast and brightness (these also work well but I’ve found it difficult to remember which combination of settings work best – perhaps in the future more automated assistance will be provided for this). In summary, usable scans can often be achieved from poor originals.

Q19. Can all types and sizes of documents be scanned successfully?

2001 Answer: Experience gained: Yes, provided you have a flatbed scanner with a sheet feeder. If the flatbed is not big enough, many of the larger documents can be cut or folded to the necessary size. The spines of gummed or stapled newsletters and booklets can be cut off (Wilson 1995b: 99). Post-it notes stuck on a page go through the sheet feeder without a problem (Wilson 1995b: 13, 39, 140). Journals can be scanned successfully on a flatbed scanner (Wilson 1995b:124).

2019 Answer: Fully Answered: As described in the 2001 answer, the vast majority of documents can be scanned successfully provided you have a scanner which can accommodate the physical size of the document. Of course, if you can’t lay a document flat on the scanning platen, as is the case when trying to scan a book or a journal, there will always be some distortion at the centre of the dual pages.  Most people are likely to have an A4 or, in some cases, A3 scanner  (though an A3 scanner does take up more space) so that they will be unable to scan documents above these sizes; However, that is of no consequence these days because digital photography is now so cheap, and powerful, and prevalent on mobile phones, that it can be used to photograph documents that are too big or difficult to scan. The digital images produced are good enough to read and can be enlarged as necessary. A good example of this is the increasingly common practice of taking photos of flipcharts of which there are a few in the PAWDOC collection. However, there are a few potential problems with photographing documents or other flat objects: one is that, unless the camera is held in exactly the same plane as the document page, the image of the document will appear shorter on one side than on the other; and the other is that, if the document will not lie flat, the document image will appear equivalently distorted in the photograph.

Q20. How can you minimize the amount of scanning that needs to be done?

2001 Answer: Experience gained: Possible strategies include:

• Stick with electronic files as much as possible
• Minimize the printing of paper (Wilson 1995b: 82, 90).
• Elect to receive electronic versions of newsletters, magazines and journals (Wilson 1995b: 126).
• Go out of your way to obtain electronic versions of paper you get from other people (Wilson 1997: 3).

2019 Answer: Fully Answered: Despite paper still being very widely used (the paperless office never actually arrived), most documents are now created and distributed in digital form – very little paper arrives in the overland or office mail today. Hence, scanning can be minimised by simply storing the electronic versions of documents you create and that you receive by email; and by seeking out the electronic versions of other hardcopy document that you receive. For example, I get the quarterly newsletter of the professional body that I belong to in hardcopy form; but I go to their website to get a digital version of the magazine. Having said that, journal and magazine publishers continue to protect their papers and articles behind paywalls; and newspapers seem to be following their example. Consequently, filing system owners may have to scan some hardcopy papers and articles for the foreseeable future.

Q21. Can scanned images be used successfully in day-to-day work?

2001 Answer: Ideas formed: Relatively few of the scanned images have been used to date. However, experience so far has not indicated any major problems with either reading them on screen or printing them out. An A4 sized display would certainly make it easier to read scanned images on screen (Wilson 1995b: 104).

2019 Answer: Fully Answered: Although I don’t access many of the PAWDOC scanned images these days, over the years I have used them and worked with them and have had no trouble doing so. Nowadays I view all my scanned documents (in both multi-page TIFF and PDF formats) in my eCopy PDF Pro application and this works fine – though it would undoubtedly be better if the screen could be in portrait mode so that the whole of an A4 page could be displayed in the way one reads the hardcopy. Recently I transferred several scanned documents onto my iPad and I’ve found I can read them all without any difficulty with the iPad in portrait mode and without needing to enlarge them. This has been a bit of a revelation since it is clearly a much easier and more pleasing experience to read these documents on the iPad than it is on my 24in desktop screen or my laptop. The combination of the iPad’s high resolution screen, it’s light weight, the fact that you can hold it as close as you want, make it a winning combination for scanned work documents. This could be the way to store digital filing systems in the future – or perhaps large screen phones could serve the same purpose?

Indexing is a key aspect of a filing system because it is one of the mechanisms used to find documents. An alternative mechanism is to search the full text of documents, but the PAWDOC system has not explored this approach because a) the PAWDOC system does not recognise the full text of all the documents it contains, and b) because I have always believed this approach would provide too many hits thereby reducing search effectiveness.

Indexing within the PAWDOC collection is performed at two levels: first, each entry (uniquely identified by a Reference Number) in the main index file contains a Title field (in which a free format description and any number of uncontrolled keywords can be specified) and a few other fields including publication date. Second, each electronic file (be it a scan of a hardcopy document, or a born-digital document) associated with a particular Reference Number contains a unique file title. The contents of these two sets of indexing information provide the means to search for a document, to decide if you have found what you are looking for, and to get a better idea of what the chosen Reference Number and/or associated electronic file(s) contain. These potential uses are worth bearing in mind when indexing information is being specified.

Inevitably mistakes are made when specifying Title information and so it is essential that index entries can be changed over time to correct typos, grammar and factual errors as necessary. Changes may also be inspired by experiencing difficulty in finding a document, and, in these cases, some additional text may be included to reflect the search terms that first came to mind, or to reflect the changes in terminology that occur regularly in language and in today’s fast moving technical and cultural environment.

Indexing is closely integrated into the process of including a new document in the PAWDOC collection, not least because adding a new file involves creating a file title which in turn includes a Reference Number and possibly some text from the associated Index entry. Hence, timings for how long indexing takes are embedded within the overall process of Accessioning (which includes scanning for hardcopy documents). For the PAWDOC collection, which uses a scanner acquired in 2012 and a laptop acquired in 2018, it takes approximately 2.5 – 4 minutes to accession a single colour A4 hardcopy page; and about 3.5 – 5 minutes for a 5 page double-sided colour hardcopy document. An electronic file of any type and number of pages takes approximately 2 – 3.5 minutes to accession. These are appreciable amounts of time for an overhead administration activity amidst a busy working day. Therefore, it is worth exploring any means of reducing these timings. The obvious way is to keep the number of index fields to an absolute minimum. Experience with the PAWDOC system indicates that integrating keywords into the Title field (as opposed to having a separate keyword field) has been very successful. Conversely, the Publication Date field has not proved to be very useful and I believe could be dispensed with. Experience with the other fields used in the PAWDOC system are summarised below:

• Reference Number – essential
• Creation Date – a useful control
• Movement Status – very useful for recording the whereabouts and status of documents
• Date Last Accessed – only useful if you particularly need it.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q10. How long does it take to index a new document?

2001 Answer: Fully answered: It takes 1 -2 minutes to make an entry in the main index and to write the Reference Number on the document. A further 30 seconds is needed to create an entry in the document management system for either an electronic file or a document that is to be scanned immediately (Wilson1995c: 1). Scanning a document will add a further 40-50 seconds for a single page – though the per-page time is reduced considerably for multi-page documents that are put through the sheet feeder (Wilson 1997: 3).

2019 Answer: Fully Answered: This question should really be ‘How long does it take to index a new document and include it in the collection?’ because the indexing process is closely integrated with the process of saving the file. The figures in the 2001 Answer for making a new entry in the main index and writing the Reference Number on the document (1-2 minutes), and for creating a new entry in the document management system (an extra 30 seconds) still stand.  Scanning a document using the Canon DR-2020U scanner I bought in 2013 and the new Chillblast laptop I bought in 2018 with a 1 Terabyte solid state drive, takes 25 seconds for one colour page or 70 seconds for 5 double sided colour pages. However, the Document Management system was eliminated from the PAWDOC architecture in 2018 and replaced by Windows folders, so the subsequent set of actions has changed to the following: a) Create the main index entry (1-2 mins); b) Create a sub-folder within the main PAWDOC folder in the Windows file system, with the new Reference Number and copy the Ref No (15 seconds); c) create the file title (Ref No   Title,  Date scanned) in the scanner control application starting by pasting in the copied Reference Number (15 – 30 seconds); d) Save the file – use the scanner control application to select the new folder just created to specify where the file should be stored and press SAVE (20-30 seconds); e) check the file – open the file to check that it has been stored in the correct place with the correct title (15 seconds); f) Check the folder – open the file to check it has been scanned correctly (only for multi-page docs or for docs where there may be an issue) (15 seconds). The process for electronic documents has also changed as follows: i) create the main index entry (1-2 mins), ii) create a sub-folder within the main PAWDOC folder in the Windows file system, with the new Reference Number and copy the number (15 seconds); iii) Create the file title – open the SAVE AS dialogue box and give the file a new file name (Ref No   Title,  Date saved) by first pasting in the copied Reference Number (15 – 30 seconds); iv) Save the file – navigate to the folder into which the file is to be saved and press SAVE (20 – 30 seconds); v) Check the file – open the file to check that it has been stored in the correct place with the correct title (15 seconds). Of course, electronic documents which are placed into an existing Ref No have no need to create a new index entry or a new folder.

In summary, overall timings for including new documents in the collection are as follows: for a 5 page double-sided hardcopy document using a new Ref No – approximately 3.5 – 5 minutes; this may reduce to about 2.5 – 3 minutes if an existing Ref No is being used. For an equivalent 5 page electronic document using a new Ref No – approx 2 – 3.5 mins; this may reduce to 1 -1.5 mins if an existing Ref No is being used. Timings for a hardcopy document with only 1 page of contents are reduced by about 45 seconds due to reduced scanning time; electronic document timings stay the same regardless of page count.

Q11. What can you do to speed up indexing?

2001 Answer: Ideas formed: Keep the number of index fields to a minimum.

2019 Answer: Fully Answered: Minimise the number of fields. Automate the generation of the Creation Date. Eliminate the Date Last Accessed field if there is no good reason for recording that information.  To speed up the overall accession process, a) store more rather than less documents in existing Reference Nos (using existing Ref Nos avoids having to create new index entries); b) use a faster scanner; c) reduce the time it takes application programs to page through 17,000+ sub-folder names in the SAVE AS function (for some reason, in my Windows 10 system, the application programs take far longer to do this than Windows File Explorer).

Q12. What is the most effective set of index fields?

2001 Answer: Experience gained: The smallest number you can manage with (Wilson 1990: 95). Reference number, title and keywords, date of creation of the record, movement status, date last accessed.

2019 Answer: Fully Answered: Keeping the number of fields to the absolute smallest number you think you can manage with will minimise the time and effort spent on putting new items into the system and on managing the system. The minimum number of fields I could manage with are Reference No, Title, Creation Date, and Movement Status. The other two fields that I use, but which I believe I could do without, are Publication Date (I rarely refer to this) and Date Last Accessed (which was included mainly for research purposes).

Q13. What criteria should be employed when defining titles and keywords?

2001 Answer: Ideas formed:

• Remember there is no need to use the actual title of the document.
• Define titles/keywords in accordance with what the document means to you – that way you are more likely to be able to retrieve it.
• Bear in mind that title/keywords serve multiple purposes (Wilson 1992a: 8):
  • To enable search and retrieval
  • To enable the user to decide if a retrieved record is what was being searched for
  • To provide the user with an understanding of what is contained in the item referenced by a retrieved record.

2019 Answer: Fully Answered: Do not feel constrained to use the actual title of the document – you are far more likely to be able to specify successful searches for items if you define Titles and Keywords which convey what the document means to you. If you are aware terminology is changing, use the latest terms you know as you are likely to become less familiar with the old terminology as time passes. Since there are no length constraints on the Index entry, use as many words as necessary to describe what the document is about. The second level of indexing – the text in the Titles of individual files – is constrained by length and for that reason might well be shorter than the main index entry. However, care should be taken not to truncate unnecessarily and not to just take the quick and easy route to create the file title by cutting and pasting a generic part of the Index entry. Certainly, where there are two or more files associated with the same Reference No, the titles of the 2nd and subsequent files should clearly distinguish them from all the other files in that Reference No. Also, bear in mind that Title fields (which include Keywords) and File Titles serve at least the following three purposes: a) to enable search and retrieval; b) to enable the user to decide if a retrieved record is what was being searched for; c) to provide the user with an understanding of what is contained in the item referenced by a retrieved record.

Q14. Are there circumstances in which keywords and titles should be changed over time?

2001 Answer: Experience gained: It is very necessary to be able to alter keywords and titles for the following reasons:

• To accommodate the user’s growing familiarity with a topic. For example, the term `OA Human Aspects’ became an abbreviation `OA HCI’ in later entries, so it needed to be placed in previous entries to ensure they would be retrieved if only `OA HCI’ was specified in the search term (Wilson 1992a: 22).
• To accommodate changing language. For example, the term `DTI pilot’ became `DOI pilot’ when DTI changed its name (Wilson 1992a, 12).
• To accommodate additional material being added to an existing hardcopy file, or being added to the associated Folder in the Document Management System.

2019 Answer: Fully Answered: Users should be able to change Titles and Keywords at will to correct mistakes or to try and improve the search success rate. Circumstances in which changes might be made for the latter reason include:

a) to accommodate the user’s growing familiarity with a topic. For example, the term `OA Human Aspects’ became an abbreviation `OA HCI’ in later entries, so it needed to be placed in previous entries to ensure they would be retrieved if only `OA HCI’ was specified in the search term;

b) to accommodate changing language. For example, the term `DTI pilot’ became `DOI pilot’ when DTI changed its name; and

c) to accommodate additional files being added to the Reference No.

Most types of things can be managed in a digital filing system. Documents, brochures and books can be either scanned or photographed; electronic documents are, of course, already in the right form; and even physical artefacts such as, for example, art works, can be photographed.

Once it’s been decided to establish a filing system, the reasons why the items concerned are being kept, and what is going to be done with them, should be written down. This should help in defining the following three key points: i) how long items are to be kept for; ii) whether any physical originals are to be retained after they have been digitised; and iii) whether every item is to be filed or just a particular subset. Point iii) is important because, if only a subset is required, the amount of time and effort that will be spent on adding items to the filing system, and managing it, will be reduced. Against this time saving must be weighed the extra effort – and potential errors that may be made – in deciding whether each particular object is to be included or not. Conversely, collecting every instance of a type of an item makes life easy and requires no decision making.

If the filing system is to replace or augment an existing collection, a decision must be taken about whether to exclude the existing items, or to undertake ‘backfile conversion’ to include some or all of them in the digital collection. Backfile conversion is often a lengthy and tortuous process and can act as a drag on proceeding with the new system; therefore, it should be avoided at all costs. If, however, it is thought to be needed, the reasons for doing it should be clearly understood before proceeding.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q5. What types of information and media can be managed by an electronic filing system?

2001 Answer: Fully answered: Just about anything can be managed by an electronic filing system including single sheets and stapled sets of hardcopy documents, scientific journals, volumes of bound material, ring binders, books, 35 mm slides, videos, electronic files and e-mail messages.

2019 Answer: Fully answered: Any kind of item can be managed by an EFS – though it may be necessary to also keep the original physical artefact to experience the full impact of certain types of items such as specially folded brochures, or significant documents with a tactile presence. Types of items that I have succeeded in managing in the PAWDOC collection include hardcopy documents, ring binders, books, journal papers, magazines, brochures, overhead slides, flipchart pages, CAD/CAM drawings, photos, videos, web pages, and email messages.

Q6. Can all types of electronic files be managed in an electronic filing system?

2001 Answer: Partially answered: Yes, certainly all types of application files can be. I have not stored executable files yet but have been advised that it can be done and how to do it.

2019 Answer: Fully Answered: Any types of file format – including executables – can be managed in this way. For example, file types in the PAWDOC system include Word, Excel, Powerpoint, PDF, TIFF, JPG, MP4, MPP, ZIP, and HTML. However, the real issue is whether you will be able to open them in future years if the particular application programmes are lost or not kept up to date. For this reason, it may be worthwhile converting some files to more common formats, such as PDF, before storing them. In the future, these so-called ‘Digital Preservation’ issues may get addressed automatically by AI programmes.

Q7. What criteria should be employed when deciding what to file?

2001 Answer: Not started

2019 Answer: Fully Answered: Individuals should file as much or as little information as they think they will need in their jobs – otherwise they will not have the motivation to spend the time introducing new material and managing the filing system. If a robust, all embracing, file facility is available within a particular system (such as in many electronic mail systems) such systems should be used to their full capability (i.e. retain everything up to any limitations set by the system administrator). Inevitably, whatever an individual decides to file, there will always be some material which is never subsequently accessed, and there will always be some required information that can’t be found in the system. This seems to be a regularly occurring phenomenon and should not deter individuals from filing – though they may influence the filing criteria that an individual applies over time.

Q8. Is it worth doing backfile conversion?

2001 Answer: Experience gained: Backfile conversion is certainly not essential, and is probably not worth doing, except for material you definitely want to keep for many years.

2019 Answer: Fully Answered: Backfile conversion usually takes a huge amount of time and effort, so a detailed analysis should be undertaken before deciding whether to do it or not. Alternative approaches to dealing with the old items should be investigated. Backfile conversion should only be carried out if there are compelling reasons for doing so.

Q9. What are the major considerations when carrying out backfile conversion?

2001 Answer: Experience gained: Backfile conversion requires a huge amount of time and effort and is not to be undertaken lightly.

2019 Answer: Fully Answered: Considerations to be taken into account when conducting backfile conversion include:

1. Ensure the physical backfile material does not interfere with any space required by the replacement filing system.
2. Decide if any of the backfiles being converted will need to be retained in their original form and, if so, define clear criteria for selecting which ones to keep.
3. Decide what quality the converted backfiles need to be and then put in place the appropriate equipment, software and procedures to achieve that quality.
4. Decide how users will be informed of which backfiles have been converted and which are yet to be converted.
5. Establish a clear and doable schedule for the conversion process.
6. Put in place motivational aids for sticking to the schedule. For example, setting a target of a certain number of conversions each day; and creating a visible progress sheet on which achievements can be ticked off.

The PAWDOC collection was set up in 1981 to explore the application of office technology. In 2001 a paper was published in the Journal Behaviour & Information Technology called ’20 years in the life of a long term empirical personal electronic filing study’. This described PAWDOC and summarised findings about its use up to that point under the following 15 headings:

• Collection content, size, growth rate and usage patterns (this post)
• Deciding what to file
• Indexing
• Scanning
• Searching
• Use of information
• Archiving
• Hardcopy/electronic mix
• Relationship between work patterns and filing activities
• Technology requirements and problems
• Portability
• Sharing files
• Confidentiality, ownership and intellectual property rights
• Reliability and Longevity
• Costs and benefits

Two extra considerations are ‘Architecture‘ and ‘Requirements and Objectives‘. Now, a further 18 years on, the findings will be further reviewed in this and subsequent posts. The first of these follows below.

Collection content, size, growth rate and usage patterns

The precursor to PAWDOC was a conventional filing system in an upright cabinet using hanging folders and crystal tabs to store documents only (not journals, books etc.). The contents were specified in an index taxonomy with entries of the type, for example, 1.6.8 – Quarterly Progress Reports, which I was constantly adding to. Consequently much space was taken up in the cabinet by folders with crystal tabs housing only a few pages.  This changed after the PAWDOC system came into use: it needed far fewer hanging folders because each folder was filled to capacity with as many of the serially numbered documents that it could take.

The PAWDOC schema was explicitly defined to support the management of multiple sets of different material owned by multiple different owners. This capability was used to manage a variety of sets of my own material in addition to documents, for example, several different journals, 35mm slides (for making presentations), ring binders, and books; and, in the early years of the system, material owned by other people and organisations – though, as time went by, I made fewer and fewer such entries not least because of the uncertainty of being able to access such material.

The system soon became an integral part of my working life, and I used it just about every day. Several hundred new items were being added each year and this steady rate of acquisition soon ran up against the physical limits of the upright cabinet, so it became necessary to archive material in boxes and then to put some of the boxes in store. This went on until I started digitising newly acquired documents in 1996 and disposing of most originals. At this point, I also started to store born digital documents regardless of the applications they were created in.

In 2001 I started a new job in Bid Management which precluded personal storage of its associated highly confidential and fast moving documentation; so my useage of the PAWDOC system did reduce from then onwards. Nevertheless I continued to use it regularly on most days, and was still adding over 200 new items each year up to when I retired in 2012.

The 45 boxes of hardcopy that I had acquired eventually came out of store around 2001 and were stored in my garden shed. Their number was overwhelming and I began to doubt I would ever get them all digitised. However I stuck at the task, sometimes going at it solidly for several days at a time when my wife was away. By the time I retired in 2012 only 4 boxes remained and these had all been scanned by 2014: it was a great relief. The huge amount of physical space taken up by the collection had been reduced to just two archive boxes of significant hardcopy documents; and, after I had conducted a digital preservation exercise on the collection in 2018, the digital footprint of the collection amounted to some 115Gb. I have ended up with a fairly complete digitised archive of all the non-highly confidential materials that I had encountered throughout my working life, including substantial amounts of material from my earlier career from 1972 with Kodak and then CPC. Since retiring I’m continuing to add a few documents (around 40 up to 2019) in three categories: significant articles relating to the work I used to do; documents relating to the digital preservation of the PAWDOC collection; and material relating to work I am doing to investigate and document the findings from the PAWDOC collection’s 38 years of existence.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q1. What are the contents of the collection?

2001 Answer: Fully answered: At the beginning of July 2001, the collection consisted of 14,100 index entries representing approximately 185,000 pages of paper, 50,000 scanned pages, over 30 scientific journals (including Behaviour & Information Technology from 1982), around 30 books and conference proceedings, 3700 MS Word files, 400 MS Excel files, 250 MS PowerPoint files, 150 other electronic files of various types, and 10 CDs.

2019 Answer: Fully answered: The PAWDOC user Guide created in 2018 says: “All types of documents were stored including letters, internal memos, circulars, reports, specifications, minutes, overhead slides, 35mm slides, notes, training materials, brochures, manuals, maps, emails, computer magazines, journal articles, conference proceedings, and videos. As Office Technology became more versatile, electronic documents such as word processor files, spreadsheets, presentations and web sites were also filed.”. In June 2019, the collection consisted of 17,293 Index entries representing 29,610 electronic files in 16,067 Windows folders, and about 340 physical hardcopy documents in two archive boxes. A further 384 old electronic backup files are also stored in a separate folder. The The checking exercise conducted in 2016 identified the following numbers of different types of files in the collection: Word – 6380; Powerpoint – 466; Excel – 625; HTML – 382; Help – 90; Zip – 92;  11 other apps – 88; Scanned documents – 28,418. The collection was primarily a work collection and therefore the number of new items being included reduced to a trickle when I retired in 2012.

Q2. How much space does the collection take up?

2001 Answer: Fully answered: The paper takes up about 4.7 sq. metres of floor space and 1.7 metres of shelf space. The scanned images and electronic files take up 2.9 GB. The scanner, magneto-optical drive and CD Writer take up about 0.25 sq metres of desk space. The Filemaker Pro index is about 8.1 MB in size and the FISH data file is 11.2 MB. The Filemaker Pro, FISH, SQL Anywhere and Easy CD Creator software packages take up approximately 38 Mb.

2019 Answer: Fully answered: The two archive boxes stand one upon the other and take up 0.2 sq m of floor space. The laptop in which the electronic files reside takes up 0.08 sq m of desk space. The electronic files of the main collection take up 45.9 Gb storage space; and the backup files take up 66.6 Gb. The Filemaker Pro software used for the index takes up 336 Mb of file space.

Q3. What is the growth rate of the collection?

2001 Answer: Partially answered: Between 1981 and 1993 an average of 543 index entries were created each year – estimated to consist of an average 29.4 pages per day (Chan 1993:25).Over the whole 20 years life of the system, the growth rate has been an average of 705 entries per year with a range of 210 ± 1202new entries a year. In 1993, it was estimated that the collection was increasing at the rate of 3.8 MB per day.

2019 Answer: Fully answered: The growth rate of the collection is shown in the chart below.

There are three distinct phases: 1981 – 2000; 2001 – 2011 when I was working on highly confidential bids; and 2012 – 2019 when I was retired. The average growth rates per year during these periods were:

• 1981-2000 – 696
•  2001-2011 – 272
• 2012-2019 – 46

Q4. How often are the contents accessed?

2001 Answer: Experience gained: Between 1987 and 1993, an average of 363 records were being accessed each year (Chan 1993: 25)

2019 Answer: Experience gained: The only data that has been collected on this question is in the date last accessed field, and unfortunately that only records the latest date an item was accessed – there may have been any number of earlier accesses. Furthermore, some items may have been accessed without the date last accessed field being updated. Having said that, 4,551 items have an entry in the date last accessed field, implying that 12,742 items have never been looked at for work purposes after they had been included in the collection (the date last accessed field was never updated when items were looked at for the purposes of controlling and writing about the collection). For the period from 2001 onwards, when I moved jobs into Bid Management, there were 751 index items with dates of 2001 or later in the date last accessed field (only 15 of these were in 2012 and only a further 15 of these were from 2013 onwards).

NB. The various references in the texts above to Chan,1993 relate to the following reference at the end of the 2001 paper in Behaviour & Information Technology:

CHAN, S. C. 1993, Feasibility of Paperless Office, Submitted in partial fulfilment of the requirement for the degree of MSc in Information Systems and Technology in The Information Science Department at City University, London (Supervisor: Dr David Bawden) [PAW/DOC/4012/08].

This entry has been jointly authored by Paul Wilson and Peter Tolmie

The PAWDOC filing system was set up in 1981 to try and understand how the newly emerging office technologies of that era might assist individuals to manage their office documents. Over its 35+ years of operation much has been learnt, and it is our intention to try and understand and describe those findings. However, the system was set up to address requirements in the office of the 1980s, and there has been a revolution in the way business operates since then. In order to be able to relate the findings to office work today and in the future, this entry explores the differences in  requirements for personal filing in the office between the early 1980s and 2019.

Perhaps the most significant difference is the transfer of huge amounts of information from paper-based documents to digital files. Note that this is not saying anything about the current volume of paper in the office (though we would suggest that there is probably less paper filed by individuals now than in the 1980s) – just that individuals now have to deal with huge amounts of electronic material in contrast to the early 1980s when they dealt with virtually none. This transition has been facilitated by a huge growth in the use of computer hardware – desktop computers, laptop computers and mobile phones – throughout the world, from a base of zero to near-universality.

The growth in the use of computers has also prompted huge changes in office work. At the beginning of the 1980s, office professionals used support staff – typing pools, secretaries, and admin support staff – to perform their administrative tasks.  However, as office technology became widespread, professionals started to do their own typing and support staff became a luxury which could be cut to reduce budgets. In 2019, only very senior management have secretaries and office workers are expected to be self-sufficient and fully competent in the use of all hardware and software relevant to the realisation of their work.

These changes were accompanied by a revolution in communications. Electronic mail has now almost entirely replaced internal memos and external letters, and has prompted massive increases in the amount and speed of communication. Email also rapidly became the key mechanism for supporting distributed teamwork – nationally and globally – and now underpins a battery of related interests, from the sharing of documents to the organisation of voice conference calls (which are the unsung foundation upon which much of business and government now operates). In more recent years, as mobile phones have permeated throughout the world’s populations, text messaging and chat applications have become an integral element of personal and business relationships. To this highly significant mix of new technologies must be added the recent massive uptake in Social Media. An unfortunate side-effect of the sheer effectiveness and pervasiveness of these mechanisms is high levels of information overload across a large proportion of office workers.

Importantly for the work to derive findings from the long-term operation of the PAWDOC filing system, the changes described above have impacted filing activities in the office. Hot desking and home working have made personal filing cabinets and bookshelves a luxury. The folder systems integral to computer operating systems (primarily from Microsoft and Apple) are now used to store the electronic documents created and received by the individual. At the same time, email systems have their own integral filing systems into which mail can be rapidly sorted and stored indefinitely in the cloud; text messages are stored on users’ mobile phones in the form of text streams by both senders and recipients; and Social Media systems have their own self-contained environments distributed across vast computing networks. The further evolution of cloud-based repositories, such as Dropbox and Google Drive has led to an added utilisation (if not trust) in distributed document stores. Even if users wanted to integrate these different collections, it would be almost impossible for them to do more than just copy selected elements from one to another or to a dedicated filing system: these stores are separate silos and will probably continue to be so for many years to come.

The design of the PAWDOC system in 1981 was based on an understanding of office filing requirements at the time. There was an expectation of how emerging office technology might be used to support those filing requirements, but little appreciation of how the technology itself would change the way business operates. Initially, then, learnings from the development of the PAWDOC system were entirely focused upon what the impact might be of new assumptions about filing built into the construction of computer systems in the early 1980s. Later on, in the middle period of PAWDOC operation, the findings speak to what it was taking to manage a filing system in a changing work environment populated by imperfect but maturing technologies. More recent findings give a somewhat different picture, as many of the troublesome technologies of the middle-era have come to be taken-for-granted resources, giving rise to new kinds of problem, of which information overload is but one potential symptom. What is clear at present is that computer technology and the business world is now changing so rapidly, the presumption present in the early days of PAWDOC – that one could readily identify needs and solutions for the future – now seems somewhat naïve (if still just as pressing).

One thing, however, we believe has remained constant and that is the attitude towards filing across the population. Most people are not motivated to put effort into filing because it is extra work for an indeterminate reward at some undetermined point in the future. A smaller subset of people is willing to put varying degrees of effort into the activity. We believe this has changed little between the early 1980s and the present day. As it happens, the PAWDOC owner was at the more extreme end of this latter group and wanted to file both effectively and comprehensively. Hence the PAWDOC collection contains most of the documents that the owner read and/or believed to be significant in his work; and consequently it should be borne in mind that the learnings derived from his experiences concern almost the worst case requirements of filing load and effort. It should be easier for most of the population. Certainly, it would seem easier, for digital copies are now retained of virtually everything as a matter of course. The extent to which that is oriented to as a personal collection of materials is a different matter, as is the probity of third parties hanging on to everything in that way. These, of course, are burning questions of the moment, and ones to which we shall ourselves return.