PAWDOC – Use of Information

The practice of sidelining text in articles, papers, and books is not uncommon and is something I started doing in the late 1970s – primarily to assist the writing of technical books when I was working at the National Computing Centre. I started to use my PAWDOC filing system in 1981, so, by the late 80s I was aware that a) there were a lot of documents in my filing system that I hadn’t looked at for several years, and b) inside all these documents were a large number of sidelined significant points. I started to think about these points as nuggets of information, some of which perhaps were the bedrock of my developing ideas, but others of which perhaps I had simply forgotten. I wondered also if an explicit examination of all these nuggets might prompt inter-relationships to be identified and new ideas to be developed. I considered this to be a potentially valuable spin-off from all the effort expended in operating a very comprehensive personal filing system. The notion of actually making use of the information that PAWDOC contained instead of having most of it just lie there statically, was very attractive.

Consequently, I looked for some simple and inexpensive tools to try out these ideas, and came across Mind Mapping software on the free disks issued with PC magazines in late 1990s. I started experimenting with one of them but found it was too bitty just pulling nuggets out of the odd paper – I felt I needed a whole set of material to work with. So I created Mind Maps for 19 esoteric books (on subjects such as The Great Pyramid), but found it too difficult to inter-relate the different Mind Maps. That’s as far as I got.

I’m still not sure if there is any merit in explicitly managing nuggets to either just cement them in one’s mind, or to inter-relate them and develop new ideas; and, I must admit, I’ve never done any serious book review to see what other work, if any, has been done on this subject. The retrospective work that Peter Tolmie is planning to do with me may throw a little light on the impact that such nuggets may have had on me – but that’s as far as it goes. My current view is that explicitly managing nuggets is probably not worth doing, and that adding extra tasks to the job of managing a personal filing system may well be the stone that breaks the camel’s back.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q27. How can an electronic filing system be used to develop and use knowledge?

2001 Answer: Ideas formed:

  • Include substantive information in the index entries, for example phone numbers, book references, and expense claim amounts.
  • Identify the nuggets of information (i.e. the valuable bits) when you first read a document (Wilson 1997: 3 – 4).
  • Capture and structure the nuggets into the overall nugget-base at the same time as indexing the item (Wilson 1997: 3 – 4).

2019 Answer: Ideas formed: The first point in the 2001 answer to this question (‘include substantive information in the index entries’) has proved useful: so much so that I started including substantive information in the file names of digital documents (for example, total amounts claimed in the file names of expense claim spreadsheets).

With respect to nuggets, those ideas emerged from my practice of sidelining text that I thought significant, I developed the idea that these pieces of information (which I called nuggets) might be picked out, recorded and combined with other nuggets to produce novel ideas and concepts. I explored technology options that might assist in this process and decided Mind Mapping software might be worth trying, and tried it out with a variety of esoteric books that I was reading at the time. Although I Mind Mapped 19 books I never took it to the next stage of combining them to see if I could develop any new concepts – there seemed to be no easy and effective way of doing so. That is as far as I got with this notion. I’m hoping that the experimental work I’m planning to do with Peter Tolmie on this subject might indicate if there is any merit in exploring these notions further or not.

Q28. What is the best way to capture and structure information nuggets?

2001 Answer: Ideas formed: By using a Concept Development tool. Some initial prototyping has been done using the Visual Concepts package and the eMindMaps package.

2019 Answer: Experience gained: I explored the use of Concept Development tools for this purpose by using the eMindMaps tool to capture nuggets from 19 separate esoteric books (on subjects such as The Great Pyramid). I found that, although it was probably quite a good way of summarising a book (or article or paper) on one page, there was no easy or effective way to combine several mind maps together or to relate an item on one mind map to an item on another mind map. So I concluded that such tools were not going to be an effective nugget management solution. I’m not intending to explore this any further, however, if I did, I would look into the collaborative concept development tools that I know were being explored by the CSCW community in the 1990s and from which commercially available software might have emerged by now. An alternative, much more feasible, solution might be simply to accumulate the nuggets in a spreadsheet. I guess the question of what tool to use is very much tied up to what one wants to do with the nuggets and what benefits can be achieved by working with them.

Q29. Is it feasible and practical to capture and structure information nuggets as well as indexing items?

2001 Answer: Not started

2019 Answer: Not started

Q30. Is it worthwhile building and developing an information nugget base?

2001 Answer: Not started:

2019 Answer: Ideas formed: I’ve always thought there was value in the nuggets I sidelined in articles and papers – which is way I started exploring this topic in the first place. However, whether there is any value in working with them in any way at all (either just accumulating them in a spreadsheet to cement each point in one’s mind, or inter-relating them in a specialist tool to develop new ideas) is still an unknown.

PAWDOC: Searching

The ability to search for and find a document is just about the most important aspect of a filing system, and that capability has undoubtedly been improved by increases in the power of the modern computer. For example, when the index for the PAWDOC collection was computerised in 1986 it took 228 seconds to conduct a standard search on 3200 records. By 2001, that standard search conducted on over 14,000 records took 7 seconds. That same search, now performed on over 17,000 records, takes less than a second on my current laptop – in fact it’s virtually instantaneous.

Of course, search speed is only half the story, since a targetted document also has to be selected and retrieved. The fact that whole digitised collections can be held on a modern laptop, means that this second part of the process can also be very quick. In fact the total end to end search and retrieval time for the current PAWDOC system is typically between 15 and 30 seconds.

However, speed is not the most important element of a successful search. Instead it is the ability to find what you are looking for – whether you are after a specific document or just doing a general search to see what you have on a specific subject. In the PAWDOC system, searches are conducted on the collection’s Index, therefore success is critically dependent on there being a match between the words specified in the search query and the words in the index entry of the targetted document(s). In a personal system, both elements – index entry and search query – come from a single individual’s mind, so more often than not a match is achieved. However, inevitably there are cases where a match isn’t achieved first time – and, sometimes, not at all. There are a variety of reasons for this including the passage of time (I placed my first document into PAWDOC some 38 years ago), and changing terminology as one becomes more familiar with a topic or as technology develops. To cope with these problems, strategies such as using terminology appearing in unsuccessful searches, and adding keywords to index entries when an item is eventually found, can be helpful.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q22. How long does it take to find items in the filing system?

2001 Answer: Partially answered:

  • Using the `t f o a d r’ test (find every record with the combination of t, f, o, a, d and r somewhere in it) on the 1988 Filemaker Index with 3200 records running on a Macintosh computer, 27 hits were found in 228 seconds. The equivalent test on the 2001 Filemaker Index with 14111 records running under Windows `98 on a Pentium II PC, took seven seconds to identify 612 hits. A search for all records containing the syllable `man’ across the same two systems took four seconds to identify 211 hits in the 1988 system, and one second to identify 1137 records in the 2001 system (Wilson 1992a: 38, 85).
  • Having identifed the correct index entry (in the 2001 system) it takes between 10 and 20 seconds to obtain the reference number, go to the hardcopy cabinet/box, find the item and pull it out.
  • Should the index entry concerned refer to an electronic file, it takes about 6 ± 10 seconds in the 2001 system to have the document management software display the relevant folder, to double click the required item and to have it open up in the relevant application.
  • The total end-to-end retrieval time for the 2001 system is about 10 ± 30 seconds for hardcopy and 10 ± 20 seconds for electronic files.

2019 Answer: Fully Answered: Using the `t f o a d r’ test (find every record with the combination of t, f, o, a, d and r somewhere in it) on the 2019 Filemaker Pro 15 Index with 17294 records running on a Chillblast Intel i7  computer with 8Gb of RAM, 1224 hits were found in less than a  second – in fact, almost instantaniously. A search for all records containing the syllable `man’ across the same system took less than a second  to identify 1538 hits (in fact it too was almost instantaneous).

Having identified the correct index entry, it takes between 10 and 15 seconds to copy the reference number, go to the Windows File Explorer screen, select the main PAWDOC folder, paste the number into the search field, press enter, and double click  the folder when it appears. Since there may or may not be multiple files in the folder, and since different files may require different applications which probably open at different speeds, it is difficult to provide a reliable figure for selecting a file and opening it. However, as a very rough guide it is likely to take between 3 and 15 seconds.

Therefore the total end to end retrieval time in the current system is approximately 13 – 30 seconds.

Q23. What can you do to speed up retrieval?

2001 Answer: Not started:

2019 Answer: Fully Answered: There are two key factors that affect retrieval times – Physical Proximity and System Integration. Physical Proximity relates to how close you are physically to the system and the hardcopy and/or digital documents. If you haven’t got the system with you then you can’t even identify the document you require, let alone retrieve it regardless of whether it is a hardcopy or digital document. If you are able to identify the document you require, and it is a hardcopy document, then the closer you are to the hardcopy documents the faster retrieval is likely to be (for example, retrieval will be faster if the hardcopies are in the same room that you are in as opposed to in a room down the corridor).If the document you require is an electronic document, then it will probably  be a little faster to retrieve if it is on the same system you are using to search for documents, than if it is on some remote server elsewhere. Therefore, from a Physical Proximity perspective, retrieval can be speeded up by making sure that the Index and the digital store and any hardcopies are all as close to the user as possible.

System Integration refers to the linkage between the searchable Index and a collection’s digital files. Zero integration requires the user to remember the Reference Number selected in the search process, to go to the database of digital files, and to use the Reference Number to open the relevant folder. In contrast, a very high level of integration might be achieved by having the files being stored under a particular Reference Number, appear somewhere in the Index screen for that Reference Number, and being able to open a particular file from there. A halfway house might be to have a macro which will use a Reference Number identified in the index to automatically open up the folder of that particular Reference Number. Therefore, from a System Integration perspective, retrieval can be speeded up by reducing the keystrokes required to go from selecting an Index entry to viewing the files associated with that Index entry.

Q24. In what circumstances are searches conducted?

2001 Answer: Partially answered:

  • `Start Work’: focused assembly of information while under no pressure.
  • `Mid Work’: a search for a specific piece of information while pre-occupied with the interrupted activity.
  • `Visitor’: a search while you are talking to someone at your desk.
  • `Phone Call’: a search while you are on the phone to someone.

2019 Answer: Fully Answered: There are probably three intersecting dimensions to the circumstances in which searches are conducted: Activity (what you’re doing at the time you conduct the search); Work Content (the topic you are working on when the search is conducted); and Location (the type of place in which the search is conducted).

Four different types of Activity are described in the original BIT answer in 2001 (Start Work, Mid Work, Visitor, Phone Call) – though I have no data on the relative frequency of each of those. However, from experience I would guess that Mid Work occurred most often with the frequency of Start Work, Phone Call and Visitor occurring in that descending order.

Work Content might be the subject you are looking into, or the project you are working on, or the organisation you are working for, or any other categorisation that summarises the type of work being undertaken. Again, no data is available to identify what types of work content have been most associated with the searches made on the PAWDOC collection.

Location can be categorised as Employer’s Office, Other Organisation’s Premises, Travelling, and Home. I know that, over the years, I have indeed conducted many searches at all these types of location.

Q25. What are the most common types of searches?

2001 Answer: Partially answered:

  • The `Familiar Item’ search for an item that you have accessed several times before.
  • The `Long Lost Friend’ search for an item you are sure is there but have not accessed recently.
  • The `Shot in the Dark’ search to see if there is any material on a subject.
  • The `Literature Search’ to find everything you have on a subject.

2019 Answer: Partially answered: The 2001 BIT answer provides one perspective on the most common types of searches conducted on the PAWDOC collection (the ‘Familiar Item’ search; the ‘Long Lost friend’ search; the ‘Shot in the dark’ search; and the ‘Literature Search’). However, another perpective might be to categorise the types of document content most frequently searched for. This analysis might be feasible to perform using the ‘Date Last Accessed’ field’ in the Filemaker Index. Although this may not be an entirely accurately record of which items have or haven’t ever been searched for, it nevertheless does provide some sort of indication. Therefore, by categorising the 4551 records which have an entry in the Date Last Accessed field (out of a total 17,294 records) and ranking the categories by number of occurrences, some indication will be gained of the types of documents that have been searched for and their relative frequency.

Q26. What are the most effective search strategies?

2001 Answer: Ideas formed:

  • Get into the habit of searching the filing system when you need some information even when you don’t think you have anything relevant; after several years you forget what you have (Wilson 1992a: 4, 25).
  • Let your mind roam freely when selecting search words; you are more likely to come up with words you originally specified as keywords (Wilson 1992a: 4).
  • Specify searches with minimal parts of words to avoid problems where spelling errors have been made.

2019 Answer: Fully Answered: In addition to the suggestions made in the 2001 BIT article (search just in case, let your mind roam freely, and use minimal parts of search words), I would add:

  • Use any older terminology you can think of if your current terminology isn’t coming up with the goods;
  • Check the results of unsuccessful searches to see if there are any terms which you might try in subsequent searches;
  • If eventually you are successfull in a search that has taken some time, consider adding some additional search terms to the index entry to give yourself a better chance of quicker success in the future.

PAWDOC: Scanning

Despite today’s widespread use of computers and email to create and distribute electronic documents, there has been little, if any, reduction in the use of paper in business. So, office workers  are likely to received, annotate or even handwrite paper documents for the foreseeable future; and scanning remains an integral part of any filing regime. However, it is much more straightforward activity than it used to be: scanners are now fast, effective and cheap and capable of digitising most documents. Contrast and brightness settings can deal with faint or over-emboldened documents; and colour scanning is no longer problematic – nearly all scanners can scan colour and the larger file sizes produced are no longer an issue as storage has become cheap and plentiful. Even large document sizes are no longer the issue they used to be because, if they are too large to be scanned on the equipment you have, they can simply be photographed with today’s high resolution cameras in mobile phones to produce easily readable images which can be enlarged at will.

The time it takes to scan a document is highly dependent on the model of scanner that is used and the size of document being scanned, so I can only provide timings for the particular scanner that I use – a 5 year old Canon DR-2020U A4 scanner. It takes roughly 1.5 – 2 minutes for a 1 page colour document; and roughly 2.5 – 3 minutes for a 5 page double sided colour document. These are overall times which include preparing the paper, specifying the name of the file containing the scanned documents, and saving the file to a particular Windows folder.

Experience has shown that working with scanned documents is perfectly doable – the image quality on today’s screens is good and, in any case, the image can be enlarged if necessary. Of course, it is much easier to do so at a fixed workstation – perhaps with a large screen; having to work with scans on a laptop in meetings or while travelling is inevitably a little less comfortable, especially as it isn’t really possible to fit the whole of an A4 page in portrait mode on a typical laptop screen and be able to read it easily. More often than not users will enlarge the image so that only half a page is showing on the screen and then scroll up and down. Having said that, I have found that the iPad – and presumably other tablets – overcomes this problem. Whole A4 scanned pages can be presented in full on an iPad in portrait mode, and are very clear and fully readable. The iPad is light to hold and it is easy to navigate through multi-page documents; and it can be purchased with sufficient storage as to hold all the digital documents in a collection  such as PAWDOC. This would seem to be the best way to put a digitised filing system to use – though great care would have to be taken to look after it as the very characteristics that make it so useable also make it extremely vulnerable to loss or theft.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q15. How long does it take to scan documents?

2001 Answer: Fully answered:

  • About 30 seconds for a black & white page. (Chan 1993: 28)
  • 40 ± 50 seconds for a black & white page (Wilson 1997: 3)
  • 30 ± 50 seconds a page for dual journal pages on a flatbed scanner (Wilson 1996b

2019 Answer: Fully Answered: Using a Canon DR-2020U scanner purchased in 2013 and a Chillblast i5 computer with 8Gb of RAM and 1Tb SSD drive purchased in 2018, scanning time for a 5 page double sided colour document is about 70 seconds; and for a single colour page about 25 sec. However, this doesn’t tell the whole story because the scanning process also involves preparing the pages and storing the output. Approximate timings for all of the different process elements are as follows: a) Prepare the paper ie. remove any staples or postITs, get pages  alligned etc.. (5 seconds if there are no problems – but could take much longer) b) Start the scan – place the pages in the scanner, select the appropriate scanning job – Colour or B&W etc. – and press START (10 seconds), c) Perform the scan (70 seconds for 5 double sided colour pages, or 25 seconds for a single colour page), d) Create the file title – paste the Ref No (which had been copied in the indexing process) into the file title box and type in the rest of the file title  (15 -30 seconds), e) Save the file – select the folder into which the document is to be placed and press SAVE (20 – 30 seconds), f) Check the folder – check that the file has been stored in the correct place with the correct title (15 seconds); g) Check the file – open the file to check it has been scanned correctly (only for multi-page documents or for documents where there may be an issue) (15 seconds).

In summary, overall scanning time for a 1 page A4 colour document is roughly 1.5 – 2 minutes; and for a 5 page double sided A4 colour document about 2.5 – 3 minutes.

Q16. What are the practical problems associated with scanning?

2001 Answer: Partially answered:

  • Scanning double-sided pages is time-consuming and error prone (Wilson 1995b: 121, 1995c: 1).
  • Paper jams and misfeeds sometime occur when using the scanner sheet feeder (Wilson 1995c: 1).
  • Some documents need preparation work prior to scanning, for example, removing staples or guillotining bound documents (Chan 1993: 34 ± 37).

2019 Answer: Fully Answered: Practical scanning problems are of two types – those that are inherent to the scanning activity; and those that are due to shortcomings in the scanner being used. The main inherent scanning problem is the need to ensure that the papers being scanned are free from staples, PostIT notes and any other additional items which will jam as the paper passes through the scanner. Problems due to scanner shortcomings may be due to a lack of capability in the scanner concerned such as no sheet feeder; the inability to scan large pages such as A3 (A3 scanners are available but they’re big and more expensive); and inability to scan double sided pages (a particular problem I had when I started scanning but I’m glad to say that my current scanner scans both sides of a page at the same time as it makes a single pass through the scanner). Scanner shortcomings may also include capabilities in which technology developments are continually making inroads, for example, sheet feeder technology (hugely better in my current scanner – but multiple pages are still pulled through occassionally); and dealing with very faint text (I would hope that future software will automatically adjust brightness and contrast settings to deal with this). Finally there may be shortcomings in a particular model of scanner. I believe this is the case in the software driving my Canon DR-2020U which seems incapable of scanning a full page which has continuous black or very dark areas on the edge of the page – the scan truncates the page to remove those areas. To deal with this I have to cover the relevant edges with white paper. This  may have been resolved in a later release of the software – though I havn’t checked for some time.

Q17. What are the practical problems of scanning documents containing colour?

2001 Answer: Ideas formed:

  • Some colours scan black when using a black and white scanner (Wilson 1995b; 76, 77, 98)
  • Colour scans take more time (Wilson 1997: 2).
  • Colour scans produce files 10 or 20 times the size of a black and white scan (Chan 1993:43, Wilson 1997:2)
  • Colour scans (which are made up of red, green and blue) may not work well on printers (which use cyan, magenta, yellow and black) (Chan 1993: 44)

2019 Answer: Fully Answered: Colour scanning no longer presents a particular problem because today’s scanner all have a colour capability, and even though colour scans take up more space, storage is now plentiful and cheap (for comparison purposes, with my current scanner, a single page with 5 colour photos on it scanned at 300dpi B&W to a PDF of 126 Kb; and at 300dpi Colour to a PDF of 801 Kb).

For the most part, colour scans are normally sharp and clear. The only problem I have had with colour scanning seems to be specific to the software used by my current 5 year old Canon DR-2020U scanner. I’m very pleased with the performance of this scanner except for tha fact that it truncates those parts of the edges of pages which have black or very dark colour. To deal with this, I have to place white paper just over the edges of the page – which can be difficult for larger pages which go up to the edge of the scanner platen. I have never experienced this problem before and hope that it has been fixed in a software upgrade.

Q18. Can poor originals be successfully scanned?

2001 Answer: Experience gained: Modifying the scanner settings can dramatically improve the quality of scans of poor originals.

2019 Answer: Partially Answered: Originals can be considered poor for a variety of reasons including skewed images, holes punched in the sides for binding purposes, and faint or heavily smudged images. My current scanner has on/off features to deal with skew (works well) and punched holes (I havn’t really tried to use this); and has settings to adjust contrast and brightness (these also work well but I’ve found it difficult to remember which combination of settings work best – perhaps in the future more automated assistance will be provided for this). In summary, usable scans can often be achieved from poor originals.

Q19. Can all types and sizes of documents be scanned successfully?

2001 Answer: Experience gained: Yes, provided you have a flatbed scanner with a sheet feeder. If the flatbed is not big enough, many of the larger documents can be cut or folded to the necessary size. The spines of gummed or stapled newsletters and booklets can be cut off (Wilson 1995b: 99). Post-it notes stuck on a page go through the sheet feeder without a problem (Wilson 1995b: 13, 39, 140). Journals can be scanned successfully on a flatbed scanner (Wilson 1995b:124).

2019 Answer: Fully Answered: As described in the 2001 answer, the vast majority of documents can be scanned successfully provided you have a scanner which can accommodate the physical size of the document. Of course, if you can’t lay a document flat on the scanning platen, as is the case when trying to scan a book or a journal, there will always be some distortion at the centre of the dual pages.  Most people are likely to have an A4 or, in some cases, A3 scanners  (though an A3 scanner does ta ke up more space) so that they will be unable to scan documents above these sizes; However, that is of no consequence these days because digital photography is now so cheap, and powerful, and prevalent on mobile phones, that it can be used to photograph documents that are too big or difficult to scan. The digital images produced are good enough to read and can be enlarged as necessary. A good example of this is the increasingly common practice of taking photos of flipcharts of which there are a few in the PAWDOC collection. However, there are a few potential problems with photographing documents or other flat objects: one is that, unless the camera is held in exactly the same plane as the document page, the image of the document will appear shorter on one side than on the other; and the other is that, if the document will not lie flat, the document image will appear equivalently distorted in the photograph.

Q20. How can you minimize the amount of scanning that needs to be done?

2001 Answer: Experience gained: Possible strategies include:

  • Stick with electronic files as much as possible
  • Minimize the printing of paper (Wilson 1995b: 82, 90).
  • Elect to receive electronic versions of newsletters, magazines and journals (Wilson 1995b: 126).
  • Go out of your way to obtain electronic versions of paper you get from other people (Wilson 1997: 3).

2019 Answer: Fully Answered: Despite paper still being very widely used (the paperless office never actually arrived), most documents are now created and distributed in digital form – very little paper arrives in the overland or office mail today. Hence, scanning can be minimised by simply storing the electronic versions of documents you create and that you receive by email; and by seeking out the electronic versions of other hardcopy document that you receive. For example, I get the quarterly newsletter of the professional body that I belong to in hardcopy form; but I go to their website to get a digital version of the magazine. Having said that, journal and magazine publishers continue to protect their paper and articles behind paywalls; and newspapers seem to be following their example. Consequently, filing system owners may have to scan some hardcopy papers and articles for the foreseeable future.

Q21. Can scanned images be used successfully in day-to-day work?

2001 Answer: Ideas formed: Relatively few of the scanned images have been used to date. However, experience so far has not indicated any major problems with either reading them on screen or printing them out. An A4 sized display would certainly make it easier to read scanned images on screen (Wilson 1995b: 104).

2019 Answer: Fully Answered: Although I don’t access many of the PAWDOC scanned images these days, over the years I have used them and worked with them and have had no trouble doing so. Nowadays I view all my scanned documents (in both multi-page TIFF and PDF formats) in my eCopy PDF Pro application and this works fine – though it would undeoubtedly be better if the screen could be in Portrait mode so that the whole of an A4 page could be displayed in the way one reads the hardcopy. Recently I transferred several scanned documents onto my iPad and I’ve found I can read them all without any difficulty with the iPad in Portrait mode and without needing to enlarge them. This has been a bit of revelation since it is clearly a much easier and more pleasing experiece to read these documents on the iPad than it is on my 24in desktop screen or my laptop. The combination of the iPad’s high resiolution screen, it’s light weight, the fact that you can hold it as close as you want, make it a winning combination for scanned work documents. This could be the way to store digital filing systems in the future – or perhaps large screen phones could serve the same purpose?

PAWDOC: Indexing (and the Accession process)

Indexing is a key aspect of a filing system because it is one of the mechanisms used to find documents. An alternative mechanism is to search the full text of documents, but the PAWDOC system has not explored this approach because a) the PAWDOC system does not recognise the full text of all the documents it contains, and b) because I have always believed this approach would provide too many hits thereby reducing search effectiveness.

Indexing within the PAWDOC collection is performed at two levels: first, each entry (uniquely identified by a Reference Number) in the main index file contains a Title field (in which a free format description and any number of uncontrolled keywords can be specified) and a few other fields including publication date. Second, each electronic file (be it a scan of a hardcopy document, or a born-digital document) associated with a particular Reference Number contains a unique file title. The contents of these two sets of indexing information provide the means to search for a document, to decide if you have found what you are looking for, and to get a better idea of what the chosen Reference Number and/or associated electronic file(s) contain. These potential uses are worth bearing in mind when indexing information is being specified.

Inevitably mistakes are made when specifying Title information and so it is essential that index entries can be changed over time to correct typos, grammar and factual errors as necessary. Changes may also be inspired by experiencing difficulty in finding a document, and in these cases some additional text may be included to reflect the search terms that first came to mind, or to reflect the changes in terminology that occurs regularly in language and in today’s fast moving technical and cultural environment.

Indexing is closely integrated into the process of including a new document in the PAWDOC collection, not least because adding a new file involves creating a file title which in turn includes a Reference Number and possibly some text from the associated Index entry. Hence, timings for how long indexing takes are embedded within the overall process of Accessioning (which includes scanning for hardcopy documents). For the PAWDOC collection, which uses a scanner acquired in 2012 and a laptop acquired in 2018, it takes approximately 2.5 – 4 minutes to accession a single colour A4 hardcopy page; and about 3.5 – 5 minutes for a 5 page double sided colour hardcopy document. An electronic of any type and number of pages takes approximately 2 – 3.5 minutes to accession. These are appreciable amounts of time for an overhead administration activity amidst a busy working day. Therefore, it is worth exploring any means of reducing these timings. The obvious way is to keep the number of index fields to an absolute minimum. Experience with the PAWDOC system indicates that integrating keywords into the Title field (as opposed to having a separate keywords field) has been very successful. Conversely, the Publication Date field has not proved to be very useful and I believe could be dispensed with. Experience with the other fields used in the PAWDOC system are summarised below:

  • Reference Number – essential
  • Creation Date – a useful control
  • Movement Status – very useful for recording the whereabouts and status of documents
  • Date Last Accessed – only useful if you particularly need it.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q10. How long does it take to index a new document?

2001 Answer: Fully answered: It takes 1 -2 minutes to make an entry in the main index and to write the Reference Number on the document. A further 30 seconds is needed to create an entry in the document management system for either an electronic file or a document that is to be scanned immediately (Wilson1995c: 1). Scanning a document will add a further 40-50 seconds for a single page – though the per-page time is reduced considerably for multi-page documents that are put through the sheet feeder (Wilson 1997: 3).

2019 Answer: Fully Answered: This question should really be ‘How long does it take to index a new document and include it in the collection?’ because the indexing process is closely integrated with the process of saving the file. The figures in the 2001 Answer for making a new entry in the main index and writing the Reference Number on the document (1-2 minutes), and for creating a new entry in the document management system (an extra 30 seconds) still stand.  Scanning a document using the Canon DR-2020U scanner I bought in 2013 and the new Chillblast laptop I bought in 2018 with a 1 Terrabyte solid state drive, takes 25 seconds for one colour page or 70 seconds for 5 double sided colour pages. However, the Document Management system was eliminated from the PAWDOC architecture in 2018 and replaced by Windows folders, so the subsequent set of actions has changed to the following: a) Create the main index entry (1-2 mins); b) Create a sub-folder within the main PAWDOC folder in the Windows file system, with the new Reference Number and copy the Ref No (15 seconds); c) create the file title (Ref No   Title,  Date scanned) in the scanner control application starting by pasting in the copied Reference Number (15 – 30 seconds); d) Save the file – use the scanner control application to select the new folder just created to specify where the file should be stored and press save (20-30 seconds); e) check the file – open the file to check that it has been stored in the correct place with the correct title (15 seconds); f) Check the folder – open the file to check it has been scanned correctly (only for multi-page docs or for docs where they may be an issue) (15 seconds). The process for electronic documents has also changed as follows: i) create the main index entry (1-2 mins), ii) create a sub-folder within the main PAWDOC folder in the Windows file system, with the new Reference Number and copy the number (15 seconds); iii) Create the file title – open the Save AS dialogue box and give the file a new file name (Ref No   Title,  Date saved) by first pasting in the copied Reference Number (15 – 30 seconds); iv) Save the file – navigate to the folder into which the file is to be saved and press SAVE (20 – 30 seconds); v) Check the file – open the file to check that it has been stored in the correct place with the correct title (15 seconds). Of course, electronic documents which are placed into an existing Ref No have no need to create a new index entry or a new folder.

In summary, overall timings for including new documents in the collection are as follows: for a 5 page double sided hardcopy document using a new Ref No – approximately 3.5 – 5 minutes; this may reduce to about 2.5 – 3 minutes if an existing Ref No is being used. For an equivalent 5 page electronic document using a new Ref No – approx 2 – 3.5 mins; this may reduce to 1 -1.5 mins if an existing Ref No is being used. Timings for a hardcopy document with only 1 page of contents are reduced by about 45 seconds due to reduced scanning time; electronic document timings stay the same regardless of page count.

Q11. What can you do to speed up indexing?

2001 Answer: Ideas formed: Keep the number of index fields to a minimum.

2019 Answer: Fully Answered: Minimise the number of fields. Automate the generation of the Creation Date. Eliminate the Date Last Accessed field if there is no good reason for recording that information.  To speed up the overall accession process, a) store more rather than less documents in existing Reference Nos (using existing Ref Nos avoids having to create new index entries); b) use a faster scanner; c) reduce the time it takes application programs to page through 17,000+ sub-folder names in the Save As function (for some reason, in my Windows 10 system, the application programs take far longer to do this than Windows File Explorer).

Q12. What is the most effective set of index fields?

2001 Answer: Experience gained: The smallest number you can manage with (Wilson 1990: 95). Reference number, title and keywords, date of creation of the record, movement status, date last accessed.

2019 Answer: Fully Answered: Keeping the number of fields to the absolute smallest number you think you can manage with will minimise the time and effort spent on putting new items into the system and on managing the system. The minimum number of fields I could manage with are Reference No, Title, Creation Date, and Movement Status. The other two fields that I use, but which I believe I could do without, are Publication Date (I rarely refer to this) and Date Last Accessed (which was included mainly for research purposes).

Q13. What criteria should be employed when defining titles and keywords?

2001 Answer: Ideas formed:

  • Remember there is no need to use the actual title of the document.
  • Define titles/keywords in accordance with what the document means to you – that way you are more likely to be able to retrieve it.
  • Bear in mind that title/keywords serve multiple purposes (Wilson 1992a: 8):
    • To enable search and retrieval
    • To enable the user to decide if a retrieved record is what was being searched for
    • To provide the user with an understanding of what is contained in the item referenced by a retrieved record.

2019 Answer: Fully Answered: Do not feel constrained to use the actual title of the document – you are far more likely to be able to specify successful searches for items if you define Titles and Keywords which convey what the document means to you. If you are aware terminology is changing, use the latest terms you know as you are likely to become less familiar with the old terminology as time passes. Since there are no length constraints on the Index entry, use as many words as necessary to describe what the document is about. The second level of indexing – the text in the Titles of individual files – is constrained by length and for that reason might well be shorter than the main index entry. However, care should be taken not to truncate unnecessarily and not to just take the quick and easy route to create the file title by cutting and pasting a generic part of the Index entry. Certainly, where there are two or more files associated with the same Reference No, the titles of the 2nd and subsequent files should clearly distinguish them from all the other files in that Reference No. Also, bear in mind that Title fields (which include Keywords) and File Titles serve at least the following three purposes: a) to enable search and retrieval; b) to enable the user to decide if a retrieved record is what was being searched for; c) to provide the user with an understanding of what is contained in the item referenced by a retrieved record.

Q14. Are there circumstances in which keywords and titles should be changed over time?

2001 Answer: Experience gained: It is very necessary to be able to alter keywords and titles for the following reasons:

  • To accommodate the user’s growing familiarity with a topic. For example, the term `OA Human Aspects’ became an abbreviation `OA HCI’ in later entries, so it needed to be placed in previous entries to ensure they would be retrieved if only `OA HCI’ was specified in the search term (Wilson 1992a: 22).
  • To accommodate changing language. For example, the term `DTI pilot’ became `DOI pilot’ when DTI changed its name (Wilson 1992a, 12).
  • To accommodate additional material being added to an existing hardcopy file, or being added to the associated Folder in the Document Management System.

2019 Answer: Fully Answered: Users should be able to change Titles and Keywords at will to correct mistakes or to try and improve the search success rate. Circumstances in which changes might be made for the latter reason include:

a) to accommodate the user’s growing familiarity with a topic. For example, the term `OA Human Aspects’ became an abbreviation `OA HCI’ in later entries, so it needed to be placed in previous entries to ensure they would be retrieved if only `OA HCI’ was specified in the search term;

b) to accommodate changing language. For example, the term `DTI pilot’ became `DOI pilot’ when DTI changed its name; and

c) to accommodate additional files being added to the Reference No.

PAWDOC: Deciding what to file

Most types of things can be managed in a digital filing system. Documents, brochures and books can be either scanned or photographed; electronic documents are, of course, already in the right form; and even physical artefacts such as, for example, art works, can be photographed.

Once it’s been decided to establish a filing system, the reasons why the items concerned are being kept, and what is going to be done with them, should be written down. This should help in defining the following three key points: i) how long items are to be kept for; ii) whether any physical originals are to be retained after they have been digitised; and iii) whether every item is to be filed or just a particular subset. Point iii) is important because, if only a subset is required, the amount of time and effort that will be spent on adding items to the filing system, and managing it, will be reduced. Against this time saving must be weighed the extra effort – and potential errors that may be made – in deciding whether each particular object is to be included or not. Conversely, collecting every instance of a type of an item makes life easy and requires no decision making.

If the filing system is to replace or augment an existing collection, a decision must be taken about whether to exclude the existing items, or to undertake ‘backfile conversion’ to include some or all of them in the digital collection. Backfile conversion is often a lengthy and tortuous process and can act as a drag on proceeding with the new system; therefore, it should be avoided at all costs. If, however, it is thought to be needed, the reasons for doing it should be clearly understood before proceeding.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q5. What types of information and media can be managed by an electronic filing system?

2001 Answer: Fully answered: Just about anything can be managed by an electronic filing system including single sheets and stapled sets of hardcopy documents, scientific journals, volumes of bound material, ring binders, books, 35 mm slides, videos, electronic files and e-mail messages.

2019 Answer: Fully answered: Any kind of item can be managed by an EFS – though it may be necessary to also keep the original physical artefact to experience the full impact of certain types of items such as specially folded brochures, or significant douments with a tactile presence. Types of items that I have succeeded in managing in the PAWDOC collection include hardcopy documents, ring binders, books, journal papers, magazines, brochures, overhead slides, flipchart pages, CAD/CAM drawings, photos, videos, web pages, and email messages.

Q6. Can all types of electronic files be managed in an electronic filing system?

2001 Answer: Partially answered: Yes, certainly all types of application files can be. I have not stored executable files yet but have been advised that it can be done and how to do it.

2019 Answer: Fully Answered: Any types of file format – including executables – can be managed in this way. For example, file types in the PAWDOC system include Word, Excel, Powerpoint, PDF, TIFF, JPG, MP4, MPP, ZIP, and HTML. However, the real issue is whether you will be able to open them in future years if the particular application programmes are lost or not kept up to date. For this reason, it may be worthwhile converting some files to more common formats, such as PDF, before storing them. In the future, these so-called ‘Digital Preservation’ issues may get addressed automatically by AI programmes.

Q7. What criteria should be employed when deciding what to file?

2001 Answer: Not started

2019 Answer: Fully Answered: Individuals should file as much or as little information as they think they will need in their jobs – otherwise they will not have the motivation to spend the time introducing new material and managing the filing system. If a robust, all embracing, file facility is available within a particular system (such as in many electronic mail systems) such systems should be used to their full capability (i.e retain everything up to any limitations set by the system administrator). Inevitably, whatever an individual decides to file, there will always be some material which is never subsequently accessed, and there will always be some required information that can’t be found in the system. This seems to be a regularly occurring phenomenon and should not deter individuals from filing – though they may influence the filing criteria that an individual applies over time.

Q8. Is it worth doing backfile conversion?

2001 Answer: Experience gained: Backfile conversion is certainly not essential, and is probably not worth doing, except for material you definitely want to keep for many years.

2019 Answer: Fully Answered: Backfile conversion usually takes a huge amount of time and effort, so a detailed analysis should be undertaken before deciding whether to do it or not. Alternative approaches to dealing with the old items should be investigated. Backfile conversion should only be carried out if there are compelling reasons for doing so.

Q9. What are the major considerations when carrying out backfile conversion?

2001 Answer: Experience gained: Backfile conversion requires a huge amount of time and effort and is not to be undertaken lightly.

2019 Answer: Fully Answered: Considerations to be taken into account when conducting backfile conversion include:

  1. Ensure the physical backfile material does not interfere with any space required by the replacement filing system.
  2. Decide if any of the backfiles being converted will need to be retained in their original form and, if so, define clear criteria for selecting which ones to keep.
  3. Decide what quality the converted backfiles need to be and then put in place the appropriate equipment, software and procedures to achieve that quality.
  4. Decide how users will be informed of which backfiles have been converted and which are yet to be converted.
  5. Establish a clear and doable schedule for the conversion process.
  6. Put in place motivational aids for sticking to the schedule. For example, setting a target of a certain number of conversions each day; and creating a visible progress sheet on which achievements can be ticked off.

PAWDOC: Collection content, size, growth rate and usage patterns

The PAWDOC collection was set up in 1981 to explore the application of office technology. In 2001 a paper was published in the Journal Behaviour & Information Technology called ’20 years in the life of a long term empirical personal electronic filing study’. This described PAWDOC and summarised findings about its use up to that point under the following 15 headings:

  • Collection content, size, growth rate and usage patterns
  • Deciding what to file
  • Indexing
  • Scanning
  • Searching
  • Use of information
  • Archiving
  • Hardcopy/electronic mix
  • Relationship between work patterns and filing activities
  • Technology requirements and problems
  • Portability
  • Sharing files
  • Confidentiality, ownership and intellectual property rights
  • Reliability and Longevity
  • Costs and benefits

Now, a further 18 years on, the findings will be further reviewed in this and subsequent posts. The first of these follows below.

Collection content, size, growth rate and usage patterns

The precursor to PAWDOC was a conventional filing system in an upright cabinet using hanging folders and crystal tabs to store documents only (not journals, books etc.). The contents were specified in an index taxonomy with entries of the type, for example, 1.6.8 – Quarterly Progress Reports, which I was constantly adding to. Consequently much space was taken up in the cabinet by folders with crystal tabs housing only a few pages.  This changed after the PAWDOC system came into use: it needed far fewer hanging folders because each folder was filled to capacity with as many of the serially numbered documents that it could take.

The PAWDOC schema was explicitly defined to support the management of multiple sets of different material owned by multiple different owners. This capability was used to manage a variety of sets of my own material in addition to documents, for example, several different journals, 35mm slides (for making presentations), ring binders, and books; and, in the early years of the system, material owned by other people and organisations – though, as time went by, I made fewer and fewer such entries not least because of the uncertainty of being able to access such material.

The system soon became an integral part of my working life, and I used it just about every day. Several hundred new items were being added each year and this steady rate of acquisition soon ran up against the physical limits of the upright cabinet, so it became necessary to archive material in boxes and then to put some of the boxes in store. This went on until I started digitising newly acquired documents in 1996 and disposing of most originals. At this point, I also started to store born digital documents regardless of the applications they were created in.

In 2001 I started a new job in Bid Management which precluded personal storage of its associated highly confidential and fast moving documentation; so my useage of the PAWDOC system did reduce from then onwards. Nevertheless I continued to use it regularly on most days, and was still adding over 200 new items each year up to when I retired in 2012.

The 45 boxes of hardcopy that I had acquired eventually came out of store around 2001 and were stored in my garden shed. Their number was overwhelming and I began to doubt I would ever get them all digitised. However I stuck at the task, sometimes going at it solidly for several days at a time when my wife was away. By the time I retired in 2012 only 4 boxes remained and these had all been scanned by 2014: it was a great relief. The huge amount of physical space taken up by the collection had been reduced to just two archive boxes of significant hardcopy documents; and, after I had conducted a digital preservation exercise on the collection in 2018, the digital footprint of the collection amounted to some 115Gb. I have ended up with a fairly complete digitised archive of all the non-highly confidential materials that I had encountered throughout my working life, including substantial amounts of material from my earlier career from 1972 with Kodak and then CPC. Since retiring I’m continuing to add a few documents (around 40 up to 2019) in three categories: significant articles relating to the work I used to do; documents relating to the digital preservation of the PAWDOC collection; and material relating to work I am doing to investigate and document the findings from the PAWDOC collection’s 38 years of existence.

Specific questions relating to this aspect are answered below. Note that the status of each answer will fall into one of the following 5 categories: Not Started, Ideas Formed, Experience Gained, Partially Answered, Fully Answered.

Q1. What are the contents of the collection?

2001 Answer: Fully answered: At the beginning of July 2001, the collection consisted of 14 100 index entries representing approximately 185 000 pages of paper, 50 000 scanned pages, over 30 scientific journals (including Behaviour & Information Technology from 1982), around 30 books and conference proceedings, 3700 MS Word files, 400 MS Excel files, 250 MS PowerPoint files, 150 other electronic files of various types, and 10 CDs.

2019 Answer: Fully answered: The PAWDOC user Guide created in 2018 says: “All types of documents were stored including letters, internal memos, circulars, reports, specifications, minutes, overhead slides, 35mm slides, notes, training materials, brochures, manuals, maps, emails, computer magazines, journal articles, conference proceedings, and videos. As Office Technology became more versatile, electronic documents such as word processor files, spreadsheets, presentations and web sites were also filed.”. In June 2019, the collection consisted of 17,293 Index entries representing 29,610 electronic files in 16,067 Windows folders, and about 340 physical hardcopy documents in two archive boxes. A further 384 old electronic backup files are also stored in a separate folder. The Checking exercise performed in 2016  identified the following numbers of different types of files in the collection: Word – 6380; Powerpoint – 466; Excel – 625; HTML – 382; Help – 90; Zip – 92;  11 other apps – 88; Scanned documents – 28,418. The collection was primarily a work collection and therefore the number of new items being included reduced to a trickle when I retired in 2012.

Q2. How much space does the collection take up?

2001 Answer: Fully answered: The paper takes up about 4.7 sq. metres of floor space and 1.7 metres of shelf space. The scanned images and electronic files take up 2.9 GB. The scanner, magneto-optical drive and CD Writer take up about 0.25 sq metres of desk space. The Filemaker Pro index is about 8.1 MB in size and the FISH data file is 11.2 MB. The Filemaker Pro, FISH, SQL Anywhere and Easy CD Creator software packages take up approximately 38 Mb.

2019 Answer: Fully answered: The two archive boxes stand one upon the other and take up 0.2 sq m of floor space. The laptop in which the electronic files reside takes up 0.08 sq m of desk space. The electronic files of the main collection take up 45.9 Gb storage space; and the backup files take up 66.6 Gb. The Filemaker Pro software used for the index takes up 336 Mb of file space.

Q3. What is the growth rate of the collection?

2001 Answer: Partially answered: Between 1981 and 1993 an average of 543 index entries were created each year – estimated to consist of an average 29.4 pages per day (Chan 1993:25).Over the whole 20 years life of the system, the growth rate has been an average of 705 entries per year with a range of 210 ± 1202new entries a year. In 1993, it was estimated that the collection was increasing at the rate of 3.8 MB per day.

2019 Answer: Fully answered: The growth rate of the collection is shown in the chart below.

There are three distinct phases: 1981 – 2000; 2001 – 2011 when I was working on highly confidential bids; and 2012 – 2019 when I was retired. The average growth rates during these periods were:

  • 1981-2000 – 696
  •  2001-2011 – 272
  • 2012-2019 – 46

Q4. How often are the contents accessed?

2001 Answer: Experience gained: Between 1987 and 1993, an average of 363 records were being accessed each year (Chan 1993: 25)

2019 Answer: Experience gained: The only data that has been collected on this question is in the date last accessed field, and unfortunately that only records the latest date an item was accessed – there may have been any number of earlier accesses. Furthermore, some items may have been accessed without the date last accessed field being updated. Having said that, 4,551 items have an entry in the date last accessed field, implying that 12,742 items have never been looked at for work purposes after they had been included in the collection (the date last accessed field was never updated when items were looked at for the purposes of controlling and writing about the collection). For the period from 2001 onwards, when I moved jobs into Bid Management, there were 751 index items with dates of 2001 or later in the date last accessed field (only 15 of these were in 2012 and only a further 15 of these were from 2013 onwards).

NB. The various references in the texts above to Chan,1993 relate to the following reference at the end of the 2001 paper in Behaviour & Information Technology:

CHAN, S. C. 1993, Feasibility of Paperless Office, Submitted in partial fulfilment of the requirement for the degree of MSc in Information Systems and Technology in The Information Science Department at City University, London (Supervisor: Dr David Bawden) [PAW/DOC/4012/08].

Personal filing in a 40 year vortex of change

This entry has been jointly authored by Paul Wilson and Peter Tolmie

The PAWDOC filing system was set up in 1981 to try and understand how the newly emerging office technologies of that era might assist individuals to manage their office documents. Over its 35+ years of operation much has been learnt, and it is our intention to try and understand and describe those findings. However, the system was set up to address requirements in the office of the 1980s, and there has been a revolution in the way business operates since then. In order to be able to relate the findings to office work today and in the future, this entry explores the differences in  requirements for personal filing in the office between the early 1980s and 2019.

Perhaps the most significant difference is the transfer of huge amounts of information from paper-based documents to digital files. Note that this is not saying anything about the current volume of paper in the office (though we would suggest that there is probably less paper filed by individuals now than in the 1980s) – just that individuals now have to deal with huge amounts of electronic material in contrast to the early 1980s when they dealt with virtually none. This transition has been facilitated by a huge growth in the use of computer hardware – desktop computers, laptop computers and mobile phones – throughout the world, from a base of zero to near-universality.

The growth in the use of computers has also prompted huge changes in office work. At the beginning of the 1980s, office professionals used support staff – typing pools, secretaries, and admin support staff – to perform their administrative tasks.  However, as office technology became widespread, professionals started to do their own typing and support staff became a luxury which could be cut to reduce budgets. In 2019, only very senior management have secretaries and office workers are expected to be self-sufficient and fully competent in the use of all hardware and software relevant to the realisation of their work.

These changes were accompanied by a revolution in communications. Electronic mail has now almost entirely replaced internal memos and external letters, and has prompted massive increases in the amount and speed of communication. Email also rapidly became the key mechanism for supporting distributed teamwork – nationally and globally – and now underpins a battery of related interests, from the sharing of documents to the organisation of voice conference calls (which are the unsung foundation upon which much of business and government now operates). In more recent years, as mobile phones have permeated throughout the world’s populations, text messaging and chat applications have become an integral element of personal and business relationships. To this highly significant mix of new technologies must be added the recent massive uptake in Social Media. An unfortunate side-effect of the sheer effectiveness and pervasiveness of these mechanisms is high levels of information overload across a large proportion of office workers.

Importantly for the work to derive findings from the long-term operation of the PAWDOC filing system, the changes described above have impacted filing activities in the office. Hot desking and home working have made personal filing cabinets and bookshelves a luxury. The folder systems integral to computer operating systems (primarily from Microsoft and Apple) are now used to store the electronic documents created and received by the individual. At the same time, email systems have their own integral filing systems into which mail can be rapidly sorted and stored indefinitely in the cloud; text messages are stored on users’ mobile phones in the form of text streams by both senders and recipients; and Social Media systems have their own self-contained environments distributed across vast computing networks. The further evolution of cloud-based repositories, such as Dropbox and Google Drive has led to an added utilisation (if not trust) in distributed document stores. Even if users wanted to integrate these different collections, it would be almost impossible for them to do more than just copy selected elements from one to another or to a dedicated filing system: these stores are separate silos and will probably continue to be so for many years to come.

The design of the PAWDOC system in 1981 was based on an understanding of office filing requirements at the time. There was an expectation of how emerging office technology might be used to support those filing requirements, but little appreciation of how the technology itself would change the way business operates. Initially, then, learnings from the development of the PAWDOC system were entirely focused upon what the impact might be of new assumptions about filing built into the construction of computer systems in the early 1980s. Later on, in the middle period of PAWDOC operation, the findings speak to what it was taking to manage a filing system in a changing work environment populated by imperfect but maturing technologies. More recent findings give a somewhat different picture, as many of the troublesome technologies of the middle-era have come to be taken-for-granted resources, giving rise to new kinds of problem, of which information overload is but one potential symptom. What is clear at present is that computer technology and the business world is now changing so rapidly, the presumption present in the early days of PAWDOC – that one could readily identify needs and solutions for the future – now seems somewhat naïve (if still just as pressing).

One thing, however, we believe has remained constant and that is the attitude towards filing across the population. Most people are not motivated to put effort into filing because it is extra work for an indeterminate reward at some undetermined point in the future. A smaller subset of people is willing to put varying degrees of effort into the activity. We believe this has changed little between the early 1980s and the present day. As it happens, the PAWDOC owner was at the more extreme end of this latter group and wanted to file both effectively and comprehensively. Hence the PAWDOC collection contains most of the documents that the owner read and/or believed to be significant in his work; and consequently it should be borne in mind that the learnings derived from his experiences concern almost the worst case requirements of filing load and effort. It should be easier for most of the population. Certainly, it would seem easier, for digital copies are now retained of virtually everything as a matter of course. The extent to which that is oriented to as a personal collection of materials is a different matter, as is the probity of third parties hanging on to everything in that way. These, of course, are burning questions of the moment, and ones to which we shall ourselves return.

Opening the channels

Since our initial phone conversation on 28th Feb, Peter Tolmie and I have Skyped twice more – we seem to have got into a pattern of speaking every four weeks or so. In our second conversation, Peter pointed out to me that my pawdoc filing system was just another manifestation of my inclination to keep things – as amply demonstrated in the various journeys documented in pwofc.com. He asked me what I thought I’d learnt from all these experiences, and I recounted a few things that immediately came to mind. Afterwards, however, I began to think that there were a great many more learnings dotted around the website. So I duly trawled through pwofc.com and recorded in a spreadsheet anything that looked like a finding. For good measure, I used another worksheet in the same spreadsheet to list all the requirements and findings specified in the paper about PAWDOC that was published in Behaviour & Information Technology (BIT) in 2001. I’ve given the spreadsheet to Peter and it will provide a base set of information for our investigations going forward.

My re-assessment of the BIT paper reminded me that one of the things I was thinking about when I wrote it was how one could use the key points in the documents you read to develop ones knowledge. This idea stemmed from my practice of putting a line next to key points – or nuggets as I termed them – in documents. I remembered that I’d made a start on this work some 17 years ago by recording in a Mind Mapping programme nuggets I found in books about the Pyramids etc. Peter and I discussed the possibility of my revisiting this material in a ‘Nugget Management’ journey sometime.

In our last Skype call on 25th April, Peter asked if I could keep an auto-ethnographic log of my keeping activities to provide us with more base material on draw on in our analysis activities. I duly created a spreadsheet with the headings listed below and am now recording all instances in which I make a specific effort to store a physical or digital artefact. The word ‘specific’ is used to exclude general keeping of things like email messages in email folders; and the word ‘artefact’ is used to explicitly require that a whole integral item is kept not just information removed from it like the name of a species from a plant label.

  • Ref No
  • Date
  • Item
  • How the instance arose
  • Reason for keeping
  • Initial actions and decisions made
  • Actions taken

Peter’s comment on my request for his views on my recording scheme was “This is great. It’s not how I would have done it myself, but that doesn’t matter at all. The main thing is that it works for you. Just different work practices because we come from different backgrounds. Nothing more.”; and I doubt that I, on my own, would have come up with the idea of a generalised keeping log. Herein are clues as to the sheer unique and precious value of collaboration with our fellows.

Getting started with the Findings

Having initiated a preservation planning regime for the collection, and having moved it onto the Windows 10 platform, I’m feeling that the only remaining things I need to do with it are to find it a permanent home and to write up the findings of this lengthy experiment. I took a step forward on the latter activity earlier this week when I had a very interesting phone call with Peter Tolmie, a UK Ethnographer based in the School of Information Systems and New Media at the University of Siegen in Germany. I was given Peter’s name by Richard Harper when I asked if he knew of anyone who is knowledgeable about how professionals manage their documents and who would be interested in working on a wrap-up paper with me. An initial phone call with Peter last Thursday indicated that we have a great many common interests – I found it a very stimulating conversation indeed. I’ve sent Peter some documents describing the collection and we’ve agreed to talk again on 21st March.

Regarding the search for a home for the collection (which is documented in various posts in this Blog going back to 2015), my current efforts lie in conversations I’m having with Dr James Peters, the Archivist of the National Archive for the History of Computing at Manchester University, who has kindly agreed to help me in my search. In a phone call last month, James told me he was waiting for a response from someone he had emailed, but that, if there was no interest from that source, he could issue a note to a relevant mailing list on my behalf. If it is to be the mailing list route, I’m hoping to get James’ advice on what needs to go in the note.

Backup Bolstering

Backing Up has always been an essential part of maintaining my personal document collection; but it was never something I enjoyed – I did it out of a fear of loss. And I have, indeed, experienced loss: in 1996 one of the MO Disks I was using became corrupted and I lost a number of files; in 2004 my laptop was stolen and my whole document collection had to be re-instated from the backups; and in 2017 I had a system crash and, although the repair company was able to recover all my data in that instance, that might not always be the case.

When I was working, I used to take a backup of the more recently created material every month or so, as well as complete versions of the whole collection as it kept growing. This produced multiple copies on many disks which increased my confidence in being able to replace any file that got corrupted or mislaid, but which required managing in its own right as the number of disks grew. As time went by I added other backup mechanisms including storing a copy on another laptop in the househoId, storing a copy on disk in a relative’s house located many miles away, and storing a copy on disk at my son’s house in New Zealand.

After I retired I tried to put the backing up on a more orderly basis and finally fixed on five different types of backup – Cloud, copy on another laptop in the house, local hard disk, remote (in the UK) hard disk, and New Zealand copy on memory stick. I scheduled backups in my iPad calendar for each of these (though, for the Cloud, it was more a matter of checking that it was working and that I could recover from it). However, the iPad calendar doesn’t have a To Do mechanism and I wasn’t looking at the calendar anything like as often as I used to at work. Consequently, I kept missing scheduled backup activities – and, in most cases, didn’t realise I’d missed them; and when I did realise I just kept putting off what was an annoying extra thing to do. One answer would have been to get a To Do app – but I’d had enough of To Dos at work.

The opportunity to come up with an alternative approach, came when I created a Users’ Guide for my document collection in May 2018. I structured the Guide so that it had a Quick Reference Guide to the Collection on the front page, and a Backup Quick Start Guide on the back page. The latter listed the different types of backups to be performed and provided cells to be filled in with a date when that particular backup had been done, as shown below.

This was a definite improvement over dates dotted about a calendar, but unfortunately the schedule was still hidden because the Users’ Guide was tucked away inside an archive storage box.

When I replaced my Windows 7 laptop for a Windows 10 version in December 2018, I decided to review all my backup arrangements again and to try to overcome this lack of visibility. The answer turned out to be really quite simple: I have a display frame for the latest issues of UK postage stamps, on the wall in front of where I sit at my desk. So, I created a table with columns for when backups have been done and when they are due; and this table now resides in the display fame as shown below.

I have a clear view of when the next backups are due every time I sit down at my desk. The next time I miss a backup it’ll be because I just don’t enjoy doing them, not because of blissful ignorance!