Templates v3.0: DONE (after a 7 year journey)

Life has become a little more relaxed in the last few days as I completed the 2021 digital preservation maintenance exercise on my PAWDOC document collection. I cracked on through the most onerous parts of the of the project – converting 1000+ old Word documents to XLSX format, and 300+ old Excel documents to XLSX format – such that I finished some two weeks earlier than planned. I guess I just really wanted to get it out of the way and was able to spend the time getting it done. However, those reasons are probably irrelevant. The key in these exercises is to be able to avoid running over the timescales you have planned – which, of course, is dependent on making realistic plans in the first place. Chicken and egg I know – but I do think its preferable to make a plan that’s doable and beat it, than to make a plan that’s iffy and fail to get there. Of course, that’s why I set great store in doing a lot of pre-work in digital preservation exercises, so that, when it comes to creating a project plan, you have a clear idea of what is to be done and how long it will take.

Anyway, I’ve now been through two cycles of preservation exercises with my large PAWDOC collection of 105,00 files; and also two cycles through much smaller collections of photos (18,000 files) and mementos (800 files); so I’m now pretty familiar with what has to be done and am simply following the maintenance plan documents that I have in place. That, of course, was the purpose of starting this Preservation Planning work in 2014 – to  gain clarity on how to do it. That clarity is now encapsulated in the four template documents that I have developed and refined; and which are now issued as final fit-for-purpose versions:

The documents steer the user through the process steps that need to be taken, as well as providing the clarity of written reference text describing what a particular collection consists of, what has been planned and what has taken place. The documents are also available on the DPC web site with a summary of the background to their development, the challenges they have helped me overcome, their key features, and their applicability.

Having completed the refinement of these templates, my own journey into Digital Preservation has come to an end; I am now settling into the regular maintenance cycles for my various collections. However, before I do, here’s a couple of suggestions for anyone reading this who wants to find out more about the topic: a) take a look at the Digital Preservation Coalition’s (DPC) website which has a wealth of useful information; and b) consider subscribing to the JISC Digital Preservation Mailing List which provides a window onto the numerous digital preservation activities going on around the world. I have found both extremely useful over the last 7 years, and thank both organisations.

Towards final versions of Templates

The first scheduled Preservation Maintenance operation on the large and complex PAWDOC collection started on 1st September. Well, actually, it started a bit before then in early August when I started to investigate the items in the ‘Possible Future Issues’ section of the PAWDOC Preservation MAINTENANCE PLAN. There were 15 such items; most relating to files that had proved inaccessible in the initiating preservation exercise three years ago, but four concerning CDs with numerous contents, that had been included in the collection. Two of these proved particularly demanding: one is a disk that was distributed with the April 2001 issue of PC Magazine; and the other is the Nautilus disk – a 1991 attempt to issue a technology magazine with lots of software, advice, news, and multimedia files on disk. I couldn’t get either to open; and without an interface there’s no way of knowing what they contain or whether the contents still work; so I decided to try and create guides to the discs by going through all the contents. It was a laborious process (the PC Magazine disc had over 1000 files and the Nautilus disc had 540+), but I did get a result, and guides to both disks now reside alongside the zipped up contents.

The challenge presented by the huge volume of files on CDs, as illustrated above, was also manifested in the maintenance process proper that I started at the beginning of September. The process requires that an inventory is made of all files in a collection (which I achieved by using the National Archives’ DROID tool); and that an attempt is made to open two or three files of each type. Problems identified in this investigation stage can then be addressed. The CDs in the collection (now all residing alongside the rest of the collection in Windows folders) comprise a large proportion of the overall collection, and this overloads the analysis and investigation process. However, many of the CDs are installation disks for the collection’s document management software (no longer used) and for old versions of its indexing software. In subsequent maintenance operations, all such sets of files will be excluded from the DROID analysis: I have decided that the mere presence of such material in the collection is sufficient to signal its previous inclusion – there is no need for it to actually work going forwards. Perhaps this is an example of a sort of additional decision that may have to be made with a digital collection as compared with collections of physical objects. Digital collections are very different animals.

The culmination of the investigation phase is to produce a Project Plan with tasks which are specific enough to enable effort and elapsed duration to be reliably estimated. I got to this point yesterday, and, as per the first task, I have started converting 28 Help files from the old .HLP format to the HTML based .CHM format. The plan prescribes a finish date of 3rd December. After that I shall be producing the final updates to the Preservation Planning templates which I have been refining since 2015, and which are published in the Website of the Digital Preservation Coalition.

Musings on Physical/Virtual Objects

I’ll be exploring the practicalities associated with destroying physical objects by rationalising my mementos, letters, and a few books. However, before diving in (my analogies are influenced by the Olympics which are underway), here’s an attempt to try and order my thoughts about the subject.

Digitisation has added another dimension to the world we perceive: in addition to the physical objects we’ve always had around us, now there are virtual objects that exist in either physical objects we can see and touch such as laptops or hard disks, or in some remote place we refer to variously as system, web, internet, cloud etc.. Some of these virtual objects, such as on-line games or tik-tok videos, are new entities that did not exist previously; other virtual objects have actually replaced physical objects that would have been created, for example, letters, LP records, printed photos, brochures etc.. A third category of virtual object replicates physical objects, for example, a scan of a physical document/painting or a 3D scan of a piece of pottery. I don’t have any numbers associated with the objects in each of the above categories; but my instinct tells me that the quantity of physical objects that we humans are now dealing with is somewhat less than it would have been if digitisation had not occurred – this despite the undoubted increase in physical objects spurred by digitisation and the growth it has spawned. My instinct also tells me that the ratio of digital to physical objects that individuals interact with is steadily increasing year by year; and that our perception of the things that make up our lives, is changing from being almost totally physically oriented to one in which the virtual is assuming a growing normalcy and importance.

Within this shifting landscape, the options we have for dealing with physical objects are changing. Where we once chose paper, we might choose to go paperless; where we once filed we might choose to scan; where we once kept an object we might choose to simply keep a photo; and where we once may have simply destroyed an object we now might digitise it first and then destroy it. These are the sorts of choices most of us are now making quite often; and they have a number of potential impacts:

  • All these choices affect the number of physical objects in the world.
  • The changing mix of physical/virtual objects in the world, and people’s perception of both types of objects, are probably going to affect what people collect, and the composition of collections eventually acquired by curating institutions.
  • When we choose the virtual as an alternative to creating a physical object we are reducing the number of physical objects that our future progeny will encounter.
  • When we choose to replicate a physical object virtually, and then destroy the physical object, we both reduce the number of physical objects that our future progeny will encounter, and prevent our future progeny being able to experience the physical essence of the object – something will have been lost.

The decision to destroy an object is often considered with either an impulse of certainty (as in the case of destroying evidence of a crime), or a tinge of regret (as might be the case with old love letters). Sometimes both emotions may be present (as perhaps with writings produced in our youth which may be both embarrassing yet integral to our past). Such feelings are not the only things that affect the decision. Others include:

Space: a shortage of space may dictate that some things have to go.

Relative age: at 70, an item from one’s youth may or may not be more or less precious than an item from your 70s when you are 90, or an item from your teens when you are in your mid-twenties.

Representation: the emotions that an object conjures up by reminding you of people or events.

Uncertainty: the possibility of wanting or needing the object again in the future must be set against the certainty that the destroyed object can never be reconstituted.

Legacy: the knowledge that other people will be encountering the objects after one’s death, and a possible accompanying desire to bequeath things to others, may inspire notions of organising and rationalising one’s possessions.

In summary, the landscape of our relationship with physical objects is changing. We are a very long way from being physical-objectless, but the direction of travel seems clear – as illustrated by this quote I read in today’s Guardian, “Zuckerberg believes the internet will take on an even bigger role in people’s day-to-day lives in the future, and instead of interacting with it via mobile phones people will be immersed via virtual reality headsets. He said Facebook would transition from a social media platform to a “metaverse company”, where people can work, play and communicate in a virtual environment. Zuckerberg said it would be “an embodied internet where instead of just viewing content – you are in it “[Neate & Rushe]. Now, of course, we should treat such conjectures with a healthy degree of scepticism – but Facebook does have over two billion users…

Whether all this really matters is difficult to say right now – things are changing at a speed which gives us little time to see impacts and make choices. However, our relationship with objects is pretty fundamental, so we should keep an eye on it and try to ensure that we understand what’s going on. My explorations in this journey will attempt to make a small contribution to that understanding.

Neate, R. & Rushe, D., Google, Apple and Microsoft report record-breaking profits, The Guardian, 27th July 2021.

The Halfway House

People have always kept precious physical objects and then destroyed some of them for a variety of reasons. However, digitisation has invaded this relationship by offering a halfway house in which a virtual representation can be retained while still destroying the object. This journey seeks to investigate the impact this might be having on collecting and collections; and on the rationales associated with keeping and discarding. I have already encountered this phenomenon in other investigations documented in this blog – in particular the Digital Age Artefacts journey, though that focused on rationales for keeping things whereas my focus in this Object Obliteration investigation will be the thinking associated with destroying things. The Electronic Bookshelf journey is also relevant since it investigated what digital material would have to be provided to persuade an individual to destroy a book (the follow-up Electronic Story Board journey may also provide further insights). However, this will be a more focused look at the issues associated with completely obliterating an object for ever, as informed by my continuing search for space in my study (space – often the final frontier on which keeping/destroying decisions have to be made) among my books, mementos and letters.

Pandemic constrained Plans

The second of our investigations involved me marking up 19 documents on which I had previously placed sidebars next to parts of the text between 22 and 39 years ago. This was completed in August last year, and, since then, we’ve been analysing the results. My collaborator, Peter Tolmie, made a numerical assessment of the match between my mark-up of the original and my new mark-up; and I reviewed my original and new mark-ups and documented my comments on what I had done. We then recorded a one-hour video conference in which we discussed what had occurred with three of the documents – we had no time to discuss any more because such a wide variety of general points emerged from this conversation. The recording was processed through the Otter transcription service from which a written transcript of the session was produced. Peter is now planning to produce an overall interim report on our findings to date.

Our original plan (documented in the post for 10th August 2019) involved undertaking a further similar investigation of mark-ups on documents; but this time conducted in the form of a face-to-face interview in order to be able to examine the reasoning for what is marked up in greater detail, and to discuss the findings of the previous two investigations. However, the pandemic has interrupted these plans. We still feel this final investigation is worthwhile undertaking if and when travel restrictions are lifted. In the meantime, however, the Otter transcript and the interim report Peter plans to produce, will be the primary outputs from which we hope to be able to summarise some hard and fast findings in a post later this year.

The Gallery Debut

The Trophy Gallery that I’ve created consists of some 150 Thumbnails representing 67 Publications, 97 Reports and 42 Awards & Certificates. They were assembled in a single Powerpoint Slide of custom size 30 x 88 cm – the size of the frame that they were to be mounted in. To print it, I split the slide into two and printed each one out at different ends of a sheet of paper of width 30.2 cm (the maximum physical width my A3 printer would take) and length 86.2 cm (twice the paper length permitted by the printer software). It took a little trial and error but I eventually was able to adjust the position of the contents of each slide so they joined up satisfactorily. The print was then placed in the frame, and the frame fixed to a place on my study wall where it is easy to see the thumbnails and to read the relevant numbers.

The end result does look reasonably presentable and is certainly accessible. The accompanying numbered electronic files are all in the SideBooks app in my iPad as shown below.

I have now packed up the Publications and Reports taking space on my bookshelf. They are all in a case and stored away in the loft.

I have to say, I did feel a tinge of sadness as I took the volumes off the shelf and put them into the suitcase – after all they not only represent some of my achievements, they are also old friends that have been with me for many years, and that remind me of times past. I think I’m unlikely to see them again. The next time they see the light of day may well be when my nearest and dearest have to decide what to do with them. Of course, this isn’t anything particularly new – no doubt there have been millions of relatives in the past who have been presented with such a dilemma (it IS a dilemma because such material is massively uninteresting to the vast majority of the population). However, something has changed: the rock solid undisturbability of those volumes on the bookshelf has been breached by the IT hurricane, and their essence can be reproduced in other ways. Authors, Owners, and Recipients, all have other options, which may, in turn, give rise to other interests, motivations, and desires.

I must confess that I was too timid to take the ultimate step of banishing the books I’ve had published, to the loft. I retained those select nine volumes to still sit on my physical bookshelves and proclaim to myself and the world that I made a tiny contribution to the development of our race’s awareness and understanding of the universe and our place within it.

With the physical items stored away, I’m going to give myself about a year to see how I get on with my Electronic Trophy Gallery. I’ll be reporting my verdict here towards the end of the year.

Designing and Assembling the Gallery

I started this journey off with the idea of having a numbered list of the trophies down the middle of a page surrounded by equivalently numbered thumbnails of one or more of the following for each trophy: Publication cover, First page, Place of creation/achievement, Associated people, Topic. However, I soon realised that not only was there insufficient space for lists and extra thumbnails, but that actually they were superfluous. All I needed was a single thumbnail to remind me of a particular achievement and a number to enable me to access the associated file.

I experimented with thumbnails of longest side 3, 2.5, and 2.3 cm; but ended up with 2.2 cm because of the limited space that I had. However, that size seems to be quite adequate, as does the 12 point font size that I used for the numbers latched onto each thumbnail. I think they’ll be plenty big enough to be able to discern when the Gallery is on the wall.

Choosing the thumbnails was sometimes easy, as in the case of a cover of a book or a photo of a swivelling workstation that I designed; but sometimes it was very difficult – especially for reports on esoteric subjects with first pages comprising entirely of small text, for example, the X500 Schema document. In those cases I resorted to overlaying some Text Headings in large bold font on the front pages; and, in three instances, I just put large coloured text in a box (for example, ‘Radii Lessons’). In some cases, it seemed appropriate to represent two or more items with a single thumbnail, and in these cases I placed all the related numbers around the edge of the thumbnail. All of these approaches seem to produce usable results.

The contents of the Gallery are in three parts – Publications (67 items), Reports (97 items), and Awards & Certificates (42 items). The Publications were self-selecting – if I’d had something published it was on my publications list. The choice of reports, however, was at my own discretion. At the time when I selected them from my archive of work documents, I’d been looking for significant pieces of work. However, about a year later I had produced a book of my IT experiences in which I had included supporting images, which must have given me different perspectives on some of the material. Consequently, when I came to assemble the reports section of the Trophy Gallery, I was surprised that some of my original selections either didn’t seem worthwhile including or that some things which I thought should have been included were missing. I guess it just goes to show that the things we choose to focus on and the stories we tell can vary hugely depending on our motivations and accumulated experiences at any one time.

Assembling the Awards & Certificates was a different experience again. For a start, they were all over the place – photos of trophies I’d thrown away, certificates in envelopes in drawers, items indexed in sets of mementos, and engraved glass tankards in kitchen cupboards. They were also different because many were very old from my childhood, and I’m sure several were just missing (for example the china lamb that I got in primary school for my Times Table, and that I think I must have sold off in my train trunk in my late twenties – a great shame). So, I made no attempt to create a definitive list – I just assembled the ones’ that I could lay my hands on quickly. Perhaps that was fortunate because even the subset I assembled seemed somehow very trivial, and I felt embarrassed to include some of the items. Is the bronze swimming personal survival certificate I got when I was 14 really a substantial achievement to be celebrated? Does the Certificate for coming second in the Intermediate Boys High Jump at the North East Derbyshire Inter-Schools Championships celebrate my athletic capabilities?

In fact, going through them made me realise that I’d never been the outstanding performer that I imagined I had been. Nevertheless, they do testify to the fact that I did DO things. The Trophy Gallery is primarily for me, and these are things I can be reminded of. They are all as relevant as each other because they are all true and have a slot in the jigsaw puzzle of my life.

Despite my misgivings about missing and trivial items, the overall assembly of Publication, Report, and Awards/Certificate thumbnails makes for a very crowded assortment of small images on two A3 pages. I suspect that, once the display is framed and on the wall, quantity is going to be more apparent than quality.

Feelings about Achievements

The Electronic Bookshelf and Electronic Story Board expeditions enabled me to eliminate most of the physical books that were no longer central to my interests. However, there is still one set that is taking up precious bookcase space in my study – the originals of my publications and reports. They are rarely accessed, but I do value them as physical testimonies to my achievements, and I’m not prepared to destroy them and be left with just their virtual equivalents. So, I’ve concluded that I would be happy to have them safely packed up and stored in a box in the loft, provided that I can install some physical reminder of them and that I am able to access their contents electronically.

My current thoughts about how I would do this entail listing the numbered items down the middle of a page and surrounding them with numbered thumbnails of any or all of the following elements which apply to each item:

  • Cover of the publication
  • First Page of the paper, article or report
  • Photo of the place it was created or delivered
  • Photo of any person(s) strongly associated with it
  • The Topic in abbreviated words and/or photos

This should enable the viewer to go from a surrounding thumbnail to the relevant item on the central list. A correspondingly numbered PDF will be provided to enable the viewer to look at the full contents on either the iPad or the Laptop.

While trying to think of an appropriate name for this journey, I came to realise that, in a way, these publications and reports are trophies of my work achievements. However, from that perspective, I realised that I have other physical items which represent other types of achievements – Cups and Tankards for my athletic successes; and Certificates for academic and other types of achievements. So, I decided to combine these other types of physical objects with my publications and reports in a single overall journey to explore the Electronic Trophy Gallery.

I already have complete lists of my Publications and Reports, and all the associated PDFs. So, my first task will be to come up with an equivalent list and associated PDFs of my cups, certificates etc.  Then it will be a matter of going through all the items to identify appropriate thumbnails. I currently envisage arranging all this material on a vertical page and mounting it in a 32x90cm frame which is currently unused in the loft and which will conveniently fit in one of the few available spaces in my study walls. Whether this approach will be viable, usable and useful has yet to be established.

New Boards Live

I finished compiling the physical story boards and associated self-contained PDFs on 24th December, and would have had them all printed out and up on the side of my bookcase before Xmas but for running out of printer ink (very poor stock control). Anyway, it arrived yesterday and I’ve now completed the job.

Regarding the cross-linking between Story Boards, this was straightforward for the in-Laptop version in which a simple link to the file of the relevant Story Board could be specified. However, in the iPad version it was necessary to include a copy of the target Story Board into the body of the relevant self-contained PDF.

I also encountered a few technology problems with the way the Sidebooks app handles PDF documents (inability to deal with sound elements, different sized documents, and links to multiple embedded photos), and these took time to understand and work around; and, in three cases required different versions of the Story Boards to be produced. In contrast, the in-Laptop solution was relatively trouble-free. This illustrates the problems of trying to produce a solution by integrating different systems.

While I was laminating the new Story Boards, I was struck again by how relatively permanent these Story Boards are. Their very physicality exudes factuality and veracity; even though I know how flimsy their fidelity is.

I will now live with these Story Board for several months, changing the Story on display from time to time, and perhaps inviting members of the family to experience them and comment. My personal overall assessment, and my thoughts on how this set of diverse Story Boards compare to the original all-book-related Story Boards, will appear here before the end of 2022.

Laptop Story Board Musings

Earlier this week, I completed the laptop version of ESB2: a PDF with thumbnails of all 35 items on the first page, each linked to its own story board fully populated with links to material elsewhere on the laptop.

Bear in mind that my following observations on this construction represent just a single point of view, and that of a builder rather than a viewer:

  • The main practical difference between the laptop version and the iPad version is that the size of a target file is immaterial in the laptop version, but impacts the overall size of the self-contained iPad PDF. For example, one link was to a 200+ page file of letters from a friend. For the iPad version I may only select a subset of the pages to include in the PDF. For the laptop version, I have defined the link to open on a particular page, but the rest of the file is available to browse through if the reader so desires.
  • In the laptop version only one single file can be linked to, whereas, in the self-contained PDF version, several different items (photos, for example) can be included at the destination of a link in the PDF either together on a single page or on multiple pages.
  • As I constructed the laptop version, I became aware that for both the laptop and the iPad versions, any story I create is only one of many possible stories that could be constructed for a particular Story Board. If another person had created it, or if I had created it a few years earlier or later, the Story Board would almost certainly be different. However, in as much as these Story Boards now physically exist, they will have a far more powerful influence in the future than those stories that didn’t get created, or than the stories that may be generated when people in the future recount their memories of these or related topics.
  • I find the main Index page both satisfying and flat. It is satisfying because I know that everything is accessible from that one single page; but it inspires no excitement because somehow there is no coherence among the 35 separate items represented on it. Somehow the 7 different types of material (pre-marriage mementos, post-marriage mementos, letters, loft items, music, photos, music and books) just produce a muddled interference with each other.
  • I set out to ensure that there were a few links between each of the 35 Story Boards; and have ended up with at least five such inter-relationships. However, I have no thoughts as to whether this is useful or not – I just feel instinctively that there would probably be many such relationships in a large collection of personal items, just as the many links between internet web sites enable almost endless web surfing. However, I am only able to explore this capability in the laptop version because the application within which the self-contained PDFs will be held on the iPad (SideBooks) does not enable links between PDFs.

I’m now proceeding with the final stage of this exercise – to construct the self-contained PDFs, and the associated physical story boards to hang on the side of my bookcase.