It’s been well over a year and a half that I displayed the second set of storyboards on the side of my bookcase – plenty of time to experience what its like to live with them. However, I can’t say I’ve given them a great deal of use. Every now and then I’ve selected another sheet, hung it on the display, and had a glance at its contents – but perhaps only a dozen times across the period. That’s not to say I didn’t find the contents interesting. On the contrary, I found I was coming across points I’d already forgotten about, or wouldn’t ordinarily connect directly with the main topic of the storyboard – a finding which I also experienced in the first trial (see the heading facts/Discovered/Re-discovered in the ESB Evaluation Results post). Other members of my family were also interested when I showed them the display and explained what it was all about – but not enough to spend any further time exploring, even when some of the topics were things they were familiar with. The fact that this set of storyboards included several other types of items other than books (which the first set was limited to), seemed to make no difference to myself or to family members. Perhaps this overall reaction is understandable since viewers have no real reason to examine the installation in more detail. Like a picture on a wall, it provides an interest for the eye but demands no closer inspection. People need to have a reason, an incentive, to do more.

Of course, the fact that this is a static, paper-based, display conceals its hidden content: it does not connect with a modern person’s notion of interactivity. A screen-based display, as originally envisaged for the storyboards, might perhaps inspire a greater curiosity; though, beyond an initial inspection, I suspect there would still have to be a rationale for exploring. Maybe such displays are, in fact, just pictures on a wall – but with an additional dimension of background links which have to be displayed automatically, in turn or in other configurations, to enable its viewers to experience its complete composition. Would that inspire people to interact with the display, or would they simply stand and watch? Well, perhaps a bit of both: at least one doesn’t preclude the other – though probably best not at the same time unless by picture-in-picture.

As with the first trial, I had created PDFs of the storyboards for display in the Sidebooks app in my iPad. In addition, I created an equivalent display in my laptop. The latter was able to link out directly to associated items held elsewhere on the laptop (as opposed to including all the material in a single PDF for the Sidebooks display). I did look at the Sidebooks storyboards occassionally in order to follow a particular link. However, I rarely, if ever looked at the laptop version. I guess the iPad was simply closer to hand and provided more immediate access. The laptop version is certainly a powerful beast providing access to the complete version of the books, for example, which were not included in their totality in the Sidebooks PDF. However, on trying it out while writing this post, I found that it annoyingly shuts down the master front-end index PDF when a link is selected and another file is opened. No doubt this will fixable somehow or other, but it’s another example of how there will always be glitches when trying to interlink systems and files.

Inspiring viewers to access storyboards may be a little difficult, but that is the least of the challenges associated with such displays. Their production demands some creative energy and is extremely time-consuming: individuals would need a huge incentive to undertake the work for rewards which are way in the future. Perhaps, it would be easier to undertake the analysis as each item is acquired, instead of trying to produce a whole set in one go. However, people have difficulty in just labelling and placing newly acquired items in an organised store, let alone going through such a rigorous analysis process as well. In discussing this with my son, he pointed out that, in any case, one’s feelings about an object may be very different at the point of acquisition from those several years later. This thought prompted him to develop the notion of revisiting an object periodically to build up a picture of how one’s feelings for an object were changing over time – a development that could be displayed graphically on a screen. However, this would demand yet more work from innocent owners who just want to add something to their collections. There is one piece of information which would not require any work to generate and which could be used by a system to generate a timeline display, and that is the date of creation of the digital object. That might be interesting – but would only be factually correct if all the items were born digital; the date of items which were acquired in physical form and digitised later would bear an incorrect acquisition date.

Summing up my experiences with my two sets of storyboards, I have to conclude that, while the results are very interesting, the work to produce them is probably going to be too great for most collectors. Perhaps the circumstance which would provide the greatest incentive to undertake such work is when a collection is to be digitised prior to the disposing of the physical items – as was the case in my first storyboard trial when I digitised 36 books to remove them from my book shelves and destroyed them in the process. The heartache an owner might feel in undertaking such a final no-going-back act, might inspire the production of a set of storyboards in memoriam.

I finished compiling the physical story boards and associated self-contained PDFs on 24^th December, and would have had them all printed out and up on the side of my bookcase before Xmas but for running out of printer ink (very poor stock control). Anyway, it arrived yesterday and I’ve now completed the job.

Regarding the cross-linking between Story Boards, this was straightforward for the in-Laptop version in which a simple link to the file of the relevant Story Board could be specified. However, in the iPad version it was necessary to include a copy of the target Story Board into the body of the relevant self-contained PDF.

I also encountered a few technology problems with the way the Sidebooks app handles PDF documents (inability to deal with sound elements, different sized documents, and links to multiple embedded photos), and these took time to understand and work around; and, in three cases required different versions of the Story Boards to be produced. In contrast, the in-Laptop solution was relatively trouble-free. This illustrates the problems of trying to produce a solution by integrating different systems.

While I was laminating the new Story Boards, I was struck again by how relatively permanent these Story Boards are. Their very physicality exudes factuality and veracity; even though I know how flimsy their fidelity is.

I will now live with these Story Board for several months, changing the Story on display from time to time, and perhaps inviting members of the family to experience them and comment. My personal overall assessment, and my thoughts on how this set of diverse Story Boards compare to the original all-book-related Story Boards, will appear here before the end of 2022.

Earlier this week, I completed the laptop version of ESB2: a PDF with thumbnails of all 35 items on the first page, each linked to its own story board fully populated with links to material elsewhere on the laptop.

Bear in mind that my following observations on this construction represent just a single point of view, and that of a builder rather than a viewer:

• The main practical difference between the laptop version and the iPad version is that the size of a target file is immaterial in the laptop version, but impacts the overall size of the self-contained iPad PDF. For example, one link was to a 200+ page file of letters from a friend. For the iPad version I may only select a subset of the pages to include in the PDF. For the laptop version, I have defined the link to open on a particular page, but the rest of the file is available to browse through if the reader so desires.
• In the laptop version only one single file can be linked to, whereas, in the self-contained PDF version, several different items (photos, for example) can be included at the destination of a link in the PDF either together on a single page or on multiple pages.
• As I constructed the laptop version, I became aware that for both the laptop and the iPad versions, any story I create is only one of many possible stories that could be constructed for a particular Story Board. If another person had created it, or if I had created it a few years earlier or later, the Story Board would almost certainly be different. However, in as much as these Story Boards now physically exist, they will have a far more powerful influence in the future than those stories that didn’t get created, or than the stories that may be generated when people in the future recount their memories of these or related topics.
• I find the main Index page both satisfying and flat. It is satisfying because I know that everything is accessible from that one single page; but it inspires no excitement because somehow there is no coherence among the 35 separate items represented on it. Somehow the 7 different types of material (pre-marriage mementos, post-marriage mementos, letters, loft items, music, photos, music and books) just produce a muddled interference with each other.
• I set out to ensure that there were a few links between each of the 35 Story Boards; and have ended up with at least five such inter-relationships. However, I have no thoughts as to whether this is useful or not – I just feel instinctively that there would probably be many such relationships in a large collection of personal items, just as the many links between internet web sites enable almost endless web surfing. However, I am only able to explore this capability in the laptop version because the application within which the self-contained PDFs will be held on the iPad (SideBooks) does not enable links between PDFs.

I’m now proceeding with the final stage of this exercise – to construct the self-contained PDFs, and the associated physical story boards to hang on the side of my bookcase.

Yesterday I finished the last of the second set of 35 story boards. Its been a long and tedious job. Tedious not just because its quite hard work putting these things together, but also because it’s not an exciting new thing I’m exploring – it’s a second time around using the same old format to explore how it works out with different types of artefacts. Anyway, it’s done now, so it just remains to create the links to all the additional material on each of the 35 pages. However, while its fresh in my mind, I’ve listed below some of my observations as I constructed the story boards:

• The general procedure I followed was: first import an image of the artefact in question, then copy and paste each of the pre-recorded thoughts, and then add in the supporting material. Having the pre-recorded material was very helpful, and made it easy to get a quick start on the board.
• Sometimes I collapsed two or more thoughts together into a single piece of text; sometimes I adjusted the text; some of the thoughts I just left out; and sometimes I created new text. Such decisions were taken depending on what was most appropriate in order to tell the story. ie. the pre-recorded thoughts were simply the building blocks; the shape of the story was created when the elements were assembled together on the story board.
• On a number of occasions, I used the net to acquire additional information and images to support the story being constructed.
• On at least one occasion, I realised that the pre-recorded thoughts contained errors. For example, I recorded that the air conditioning was noisy on a particular foreign holiday; but realised in the course of creating the story board that it was, in fact, on another holiday that we had had that experience. It reminded me yet again, that humans often remember things incorrectly, and that one must always be aware of that possibility; and that, by setting down such erroneous rememberings in a physical artefact like a story board, incorrect information is given a degree of credence which may be hard to dislodge.
• I found it was hard to come up with variety in the way I was presenting each story board: I got into habits of where I was placing things on the story board, what font sizes I was using, and my use of coloured text and text boxes. I think this was partly because it was a prolonged process and I just wanted to get on and finish it; to get arty about each story board would have just taken too long.
• I became very aware that I was creating a particular version of the story in question – a version that was fashioned from the state of my mind at that point in time, and from the particular artefacts I was coming across and assembling to support the story. Each story that emerged was very much just one point of view out of the totality of views that were available from the all the possible memories and artefacts that could be brought to bear. However, that single point of view will gain a high degree of visibility, credibility and endurance by virtue of becoming a physical artefact (digital or printed) in its own right, to the probable detriment of all the other possible points of view.

The first ‘Record’ stage of the second ESB trial was completed earlier this week. Five examples from each of seven different collections were randomly selected:

• Mementos from two separate collections, each with their own index
• Letters scanned and held digitally on my laptop
• Items stored in the Loft and recorded in the uGrokit app
• Music held on my laptop
• Books scanned and held digitally on my laptop
• Photos held digitally on my laptop

Random selection was achieved by using the free random number generator at Random.Org, and applying it to serial numbers in indexes, or to the numbers of folders and files in the laptop, depending on how each different collection was managed/stored.

Taking each collection in turn, in the order listed above, each item was considered and the thoughts generated were noted. The thoughts were then categorised, and a set of generic Category Prompts derived; these were then used to prompt thoughts in the next collection, and the process was repeated. As this exercise was completed across each collection, the list of generic Categories was built up. The final set of Category Prompts, expressed as a series of questions, is listed below:

Use the questions below to comment or speculate on the item

1. What facts do you know, don’t know, or had forgotten about this item?
2. What feelings does this item inspire in you?
3. What does this item remind you off
4. What opinions would you put forward about this item
5. What action has the item prompted you to take with what results?
6. Does this item inspire any thoughts about Collecting and Indexing?

The items selected, and their associated original thoughts, were then revisited using the above questions to identify any further thoughts; and all thoughts were then allocated to one or more of the six categories. Statistics associated with this whole process are shown in the following table.

The next stage of this work will use the Recorded Thoughts to Create a separate Story Board for each item.

[NB. While undertaking the above process, I started to think about how the creation of Story Boards could be automated. One possibility, which I shall ponder in the coming months, is to pose a question to users when they store collection files, for example, ‘What emotions and thoughts do you want to remember about this item?’. This could be augmented by the full set of Category Prompts if requested by the user.]

The first iteration of this Electronic Story Board (ESB) work indicated that the concept might work for other types of items than books. So, I am planning to undertake another trial using mementos, photos, letters, household files, music, and a few more books. My intention is to explore how to include these different types in ESBs, and to see how they might inter-relate.

I shall continue to use the physical apparatus from the first ESB trial (designed to hold and display A4 laminated sheets); and to create self-contained ESBs in A4 PDF files for use within SideBooks on my iPad. These ESBs have to be self-contained, with subsets of Supporting Information included behind the first page, as I am not able to create PDFs with links to other files within SideBooks.

However, it is possible to create PDFs with links to other files within my laptop. Therefore, in addition to the set of PDFs for SideBooks, I will create an extra set with links to files of other items, for use on the laptop. For example, instead of including the first few chapters of a book within a self-contained ESB PDF, I will just include a link to the file containing the whole of the book’s contents; or, instead of including a photo within the PDF, I’ll just include a link to the relevant jpg file. The result will be smaller ESB files and, where appropriate, all the contents of each piece of Supporting Information will be accessible. This will provide a much closer simulation of the ESB system that I envisage – albeit without the immediacy of being able to manipulate a large wall display in front of you, and/or of interacting with a portable iPad. The possibilities of interacting with the ESB using voice commands will also be explored by using the Amazon Echo device in my study to call up music.

The physical apparatus being used will limit the number of ESBs to about 35 – around 5 of each type. To select the items concerned, I intend to use a random number generator to choose the first two or three mementos, and then to use any items (of any of the types being investigated) that emerge in the process of recording Associated Information. I will continue to apply this approach for each type of item until enough instances of each type have been identified. The aim is to produce both a random selection of items, and at least a few inter-relationships between the items.

To establish the Associated Information for each item, an initial assessment will be made and written up in free text form. When all items of a particular type have been assessed, a set of Category Prompts for that type will be derived from the set of free texts, and then applied to each item of that type (this process can be short-circuited for the Book items since Category Prompts for Books have already been identified in the first ESB trial). The Category Prompts will always include an ‘other comments’ section to ensure that all the points in the free text can be captured within one or other of the responses to the Category Prompts.

Shortly after creating this new set of ESBs, I’ll post a summary of the experience and of my initial impressions, here in this blog. A more detailed evaluation will then be conducted after the ESBs have been in place on the side of my study bookcase for about 15 months.

The investigation described in previous entries used Books as the type of collection to explore the concept of the electronic story board. It showed that the collected material on an ESB tends to expand the reader’s attention beyond the particular item concerned; and that the set of information that the reader brings to mind when thinking about the item concerned becomes an entity in its own right when assembled together on an ESB. It also showed that a reader often explores the item itself, and some of these additional facets, when looking at an ESB.

If this is the case for books, it seems likely that the same effect can be achieved for other types of items – Mementos, Photos, Letters, Files – perhaps even Music. Mementos would seem to be particularly amenable to this kind of treatment since they are likely to evoke even more and stronger memories and feelings than Books. However, it must be remembered that, in the investigation using Books just completed, a particular set of categories was identified to help the owner bring to mind the related information; and it may well be that different sets of categories may be required for other types of items.

If ESBs can be used for different types of collections, each with a different set of category prompts, it may be worthwhile creating a model of all the key components with standardised terminology. I propose the following:

• Collection Type (books, mementos, photos, letters, files etc.)
• Recorder (the person who is identifying the associated information)
• Category Prompts (the set of categories used to prompt the Recorder to identify associated information)
• Item (the particular item within a collection that is the main focus of a particular ESB)
• Associated Information (the information generated by a Recorder to surround a particular Item)
• Rig (a standard layout for the way Items and their Associated Information are arranged on a collection of ESBs)
• Supporting Information (additional material related to Items and to Associated Information, which can be accessed by links from an ESB)
• ESB (a numbered display of an item and its Associated Information, with links to their Supporting Information)
• Reader (the person who looks at an ESB and who may also access its links – this may be a different individual from the Recorder)

The diagram below illustrates how these components might fit together.

I’ve now been through all 34 ESBs and made notes of between 30 and 300 words on my interaction with each one. This entry analyses those notes and derives some implications for the design of ESBs. The analysis assessed each part of the notes text and identified specific actions or observations as an itemised list. After completing this exercise for all 34 books, the itemised lists were inspected and generic statements derived for each item. For example, specific item d) for Book No 17 was ‘Read press release of merger’, and from this the generic statement ‘Prompted me to look at the related facts in the iPad version’ was derived. The generic statements were gradually standardised as the analysis proceeded and during a subsequent refinement process. The standardised generic statements were then grouped into two main sets (ESB Composition, and Reader Behaviour) and placed into one of seven categories – Layout, Content, Impact, Information access/search prompted, Facts discovered/re-discovered, Thoughts generated, and Reflections about the book. The generic statements, and the number of books for which a particular statement occurred, are shown in the following table.

Observations relating to ESB composition

The observations relating to the design of the ESB fall into the following categories: Layout, Content, and Impact.

• Layout: All the ESBs were assembled using a standard template in which the book’s spine was placed in the centre of the page with the front cover immediately underneath it. Related points were placed around these two elements with those more intimately related to the book being closest to them. However, some of the spines and covers were smaller than others, and this clearly made a difference. In one case the spine was not recognisable, and in another it was mistaken for the wrong book. Another observation recorded that a cover was particularly noticeable. A related observation noted that some text on one of the related facts on an ESB was too small to read.
• Content: There were several remarks about the range of material on the ESBs such as ‘Lot in the ESB’, ‘very interesting ESB’, ‘ESB seems so complete’, and ‘the range of topics on this ESB is relatively narrow’. One observation pointed out that some information on the ESBs is more familiar than other information. In three instances the presence of photos was remarked upon in a positive way, for example ‘has photos of people I know’. A feature which was not explicitly remarked upon, but which was identified during the analysis process, was that there were four instances of two ESBs which were related in some way or other.
• Impact: Some remarks made it clear that some ESB’s distracted attention from the book and appeared to be texts in their right. For example, ‘With the ESBs you no longer focus on the book (which is what you do with a physical bookshelf) but on all the other info around it’, and ‘ESBs have become entities in their own right and the books are fading into the background’.

Observations relating to Reader Behaviour with ESBs

The observations relating to reader interaction behaviour with ESBs fall into following categories: Information access and search, Facts discovered/re-discovered, Thoughts generated, and Reflections about the book.

• Information Access and Search: Quite often, a particular element on the ESB seemed to catch the eye (8 specific instances were noted). For example, ‘Noted that Forbes in 2002 voted it one of 3 most important business books in the last 20 years’, and ‘Noted that though it is the 53^rd edition it was still fetching £10 on eBay’. Following an initial look at the ESB, I typically sought additional information either by following the link to the book itself (15 instances noted), following the links to the related information (another 15 instances noted – not necessarily the same 15), and conducting a search on the net (four instances). It is striking that several of the cases in which additional information was sought, involved reading texts I had written (6 instances) or reading documents related to work I had done (8 instances).
• Facts Discovered/Re-discovered: In the course of seeking out additional information, I noted 13 instances in which I rediscovered information I’d forgotten – nine items I’d forgotten since producing the ESBs, and 4 items I’d forgotten a long time previously. For example, ‘The ESB confirmed I visited the Media Lab twice and with whom’, and ‘Noted that at least one was written while I was at NCC’. Furthermore, there were eight instances in which I discovered new facts from within the material that the ESBs were linked to, or from the searches I conducted on the net, for example, ‘Last page of the book refers to collaboration between NCC and CIMTECH which I’m not sure I heard about’, and ‘Read Bell’s Wikipedia entry and found he was involved in the design of the Vax computer which DEC gave us for Hicom’.
• Thoughts Generated: As one would expect, reading the ESBs and the linked material prompted a whole raft of thoughts. The majority of those noted were related to something I had observed, experienced, or done (17 instances). For example, ‘Reflected that NCC’s demise is a sad story – but not, of course for the commercial operation NCC Group’, and ‘reflected on how right the Future Shock predictions were’, and ‘Read the last page of the first chapter and thought that the Harry Potter books might have been a more pleasurable experience than the films – perhaps true for many books.’. In another case, the experiences generated a simmering emotion within me which were re-ignited on reading an ESB. Another set of thoughts were about people I was reminded of – six instances of these were noted.
• Reflections about the Books: The notes made on the ESBs included several reflections on the books themselves. Many of these (9 instances) were compliments about the books, for example, ‘Hardcopy was a nice design and had lots of useful info – summed up technology and capabilities of the time’, and ‘Was reminded that this is a great read’. A further two instances recorded a desire to re-read the books concerned again. There were seven observations about my relationship with the books, for example, ‘Realised I hadn’t looked at the contents of this book for a long time’, and ‘Book didn’t live up to my expectations’, and ‘Don’t think I ever read this book but watched the film’. In two cases I reflected on the physical characteristics of the book, for example, ‘Glad I kept hardcopy since tabbed books are hard to represent in scans’. Finally, for one of the ESBs, I wondered what had happened to the topics covered in the book.

Implications for ESB design

The amount of material to include on an ESB is totally dependent on the analysis of the owner’s thoughts about the book. Some books will stimulate the owner more than others. Consequently, some ESBs will inevitably contain more information than others, and be more interesting to the owner than others. However, the most significant finding from the observations about ESB composition is that the ESBs become entities in their own right, and that attention is drawn away from the books around which they are structured. Consequently, the fact that some of the book spines and covers were too small to recognise and read, becomes even more significant. No matter how much material is available to include on the ESB, the book spine and cover must be easily readable.

Two other points regarding ESB content emerged from this investigation: photos of people were highlighted a few times, so it seems worthwhile including such items where possible; and it was noted that some ESBs were related to each other. This latter point could be simply dealt with in the physical versions of the ESBs by adding a note such as ‘See also ESB #’. However, with a large electronic display it may be more useful to link directly to the related background information rather than to another main ESB – this aspect has yet to be explored.

Other than these two points, the general design of the ESB’s with the book spine and cover in the centre and other material around it, seems to work well. Of course, with a large electronic display, the constraints of an A4 page would not apply, but the principle of book in the centre with material around it would still apply. However, if the display first presented a bookshelf display of all the spines, from which a book was selected, the ESB would not need the spine and could just display the folded-out dust jacket or the front and back covers – this aspect too has yet to be explored.

Reader behaviour observations indicated that the links to extracts from the books and to related material, were well used and useful. The fact that net searches were made for additional information, and that new facts were identified in some cases, indicates that a facility to enable a reader to add additional material to a fully electronic ESB might be useful. Readers might also use such a facility to record some of the many thoughts which the observations in this investigation make clear are occurring throughout the interaction with a particular ESB.

Yesterday I started an evaluation of my Electronic Story Boards. Its been over a year and a half that I first put them together and since then I’ve looked at them occasionally; referred to them when I needed some specific information; and even forgotten that some information I knew I had was actually on one of them. However, I haven’t yet made a methodical assessment of how interesting, useful or effective they are. I’m going to try and do that by looking at a different story board every day starting with No 1 and working my way through to the final one – No 35.

No 1 is the Levinson book on Pragmatics, and it’s story board effectively summarises my involvement in the Cosmos project. After looking at it, two words immediately came to mind – Rich, and Personal. That one single page is rich in content – every element bringing back powerful memories; and Personal – because all the content is to do with me.

Later on yesterday, I took a look at the electronic version on the iPad. It was simple to find – all 35 story boards are represented as thumbnails on a single Sidebooks screen on the iPad. Selecting the Pragmatics Story Board brought up a full screen image that looked exactly like the laminated version I’d been looking at on the side of my bookcase. It was just as rich and personal, and it also enabled me to click the arrows and bring up further pages of related material. But, interestingly, those further pages didn’t add a great deal to the experience. The sense of wonder and powerful feelings that I felt, were generated by the material on the main story board: the additional material didn’t really augment them. However, I thought, those supporting pages would certainly be useful if you were specifically looking for detailed information.

That was my initial experience in this 35 day evaluation. I’ll make notes as I go, and summarise my conclusions in 5 or 6 week’s time.

This afternoon my Electronic Story Board went live on the side of the bookcase.

Well, perhaps not so much ‘live’ since this is a simulation in the absence of electronic paper; though the electronic interaction part of the installation does exist – it’s just that its on my iPad in a very useful app called Sidebooks.

For the last week I’ve been slogging through each of the 35 Story Boards, getting the numbers of each one in line with the number of the spines on the virtual bookshelf and making minor adjustments and corrections. This has been a laborious process because the front page Story Boards have been created in PowerPoint, exported as JPGs, and imported into the PDFs. Unfortunately, its the front page Story Boards that have the links to all the accompanying pages in the PDFs, so importing a new version of the Story Board requires all the links to be reset in the new Story Board page. Anyway that all got finished yesterday, after which I was able to finish laminating the front page Story Board pages.

Today, I spent the morning creating the physical Story Board with its holder for the laminated pages, Its sticking-out very small screws on which the hole-punched laminated pages are to be suspended, and its holes and cord to suspend it down the side of the bookcase. The result looks reasonable and seems to work: I’ve just been up to it and changed the existing Story Board on display (7. Grief – CSCW A Book of Readings to 16. Pawson – Expressive Systems) and it was quite quick and easy. However, I think its going to be a while before I know how I  feel about this PHYSICAL Story Board, because its significantly different from the ELECTRONIC PDF versions that I’ve spent a lot of time creating over the last month. Watch this space.