The Index to my PAWDOC collection of work documents is in a Filemaker database, and I’d recently been told that Filemaker had added some AI functionality into its product, and that it also had a number of AI-related videos for users to learn from. So, at the start of this investigation into providing AI support for larger collections like PAWDOC, I took to watching some of the videos. One was entitled ‘Learn about AI prompting basics (30May2024)’ and it outlined some of the key things to remember when constructing an AI prompt. In particular:

• Clearly define the audience.
• Define your goals.
• Provide a step-by-step approach (do a, b, c, etc).
• Include examples of what you want and say why they work.

At the time, I was thinking about putting a protective covering on the top of books to stop dust settling on the top of the text block, and I thought I could get AI to produce an outline which I could print on card and cut out. So, I set about developing a prompt using the guidelines listed above. It turned out to be an interactive process which took me at least a dozen attempts before I was satisfied with the results – illustrated in the image below.

The way the cover works is that you score a line where the image ends, cut away the two small rectangles on either side, and fold down the resulting tab and place it inside the spine of the book. The rest of the cut-out then sits on the top of the text block as shown in the top-down view of some of the books below. When reading the book, the cover can act as a bookmark.

Perfecting the prompt was a matter of getting the cut-out lines to be drawn in the right places; of getting the text boxes to be of the right size with minimal empty space; of getting the whole of the specified area to be covered with interesting illustrations; of figuring out whether its best to have it produced in a Word or Powerpoint document (I ended up with Word); and of realising that Claude AI would produce more interesting illustrations than ChatGPT (particular AI models are better at some things than others). Getting all of these things right took rounds of experimentation until I came up with the final prompt.

I had the same experience when investigating AI support for my large PAWDOC document collection for which I devised 6 evaluation questions, each of which were to be asked of a subset of the PAWDOC Index and file titles (the need to use subsets was because, unlike AI support for the Mementos collection which I have written about in earlier posts, the PAWDOC Index and file titles have some 16+ times the number of characters – a size which could not be ingested and used in a single prompt in today’s AI systems. So, I decided to see what sort of results could be obtained by splitting the information into 10 subsets, getting answers for each subset, and then stitching the 10 answers together). The 6 questions I specified were:

1. List all the people named in this part of the Index and its associated files, and the organisation they belong to if any.
2. Describe Paul Wilson’s career over the period covered by this part of the Index and its associated files.
3. What significant changes in Information Technology occurred during the period covered by this part of the Index and its associated files.
4. Document all the travel undertaken by Paul Wilson over the period covered by this part of the Index and its associated files.
5. What training was undertaken by Paul Wilson over the period covered by this part of the Index and its associated files, and how important were particular elements to his subsequent career.
6. What are the strangest or most unusual things to be found within this part of the Index and its associated files, including unlikely coincidences, and events with unexpected outcomes.

These all look fairly straightforward but the actual prompts I ended up with were considerably more extensive. For example, the prompt for the sixth question was:

PAWDOC is a work document collection built up from 1981 to the present day. The attached Guide outlines how PAWDOC is constructed and what it contains. Also attached is a subset of PAWDOC’s Index and the digital files associated with those index entries. Note: None of the actual PAWDOC documents have been provided. The following request is to be undertaken using just the information in the Index entries and the File Names provided in the attached files: identify the training that was undertaken by Paul Wilson over the period covered by this part of the Index and its associated files, including the objectives and composition of the training, how it was delivered, how much time was spent on it, how relevant it was to his subsequent career, and citing the Reference Numbers from which the information was obtained. NB. Be sure to inspect all supplied data first, then analyse.

All the prompts required a number of iterations before they were producing good results. They are reproduced in the attached document together with additional prompts to stitch the 10 responses to each question together. Note the prompt for the first question which was especially demanding.

These experiences are not novel – they merely reflect a well-known fact that to get the best out of an AI capability the prompts need to be well thought out and will likely require several iterations before they are deemed satisfactory.