{"id":2923,"date":"2026-07-11T17:53:03","date_gmt":"2026-07-11T16:53:03","guid":{"rendered":"https:\/\/www.pwofc.com\/ofc\/?p=2923"},"modified":"2026-07-11T20:52:06","modified_gmt":"2026-07-11T19:52:06","slug":"phase-6-the-filemaker-investigation","status":"publish","type":"post","link":"https:\/\/www.pwofc.com\/ofc\/2026\/07\/11\/phase-6-the-filemaker-investigation\/","title":{"rendered":"Phase 6 &#8211; the FileMaker Investigation"},"content":{"rendered":"<p>My PAWDOC work archive of some 17,000 Index entries and 31,000 associated files has an Index in the FileMaker database app. That was the app I chose in 1986 when I moved the index from a card-based system \u2013 and I\u2019ve stuck with FileMaker ever since despite it becoming a hugely more complex and capable beast. A few months ago, I discovered that FileMaker was offering a free 45-day trial of its latest AI-enabled version, so I thought I\u2019d try it out as part of this investigation into AI for personal archives. I figured that it would help me understand a bit more about AI, and that perhaps what FileMaker was doing would be generalisable to all specialist archive-supporting apps. This post summarises what I discovered and concluded.<\/p>\n<p>There is an extensive range of <a href=\"https:\/\/community.claris.com\/en\/s\/global-search\/%40uri#?cludoquery=ai&amp;cludoCategory=Claris%20Community&amp;cludoFilter=Training,Videos&amp;cludopage=1&amp;cludorefurl=https%3A%2F%2Fcommunity.claris.com%2Fen%2Fs%2F&amp;cludorefpt=Claris%20Community%20%7C%20Join%20discussions%20and%20access%20resources.&amp;cludoinputtype=standard\">videos and training materials<\/a> for FileMaker available online. <a href=\"https:\/\/www.pwofc.com\/ofc\/2026\/06\/14\/a-prompt-about-prompts\/\">I had already looked at a number of the videos<\/a> as part of my Phase 4 investigations, and now I went through two of the training modules:<\/p>\n<ul>\n<li><a href=\"https:\/\/community.claris.com\/en\/s\/learning-activity-detail-standard-lp?ltui__urlRecordId=a56Vy0000007i5xIAA&amp;ltui__urlRedirect=learning-activity-detail-standard-lp&amp;ltui__parentUrl=learning-plan-detail-standard&amp;ltui__tracId=a5GVy000001VgOzMAK\">AI Fundamentals I: Semantic search in Claris FileMaker<\/a><\/li>\n<li><a href=\"https:\/\/community.claris.com\/en\/s\/learning-activity-detail-standard-lp?ltui__urlRecordId=a56Vy000000N08zIAC&amp;ltui__urlRedirect=learning-activity-detail-standard-lp&amp;ltui__parentUrl=learning-plan-detail-standard&amp;ltui__tracId=a5GVy000001VgOzMAK\">AI Fundamentals II: Using LLMs in Claris FileMaker<\/a><\/li>\n<\/ul>\n<p>These describe the following three AI-related features:<\/p>\n<ul>\n<li>A Semantic Search facility<\/li>\n<li>Generate response from model<\/li>\n<li>Perform Find by Natural Language<\/li>\n<\/ul>\n<p>These features have been built into FileMaker using about a dozen AI-related FileMaker Scripts &#8211; sets of predefined instructions carried out by the application when a particular script is run by the user. \u00a0The <strong>Semantic Search<\/strong> works by first generating the vector value of the contents of a field (<a href=\"https:\/\/www.pwofc.com\/ofc\/2026\/04\/17\/phase-1-summary-results\/\">see Terminology in the post at this link<\/a>), putting that value into a file, and storing that file with the relevant record in the database. This is known as embedding and is enabled by one of the pre-built AI scripts called \u2018Insert Embedding in Found Set\u2019. Another script (Perform Semantic Find) turns a natural language search term input by the user into a specific vector value, and compares that vector value with the vector values for the records which have already been given vector values. The result of the comparison is a Cosine Similarity Coefficient ranging between +1 and -1 and typically calculated to 6 or 7 decimal points. FileMaker then lists the top X records which have the highest Cosine Similarity score (where X is a user-specified variable). A more sophisticated version of this is a hybrid version of this search which provides a screen in which the user defines BOTH a search term for the FileMaker Find facility to derive a subset, AND a natural language search request to be used on the contents of a specific field in all the records in the subset. In both the Ordinary Semantic Search and the Hybrid Semantic Search, the vector embedding files must already be in place in the database: records without a vector embedding file will not be included in this search. As with all AI usage, there may be limits or cost implications related to the number\/size of requests that are made to the model.<\/p>\n<p>The <strong>Generate Response from Model<\/strong> feature enables a general-purpose interface to be created with a local or remote AI model. This can be used to gather information from the database and for that information to be sent with an instruction Prompt to the model. The response from the AI is then displayed in FileMaker in a pre-specified way. For example, information in certain fields in a Product Database could be sent to the AI together with instructions to provide single line summaries of each product consisting of short description, category and price. The specific information to be gathered, and the instructions, could all be pre-specified in particular scripts, or could be defined by the user. Inevitably, the scripts to do this may be a little complicated; and appropriate interface screens (\u2018layouts\u2019 in FileMaker terms) will need to be built. As with the Semantic Search feature, there may be limits or cost implications related to the number\/size of requests that are made to the model.<\/p>\n<p>Unlike the previous two features, the <strong>Perform Find by Natural Language <\/strong>facility does not send any of the database contents to an AI model. Instead, it sends the prompt request created by the user along with further instructions plus the database schema. The AI then creates and returns the Find text which FileMaker includes in the relevant fields before running a Find operation.<\/p>\n<p>The training courses did a good job of explaining how all three of these features work. I then tried out the Semantic Find function on my PAWDOC database using the trial 2026 version of FileMaker. This involved creating <a href=\"https:\/\/www.pwofc.com\/ofc\/wp-content\/uploads\/2026\/07\/Filemaker-scripts-for-Phase-6-tests.docx\">additional scripts<\/a> around the primary scripts to specify things like the AI model being used, the fields for storing inputs and outputs, and the particular fields to be interrogated in the database. I initially followed the example of the training courses and obtained a trial API key to be able to access models in the cloud from the Cohere provider. However, I also followed the steps described in a very informative blog post called &#8216;<a href=\"https:\/\/www.geniewave.com\/blog\/using-lmstudio-as-a-local-llm-for-filemaker-2024-development\">Using LM Studio as a local LLM for FileMaker AI Development<\/a>&#8216; which showed how the same three features could be used with a local model residing on your own local machine. The examples in the blog post use the LM Studio app which I happened to already have on my laptop. So, I ended up carrying out a set of tests using both a remote and a local AI model. The tests were performed on the PAWDOC Title field to a) embed the contents, and b) perform semantic searches on a subset of the PAWDOC records. The models used were LM Studio\u2019s &#8220;text-embedding-nomic-embed-text-v2-moe&#8221; on my laptop, and Cohere\u2019s &#8220;embed-english-light-v3.0&#8221; in the cloud. \u00a0For comparison purposes, I also conducted some equivalent searches using the standard FileMaker Find function. My observations from <a href=\"https:\/\/www.pwofc.com\/ofc\/wp-content\/uploads\/2026\/07\/Assessment-of-test-results-exploring-FileMaker.docx\">these tests<\/a> include the following:<\/p>\n<p><em>Embedding<\/em><\/p>\n<ul>\n<li>The contents of the Title field in most of my index entries probably contain between about 4 and 30 words. I found that for both models, embedding one record took less than a second, and to embed 60 records took about 1 or 2 seconds. 2803 records took LM Studio 29 seconds to embed and Cohere probably took around 50 seconds (I couldn\u2019t get an accurate Cohere figure because of usage limits associated with my free Cohere API key).<\/li>\n<li>It took the LM Studio model 3 minutes and 4 seconds to create embedding files for all 17,381 PAWDOC records; and in the process it added about 25Mb to the 15Mb file.<\/li>\n<li>Embedding file names from both models were similar. For example, an LM Studio file was called emb_05D4B.fve, and a Cohere file for the same record was called emb_2E1C6.fve.<\/li>\n<\/ul>\n<p><em>Quality of Search Results<\/em><\/p>\n<ul>\n<li>Both LM Studio and Cohere had been set up to produce a list of the 10 most relevant hits. However, many of the results seemed to be of dubious relevance, for example, the results of a search on \u2018shipping industry\u2019 included the record \u2018CSC top ten clients worldwide\u2019; and the results of a search on \u2018work in the oil and gas industry\u2019 included the record \u2018CSC H&amp;S guide to working alone\u2019.<\/li>\n<li>In some cases, the semantic search results seem to fail almost completely to pick up relevant records. For example, while most of the semantic search terms did not produce any hits at all when used in a standard FileMaker search, one (UK management) did produce 25 hits. However, only 40% of those hits actually appeared in the semantic search results \u2013 three of them appeared in both the LM Studio and Cohere lists, and 2 appeared in one or the other.<\/li>\n<li>In a test on the \u2018NCC\u2019 subset of 237 records, the semantic search on &#8216;screen layout standards&#8217; produced 6 records common to both the LM Studio and Cohere results, of which the top three were the same but in a different order. A standard FileMaker search on &#8216;screen layout&#8217; produced no hits; and another on &#8216;standards\u2019 produced 4 hits of which 3 were in the results for both models. While some of the semantic search results were a little off-beam (\u2018standards for the use of overhead projectors&#8217;, for example), this was a good result from the semantic search which couldn&#8217;t be reproduced in the standard search using the same search words.<\/li>\n<li>In the semantic search on the term &#8216;Workshops in non-UK countries&#8217; neither LM Studio nor Cohere seemed to take on board the qualifying phrase &#8216;in non-UK countries\u2019: six of the LM Studio records included one or other of the following words in the Title field; Lancaster, London, UK; and 3 of the Cohere records included either Lancaster or UK.<\/li>\n<\/ul>\n<p><em>Comparison of the hits produced by LM Studio and Cohere<\/em><\/p>\n<ul>\n<li>Five tests were carried out on the subset \u2018CSC\u2019 which contains 2803 records. The semantic search test phrases were &#8216;bad experience&#8217;; &#8216;shipping industry&#8217;, &#8216;social events&#8217;, &#8216;work in the oil and gas industry&#8217; and &#8216;UK Management&#8217;. All produced 10 answers (as specified in the script) from each of the LM Studio and Cohere models. Across all these answers only 12% were to be found in both the LM Studio and Cohere lists \u2013 a surprisingly low degree of agreement between each set of results.<\/li>\n<li>In the semantic search test for \u2018travel by train\u2019 on the subset \u2018NCC\u2019 (containing 237 records), the same four records appeared in the top 5 spots in the results for both models. Overall, in the 10 records identified by each model, 7 were common to both model&#8217;s results &#8211; a rather better result than achieved in tests on the larger \u2018CSC\u2019 subset (described in the bullet point above) possibly due to this being a much smaller set of records.<\/li>\n<\/ul>\n<p><em>Cosine Similarity Scores<\/em><\/p>\n<ul>\n<li>The 5 semantic searches for &#8216;bad experience&#8217;; &#8216;shipping industry&#8217;, &#8216;social events&#8217;, &#8216;work in the oil and gas industry&#8217; and &#8216;UK Management&#8217; produced 10 answers (as specified in the script) for each of LM Studio and Cohere. On average there was only 12% commonality between the records identified by LM Studio and those identified by Cohere. Despite this significant difference in results, the Cosine Similarity Coefficient of the top ranked items in all 5 tests were quite close (LM Studio first vs Cohere second): +0.479378961 vs +0.477162529; +0.340306588 vs +0.34258184; +0.419345513 vs +0.377885542; +0.293494392 vs +0.278173457; +0.267438353 vs +0.28565865.<\/li>\n<li>Some of the results of the semantic searches for \u2018shipping Industry\u2019 and \u2018bad experience\u2019 were of dubious relevance. For example, \u2018shipping industry\u2019 produced \u2018CSC World Fall \u201985\u2026. OA business, interconnection\u2019, and \u2018bad experience\u2019 produced \u2018CSC User Experience Teleconferencing minutes\u2019. However, that is perhaps understandable given the low Cosine Similarity numbers for both models, and the need to provide 10 answers as specified in the script. The top hits in LM Studio and Cohere respectively, for each of the searches were +0.293494392 &amp; +0.278173457 for \u2018shipping industry\u2019, and +0.267438353 &amp; +0.28565865 for \u2018bad experience\u2019. If there are few or no items of high relevance then the records identified are, by definition, going to be of less relevance.<\/li>\n<li>For the semantic search for &#8216;Workshops in non-UK countries&#8217; there was a particularly strong similarity in the results for LM Studio and Cohere: the top two hits were exactly the same though with slightly different Cosine Similarity scores: +0.456294376 vs +0.513860943 and +0.44603088 vs +0.471864208.<\/li>\n<\/ul>\n<p><em>Comparison with Standard FileMaker Find Results<\/em><\/p>\n<ul>\n<li>In comparison with the semantic search on &#8216;Workshops in non-UK countries&#8217;, a standard FileMaker search on &#8216;Workshops&#8217; produced 10 hits, three of which appeared in the results for both models and two of which appeared only in the Cohere set. A second standard search using the singular &#8216;workshop&#8217; produced 210 hits. I would have thought the AI would give as much weight to the singular as to the plural in this case.<\/li>\n<li>To compare the results of the semantic search for \u2018work in the oil and gas industry\u2019, three separate standard FileMaker searches were conducted for the terms \u2018oil\u2019, \u2018gas\u2019, and \u2018industry. The \u2018oil\u2019 search produced 8 hits, none of which appeared in either semantic search results for LM Studio or Cohere. The \u2018gas\u2019 search produced 3 hits, one of which appeared in both the LM Studio and Cohere lists and one just in the LM Studio list; and the \u2018industry\u2019 search produced 12 hits of which one appeared in both the LM Studio and Cohere lists and one just in the Cohere list.<\/li>\n<li>A semantic search was conducted on the term \u2018travel by train\u2019. A standard FileMaker search for &#8216;train&#8217; produced 52 hits, but, surprisingly, only 1 of these (the top answer in each model) appeared in the 10 results from LM Studio or Cohere respectively.<\/li>\n<\/ul>\n<p>While the tests from which the above observations are drawn were by no means rigorous or definitive, they do provide enough insights to draw the following general conclusions:<\/p>\n<ol>\n<li>Different LLM models may well produce very different results so choosing the right model for the job you want to do is key. However, that\u2019s not the only consideration: as the Claris Fundamentals II course puts it, \u201cyou want to use the smallest, fastest and least expensive model possible to get the job done\u201d.<\/li>\n<li>FileMaker doesn\u2019t have its own AI capabilities \u2013 it engages with independent models through its Script programs which can get complicated and require developer knowledge. This is likely to be the case with most specialist applications.<\/li>\n<li>PAWDOC is a very simple database in a very sophisticated app with just a single user. Based on what I have learned throughout this AI journey I would choose to work directly with an AI model rather than try to build scripts\/macros\/programs to enable interaction with an AI model from within an application. Owners of larger and more complex databases with hundreds of users may not have that option.<\/li>\n<li>Standard FileMaker searches and the ability to search again within a found set are sufficient for my use of PAWDOC today.<\/li>\n<li>The Cosine Similarity Coefficient is a useful measure; and it\u2019s also helpful to be able to specify how many hits you get in a semantic search. However, it must always be remembered that semantic search results are never absolutely right \u2013 they just deliver the best matches along with a Cosine Similarity rating to indicate how close a match they are.<\/li>\n<\/ol>\n<p>Below is a breakdown of the time I spent on Phase 6 and across all phases.<\/p>\n<table>\n<tbody>\n<tr>\n<td width=\"130\"><em><u>Activity<\/u><\/em><\/td>\n<td width=\"331\"><em><u>No of Tasks or task breakdown<\/u><\/em><\/td>\n<td width=\"76\"><em><u>Elapsed time<\/u><\/em><\/td>\n<td width=\"76\"><em><u>Time spent<\/u><\/em><\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 1<\/td>\n<td width=\"331\">70 <u>(started 05Mar2026)<\/u><\/td>\n<td width=\"76\">43 days<\/td>\n<td width=\"76\">105 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 2<\/td>\n<td width=\"331\">8<\/td>\n<td width=\"76\">4 days<\/td>\n<td width=\"76\">11 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 3<\/td>\n<td width=\"331\">\u00b7 Create test files, test, analyse results<br \/>\n\u00b7 Research &amp; draft pwofc.com posts<\/td>\n<td width=\"76\">3 days<br \/>\n4 days<\/td>\n<td width=\"76\">15 hrs<br \/>\n12 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 4<\/td>\n<td width=\"331\">\u00b7 Create test files, test, analyse results<br \/>\n\u00b7 Research &amp; draft pwofc.com posts<\/td>\n<td width=\"76\">14 days<br \/>\n11 days<\/td>\n<td width=\"76\">80 hrs<br \/>\n31 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 5<\/td>\n<td width=\"331\">\u00b7 Create test files, test, analyse results<br \/>\n\u00b7 Research &amp; draft pwofc.com posts<\/td>\n<td width=\"76\">4 days<br \/>\n2 days<\/td>\n<td width=\"76\">13 hrs<br \/>\n8 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Phase 6<\/td>\n<td width=\"331\">\u00b7 Do training &amp; draft pwofc.com post<br \/>\n\u00b7 Create test files, test, analyse results<\/td>\n<td width=\"76\">7 days<br \/>\n6 days<\/td>\n<td width=\"76\">16 hrs<br \/>\n12 hrs<\/td>\n<\/tr>\n<tr>\n<td width=\"130\">Totals<\/td>\n<td width=\"331\"><\/td>\n<td width=\"76\">85 days<\/td>\n<td width=\"76\">\u00a0303 hrs<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n","protected":false},"excerpt":{"rendered":"<p>My PAWDOC work archive of some 17,000 Index entries and 31,000 associated files has an Index in the FileMaker database app. That was the app I chose in 1986 when I moved the index from a card-based system \u2013 and &hellip; <a href=\"https:\/\/www.pwofc.com\/ofc\/2026\/07\/11\/phase-6-the-filemaker-investigation\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[40],"tags":[],"class_list":["post-2923","post","type-post","status-publish","format-standard","hentry","category-ai-for-personal-archives"],"_links":{"self":[{"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/posts\/2923","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/comments?post=2923"}],"version-history":[{"count":7,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/posts\/2923\/revisions"}],"predecessor-version":[{"id":2935,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/posts\/2923\/revisions\/2935"}],"wp:attachment":[{"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/media?parent=2923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/categories?post=2923"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pwofc.com\/ofc\/wp-json\/wp\/v2\/tags?post=2923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}