Technical considerations
How Kairos works: a technical summary by Claude AI
- The principle: one master, one derived display: Kairos is not a conventional content-managed website. Nothing is written or edited in it. Every document exists
first as a hand-authored page on the Laetus-in-Praesens site, and those pages remain the authoritative master
copy. Kairos is a derived layer: a searchable, cross-referenced, statistically profiled display
of what the master pages contain, regenerated from them whenever they change. That single decision governs everything else. The display layer can be rebuilt, moved or replaced without risk to
the content, because the content never lives only there.
- Three stages
Source pages (PHP/HTML, ~2,500 documents, 1960s–2020s)
| read in "scraping mode"
v
OpenInsight processing pipeline (16 procedures, AA01–AA16)
| writes plain-text files
v
15 semicolon-delimited CSV files, one per record type, one shared 24-field layout
| imported by a custom Drupal module
v
Drupal 11 display (about 50,000 records, Views, search, statistics)
- Source: The documents are PHP pages arranged in decade folders (docs60s … docs20s and others), each carrying
standard metadata (title, author, date, keywords, description) and a body divided by section headings. Images,
animations and 3D (X3D) models sit in folders beside the pages and are never copied into the database: Kairos
records only point to them.
- Processing (OpenInsight): The processing is done in OpenInsight, a MultiValue database with its own BASIC+ language, in which the author has
maintained the indexing of his work for over twenty years. Sixteen procedures run in sequence under a single driver:
| Stage |
What it does |
| AA01–AA02 |
Read every source page; split it into its sections; extract titles, years and metadata. |
| AA03 |
Resolve every link in every page: links between documents become stable Kairos addresses, and are counted as citations in both directions. |
| AA04–AA05 |
Tables of contents; the document records themselves (introductions and sections). |
| AA06–AA10 |
Bibliographic references, authors cited, cross-reference records, duplicate detection. |
| AA11–AA12, AA16 |
Visual records: images, animations, galleries, and tables (with and without images). |
| AA13–AA14 |
Indexing; clustering of documents under 54 topics by keyword. |
| AA15 |
The statistical summary (documents, citations and topics by decade), generated as a ready-made page. |
The output is fifteen plain-text CSV files. Every record, whatever its type, has the same 24 fields (type, address,
titles, year, citation counts, body and so on); the meaning of some fields varies by type, and is documented in the
procedures and in the column labels of the Kairos listings. Each record's address (its "alias") is built from the
source file's name plus a suffix for its type and section, so addresses are stable from one regeneration to the next.
A good deal of the pipeline is defensive. The source pages span six decades of authoring habits and tools, so the
procedures repair character-encoding damage, escape quotation marks, and check that every record has exactly the
right number of fields before it is written. They also keep censuses of anomalies (undated pages, conflicting
titles, broken links), which become worklists for correction at source, never in the display.
- Display (Drupal 11): The CSV files are uploaded to the web server and imported into Drupal 11 by a small custom module
(migrate_helios), built on Drupal's standard migration framework. All records go into a single content type
with 23 fields; each import matches records by their address, so a re-import updates what exists, adds what is new,
and lists (optionally removes) what has disappeared from the source. A command-line driver imports all fifteen files
in one unattended run, checking their encoding first and logging the outcome per type.
Drupal then provides what a set of static pages cannot: the filterable listings (Documents, Authors cited,
References, Visuals), full-text search, and stable addresses for every section of every document. A minimal custom
theme reproduces the plain appearance of the earlier site and adapts the layout to phone screens. The explanatory
Directions pages are deliberately kept as static HTML files outside Drupal
Scale (October 2026)
| Documents (introductions) |
about 2,500 |
| Document sections |
about 13,600 |
| Citation links between documents |
about 39,800 |
| External works cited / authors cited |
about 30,500 / 12,900 |
| Images / animations / tables / galleries |
about 6,500 / 4,800 / 4,500 / 640 |
| Topics |
54, in 14 groups |
| Records in the display layer |
about 50,000 |
Future-proofing: what has been designed to last
- The content never depends on the display. If Drupal were abandoned tomorrow, nothing would be lost:
the source pages are complete in themselves, and the whole display can be regenerated from them.
- Plain text at the joint. The only thing passing between the processing and the display is a set
of CSV text files with a documented layout. Any future system able to read a spreadsheet-like file can take over
either side of that joint.
- Stable addresses. Because record addresses derive from source file names rather than database
numbers, links into Kairos survive re-imports, server moves, and the change of domain planned for a Swiss mirror
(laetusinpraesens.ch).
- Media stay where they are. Images and 3D models are referenced, not ingested, so they never need
migrating with the database.
- A gentler upgrade path ahead. The move from Drupal 7 to Drupal 11 was a rebuild, because Drupal
was rewritten in between. Upgrades from Drupal 11 onward are designed to be incremental, and the site uses only
standard components plus one small module, which keeps them manageable.
- The processing is readable. The OpenInsight procedures have been documented in their own
headers and saved as plain text; the CSV contract is written down. Should OpenInsight itself become unavailable, its
licence being a real dependency, the pipeline's logic is explicit enough to be re-implemented in a common language
such as Python.
Acknowledged weaknesses
- The pipeline depends on a proprietary database and on one person's long familiarity with it.
- Topic clustering is by keyword and inherits the unevenness of the keywords given to each document over sixty years.
- Some lower-priority record types are less mature than the document records, and are being refined.
- Hosting is a modest shared server: ample for reading, but heavy administrative operations need care.
Conversion by program: As described in more detail in the
Content overview, the documents on this Kairos site derive from a set of articles held in PHP format on the related
Laetus-in-Praesens website. These typically longer "
articles" have been split into the shorter "
documents" imported into the free and open-source
content management system Drupal.
The approach taken to the conversion of PHP articles into Drupal records is one which reflects the biases of a programmer exploring solutions such as to avoid manual copying of portions of those articles into the content management system (CMS). The challenge was whether the conversion could be extensively ensured by program, notably in order to extract other information and build that into other records to be imported into the CMS. The intention was therefore to build many of the CMS records prior to importing rather than endeavour to generate additional content types within the CMS facility. The approach was framed in this way because of extensive expertise in manipulation of text with a DOS-based application -- and little expertise in the PHP-related programming required for the CMS.
The approach was successively enabled using a suite of programs developed with the Advanced Reverlation (AREV) application through a DOS box in Windows XP. The resulting pre-formatted flatfiles (CSVs) were then imported into Drupal 6 and later Drupal 7. This required some ingenuity to circumvent a 64k constraint on record length within AREV for documents of greater size. Cessation of support for Windows XP, encouraged a shift to Open Insight, a later variant of AREV. The upgrade to the Kairos site in December 2018 was enabled using Open Insight 9.4. Considerations is being given to upgrading that facility to Open Insight 10.04.
Article formatting: Advantage has been taken of the fact that the PHP articles were in a format which had remained standard and relatively stable over decades, both from the earliest (in the 1960s), and since first placed onto a website (in HTML format) in the early 1990s. Articles from earlier periods were adapted to that format as they were digitized. The key factor enabling conversion was the presence in those files of HTML title delimiters defining the sub-titles of what could then be split out as Drupal records. The conversion challenge was defined such as to avoid any additional mark-up -- otherwise required to facilitate the process, using programming "tricks" to circumvent anomalies. This could well be described as a less than efficient process (if not stupid!), but it did offer some nice programming challenges for someone anxious to avoid manual manipulation (at all costs!).
Retaining relationship to original version: The conversion challenge was also seen as a means of preserving a degree of complementarity between the PHP articles in the Laetus facility and the variant on the Kairos facility. The intention was not to switch to writing articles within the Drupal CMS, or updating them there, since it has been far more convenient to continue the process of writing/editing of the PHP variants within the Laetus facility using Dreamweaver. This was one reason for using record (node) identifiers within the CMS based on the original PHP file/folder name -- rather than switch to a numeric node identifier as is most commonly the case for a CMS.
Note that it is the Laetus version which is considered to be the master copy. Only minor editing is done exceptionally on documents in the Drupal context -- most notably in the event of conversion issues which have not been resolved by program. Errors emerging in the Drupal may well be used as a means of detecting and correcting errors in the Laetus version or in the conversion scripts.
Constructing records for import: Building the various CMS record types prior to import, rather than depending on (absent) Drupal skills to manipulate the basic imported documents, has meant that new record types can be created and populated as required in order to enhance the CMS facility. Of particular interest are those relating to the pattern of links.
Drupal node import: Of interest in the strategy adopted is the constraint imposed by the state of development of the Drupal "node import facility". Basically the options available for updating any node of a particular content type by the import process are to delete such nodes individually, or in a batch process (VBO) -- and then to import the corrected set of nodes into the IDs thereby made "free". The provisions for "overwriting" a node, without prior deletion, have been progressively developed stage within the Drupal community -- but primarily for numeric nodes. In practice this means that it is easy to batch delete all the bibliographic reference records, or the associated author records, and then to re-import a set generated from PHP articles after corrections (in the light of errors that became apparent from Drupal sorts). The advantage of the alphanumeric node naming system is that the links to the other documents are not affected by this process since the pointers from those documents remain valid.
Upgrading and adaptation:
- Drupal content management system:
- Drupal 6: The Kairos facility functioned successfully within the Drupal 6 content management system for 7 years.
- Drupal 7: A much delayed necessary upgrade to Drupal 7 was implemented in January 2017. The process of doing so left a number of relatively minor issues unresolved. These include breaks in some documents due to formatting issues during the import into Drupal 7. In particular totalling of numeric fields in search results is no longer working. It was considered more appropriate to avoid delay in the upgrade to Drupal 7 and to repair such issues at a later date (with others which may become apparent). With respect to errors related to formatting, users are always able to revert to the original document on the Laetus-in-Praesens site. A further major upgrade has been made implemented in December 2018.
- Drupal 8/9/10?: Consideration has been given to switching to Drupal 8, and experimental upgrades have been undertaken. The issue faced by many is whether the switch from the proven viabilty of Drupal 7 is appropriate, and when -- especially in the light of the anticipated subsequent upgrade to Drupal 9. The improvements within Drupal 7 in 2018, have been seen as usefully enabling any such later transition.
- Drupal 11: Tjhe direct upgrade from Drupal 7 to Drupal 11 was made in 2026.
- Constraints: (revised October 2026).
- HTTP to HTTPS: Although seemingly of marginal significance, the switch from HTTP to the security certification of HTTPS contributed to other transitional difficulties, especially in the related Laetus site with its add-on domains. Each domain and sub-domain requires its own certificate, renewed automatically every few months, and a renewal fails silently whenever a domain lapses or no longer points to the server: certificates for two of the add-on domains were found to have expired in this way. A less visible consequence is that a secure page will not load insecure ("http") content: images, animations or embedded pages addressed by older links are simply blocked by the browser. As part of the Drupal 11 transition, the processing of Kairos records therefore converts every link to a full secure address of the Laetus site.
- HTML 4 to HTML5: Web pages of any age tend to be characterized by use of features which are increasingly deprecated in the light of the more stringent requirements of HTML5. Multiple errors and warnings are signalled by any code validation application. It is very fortunate that browsers remain tolerant of many deprecated features (alignment and colour attributes in tables, anchors marked by "name" rather than "id", and the like). The issue is how to navigate the transition to current standards, for which features, and when. Two pressures have now made this concrete. Firstly, content management systems are less tolerant than browsers: the editor of Drupal 11 rewrites or strips some older markup when a page is saved, so that pages are better prepared in a conventional HTML editor and inserted whole. Secondly, reading on smartphones requires an explicit declaration of the page's viewport, absent from pages written before smartphones existed; this has been added to the Kairos guidance pages. The approach adopted is to update a feature when a page is otherwise being revised, rather than to undertake a wholesale conversion. This remains a continuing concern.
- PHP 5 to PHP 8: This server-side scripting language has been in progressive development since the 1990s (there was no released PHP 6; versions passed from 5 to 7 and then 8). The difficulty in the development of websites has arisen from failure to update code using features deprecated in successive versions of PHP, and a failure to exploit their enhanced efficiency. This notably engendered errors on the Laetus site with respect to the operation of menus. The Drupal 7 version of Kairos remained dependent on PHP 7.4, whose support ended in 2022. The transition to Drupal 11 required PHP 8.3, which is now used for all the domains on the server, including the server-side includes of the Laetus pages. Security support for PHP 8.3 is expected to continue until the end of 2027, so that a further upgrade will be required within a few years, though Drupal 11 is designed to make such steps incremental rather than a reconstruction.
- Windows: from 7 to 10 and beyond: Many users of the Windows platform are aware of the issues engendered by the unpredictable consequences of "invasive" upgrades. These have been noteworthy for the manner in which they rendered inoperable various animations and visualizations operating on the Laetus site (especially early demonstrations of virtual reality possibilities, which depended on browser plug-ins no longer supported). Those 3D models are now displayed by a script within the page itself (X3DOM), requiring no plug-in; models originally published without such a page are progressively being given one. In the case of the Laetus/Kairos processes, the constraint required use of an outdated version of Windows long after it was no longer maintained, in order to benefit from the DOS box facility required to operate the DOS-based AREV facility (as noted above). This constraint has been circumvented by use of the OpenInsight variant. Support for Windows 10 itself ended in October 2025, and the move of OpenInsight to a new machine was briefly blocked by a licensing difficulty. The unpredictable consequence of upgrades, of the operating system and of proprietary software licences, is necessarily a continuing concern.
- Migration/Import from Laetus to Drupal: The ability to update the Kairos site was considerably delayed because of constraints on the Drupal migration facility resulting from the continuing use of alphanumeric nodes. The justification for this use was noted above. A further factor is that search engines have long provided access to Kairos documents using their URLs, making it inappropriate to switch to any other modality. The Drupal implementation for Kairos is considered to have broken some fundamental design rules, being ambiguous by design, most notably with respect to its use of record types: all records share a single structure of 23 fields whose meaning varies with the type of record, and Drupal 11 refused the field name originally intended to distinguish those types. This ruled out more conventional solutions to the problems encountered. These constraints were resolved in the Drupal 11 version by a custom import module which matches every record on its alphanumeric address, so that existing addresses are preserved and search engine links continue to work. Re-imports now update existing records and add new ones in a single unattended run, with records no longer present in the source listed for removal rather than silently retained. The ambiguity of the record types is handled in the listings themselves, each of which labels the shared fields according to their meaning for that type.
- Server-related issues: A successful switch in server also introduced unforeseen delays because of the manner of attribution of security certificates for HTTPS. The switch of Kairos from Drupal 7 to Drupal 11 (October 2026) was made by redirecting the existing Kairos address to the new installation on the same server, the earlier version being kept dormant as a fallback; certificates, PHP versions and settings proved to be attached to domain names rather than to folders, a source of some confusion. A shared server also imposes limits on memory and processing time: some administrative operations on a database of some 50,000 records are cut off before completion, and the editing interface of Drupal may report a failure for a change that has in fact been saved. A mirror of Kairos on a Swiss server (laetusinpraesens.ch) is envisaged, with the expectation that such server-related issues will need to be addressed afresh there.
Character encoding issues: Working with articles published as early as the 1960s, making extensive use of accented characters, has required a degree of flexibility in adapting the conversion to handle characters which pre-date the currently favoured UTF-8 standard. Some of the articles are in French and other languages. Many cite authors and articles requiring such characters. Although the conversion process enabled some of these anomalies to be "corrected", These issues have not been completely resolved and are reflected in the contrating ways in which Drupal itslef handles such encoding in different contexts.
Augmenting access possibilities: A major motivation for exploring a CMS variant was to segment the longer PHP files (some over 150k) into more "readable" forms as CMS records. This was seen as particularly valuable in that the sub-titles attributed to the HTML title-delimited segments were interesting to extract in order to benefit from the Drupal Views facility, in addition to enabling more speciic access via search engines.
Benefitting from extensive hyperlinking: A significant characteristic of the PHP articles is the degree of hyperlinking between them. The conversion was designed to derive further information from this pattern of links, notably by generating "checklists" of citations "from" and "to" the CMS records. Unfortunately, as noted above, the links have been enabled to the "main" document introducing a set, and not to the individual documents of the set. This could be improved in the future.