AI Accelerator - Pipelines 4.1.1 release notes

Knowledge bases with volume source need to assign an incrementing ID to each listing done on the source. This is necessary to detect when a listing is complete and recognize which previously seen objects were not encountered any more. In this release the method for tracking run IDs is changed to a table column since the Postgres sequence used before did not work on Postgres Distributed (PGD) clusters.

Volumes for AIDB can be created in arbitrary schemas in Postgres either by exlicitly passing a schema to the create function, or by setting a "current schema" via the search path. When using volumes in AIDB pipelines, the volume reference needs to be stored. AIDB will now store and explicitly use a schema when later executing a pipeline. Previously, the search path was used to find volumes.

When background workers are enabled for AIDB, a root-worker will start a database dispatcher for each database within the Postgres instance. This database dispatcher checks if AIDB is installed in the DB and if any pipelines need to be run in the background. The database dispatcher is now no longer a permanent process by default. When a database does not use AIDB, the database dispatcher exits and will be re-started to check for AIDB every 2 minutes. This avoids holding a permanent connection to the DB that would block e.g. "drop table" commands. If AIDB is installed in the DB, then the database dispatcher worker will keep running. Users can drop the extension in order to release the connection.