From c4d6d9019653c7cd124a2f45ae6c3aad4ff916d8 Mon Sep 17 00:00:00 2001 From: lukemartinlogan Date: Tue, 25 Aug 2026 20:02:01 +0000 Subject: [PATCH] docs: split transparent LLM summarization into the summarizer ChiMod The summarizing LLM moved out of clio_cae_core into its own interposer ChiMod, clio_cae_summarizer (pool 401.0). It speaks the CTE core's task interface, overrides only PutBlob, and forwards every other core verb to next_pool_id -- so the label_endpoint / label_prompts / label_matches keys move off the CAE core's pool entry and onto its own. - deployment/configuration: drop the labeling keys from the CAE core table, add a Summarizer Module Parameters section with the full per-rule field table, compose ordering, and the synchronous-inference caveat. - chimod-chain: document the summarizer alongside cache/replication/indexer/ compressor -- it ships in CAE but interposes on the same vocabulary. Adds it to the well-known pool-id table and notes it is opt-in, not part of the shipped default chain. - context-assimilation-engine/overview: point at both of the above. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01RBv432JWeCysKtfAeVoob8 --- docs/deployment/configuration.md | 75 +++++++++-- .../context-assimilation-engine/overview.md | 2 + .../context-transfer-engine/chimod-chain.md | 118 +++++++++++++++++- 3 files changed, 179 insertions(+), 16 deletions(-) diff --git a/docs/deployment/configuration.md b/docs/deployment/configuration.md index 07d3ca3b..bb8f6848 100644 --- a/docs/deployment/configuration.md +++ b/docs/deployment/configuration.md @@ -473,15 +473,17 @@ CAE is the assimilation / discovery entrypoint: CEE calls `ParseOmni` here, and CAE forwards the data-path tasks it owns (`GetOrCreateTag`, `PutBlob`, `GetBlob`, `SemanticSearch`) on to CTE at `next_pool_id`. +Transparent LLM summarization used to be configured on this pool +(`label_endpoint` / `label_prompts` / `label_matches`). It now lives in the +`clio_cae_summarizer` ChiMod documented below — move those keys onto its pool +entry and point this pool's `next_pool_id` at it. + | Parameter | Required | Description | |-----------|----------|-------------| | `pool_name` | Yes | User-defined pool name. | | `pool_query` | Yes | Routing policy (`local`, `dynamic`, `broadcast`). | | `pool_id` | Yes | Unique pool ID. Canonical CAE pool ID is `"400.0"`. | | `next_pool_id` | No | CTE pool the data path is forwarded to (`"512.0"`). | -| `label_endpoint` | No | Ollama-compatible server URL for transparent LLM labeling. | -| `label_prompts` | No | Named prompt templates. | -| `label_matches` | No | Rules (`tag_re`, `blob_re`, `model`, `prompt`, `context_length`) selecting which blobs get labeled. | ```yaml - mod_name: clio_cae_core @@ -491,17 +493,51 @@ and CAE forwards the data-path tasks it owns (`GetOrCreateTag`, `PutBlob`, next_pool_id: "512.0" ``` -### Transparent LLM labeling (optional) +--- + +## Summarizer Module Parameters (`clio_cae_summarizer`) -When configured, `PutBlob` calls `model` on `label_endpoint` for every blob -whose tag and name match a rule, and stores the response as -`{blob_name}_label` in the same tag. Leave it out for a pure-passthrough CAE. +Transparent LLM summarization. This used to be configured on `clio_cae_core` +itself; it is now its own ChiMod, an **interposer** on the CTE core's task +interface — it forwards every core verb to `next_pool_id`, and additionally +prompts a model on each matching `PutBlob`, storing the response as +`{blob_name}_label` in the same tag. Leave the pool out entirely for a +pure-passthrough deployment. + +| Parameter | Required | Description | +|-----------|----------|-------------| +| `pool_name` | Yes | User-defined pool name. | +| `pool_query` | Yes | Routing policy (`local`, `dynamic`, `broadcast`). | +| `pool_id` | Yes | Unique pool ID. Canonical summarizer pool ID is `"401.0"`. | +| `next_pool_id` | No | Pool the chain is forwarded to (usually the CTE core, `"512.0"`). | +| `label_endpoint` | No | Ollama-compatible server URL. Without it no rule can fire. | +| `label_prompts` | No | Named prompt templates, referenced by a rule's `prompt`. | +| `label_matches` | No | Rules selecting which blobs get summarized. Empty (the default) makes the pool a pure forwarder. | + +Each `label_matches` entry takes: + +| Field | Default | Description | +|-------|---------|-------------| +| `tag_re` | — | Regex matched (`regex_search`) against the blob's tag name. | +| `blob_re` | — | Regex matched (`regex_search`) against the blob name. | +| `model` | — | Model name passed to the inference server. | +| `prompt` | — | Key into `label_prompts`. | +| `context_length` | `4096` | Ollama `num_ctx`. Also drives chunking: payloads larger than the per-request budget are split, prompted per chunk, and the responses concatenated. `0` disables chunking and takes Ollama's default (~2048), which silently truncates. | +| `num_predict` | `0` | Cap on response tokens. `0` = no cap. With chunking the final summary is roughly `num_predict` x (#chunks). | + +Compose the summarizer **after** the pool it forwards to, and point whatever +sits above it (typically the CAE core's `next_pool_id`) at `401.0`: ```yaml -- mod_name: clio_cae_core - pool_name: cae_main +- mod_name: clio_cte_core # composed first - the summarizer forwards here + pool_name: cte_main pool_query: local - pool_id: "400.0" + pool_id: "512.0" + +- mod_name: clio_cae_summarizer + pool_name: clio_cae_summarizer + pool_query: local + pool_id: "401.0" next_pool_id: "512.0" label_endpoint: "http://127.0.0.1:11434" label_prompts: @@ -512,8 +548,27 @@ whose tag and name match a rule, and stores the response as model: "gemma3:1b" prompt: "summarize" context_length: 4096 + +- mod_name: clio_cae_core + pool_name: cae_main + pool_query: local + pool_id: "400.0" + next_pool_id: "401.0" # data path runs through the summarizer ``` +:::warning Inference is synchronous on the handling worker +The model call runs inline in the `PutBlob` handler, so a rule that matches a +hot write path serializes that path behind the model. Scope `tag_re` and +`blob_re` tightly. Summarization failures are logged and swallowed — they +never change a `PutBlob` return code, and the original blob is always stored. +::: + +:::note Where the summary is written +The summary blob is stored through `next_pool_id`, i.e. *below* the +summarizer, so it never re-enters the handler. A rule with `blob_re: ".*"` +does not loop. +::: + :::danger Never put CAE in front of CTE at pool 512.0 CAE only mirrors the four data-path method ids above. The rest of its method ids **collide** with CTE's (CAE `kParseOmni` == CTE `kRegisterTarget` == 10), diff --git a/docs/sdk/context-assimilation-engine/overview.md b/docs/sdk/context-assimilation-engine/overview.md index 890dbd19..93219df5 100644 --- a/docs/sdk/context-assimilation-engine/overview.md +++ b/docs/sdk/context-assimilation-engine/overview.md @@ -8,6 +8,8 @@ sidebar_position: 1 The Context Assimilation Engine (CAE) is a CLIO Runtime module (`clio_cae::core`) that ingests external data sources into the CLIO Runtime. It reads data from files, HDF5 datasets, or remote Globus endpoints and stores them as blobs in the Context Transfer Engine (CTE). The CAE is registered as a Module container with pool ID `400.0`. +CAE ships a second, optional ChiMod: the **summarizer** (`clio_cae_summarizer`, pool `401.0`). It interposes on the CTE core's task interface and attaches an LLM-generated summary to each matching blob on the way through. See [the interposition chain](../context-transfer-engine/chimod-chain#summarizer-chimod-clio_cae_summarizer) for its behavior and [deployment configuration](../../deployment/configuration) for its keys. + ## Architecture ``` diff --git a/docs/sdk/context-transfer-engine/chimod-chain.md b/docs/sdk/context-transfer-engine/chimod-chain.md index 0b56c866..2dcb40dc 100644 --- a/docs/sdk/context-transfer-engine/chimod-chain.md +++ b/docs/sdk/context-transfer-engine/chimod-chain.md @@ -1,7 +1,7 @@ --- sidebar_position: 2 -title: Cache / Replication / Indexing ChiMods -description: The CTE interposition chain — node-local caching, persistent replication, and the semantic-search index, stacked over the CTE core. +title: Cache / Replication / Indexing / Summarizer ChiMods +description: The CTE interposition chain — node-local caching, persistent replication, the semantic-search index, and LLM summarization, stacked over the CTE core. --- # The CTE Interposition Chain @@ -47,12 +47,17 @@ The `clio_cte_filesystem` ChiMod (pool `560.0`, driven by the FUSE and POSIX adapters) sits above the whole thing and points its own `next_pool_id` at the chain top. +An interposer does not have to ship in the CTE package — it only has to speak +the CTE core's vocabulary. The **summarizer** (`clio_cae_summarizer`, pool +`401.0`) lives in the Context Assimilation Engine and slots into the same +chain; see [its section below](#summarizer-chimod-clio_cae_summarizer). + :::info Separation of concerns Each layer owns exactly one axis: **replication** = reliability, -**cache** = locality, **compressor** = encoding, **indexer** = search. They -compose because they all speak the same task vocabulary, and each is -independently removable — delete its `compose` entry and re-point the entry -above it. +**cache** = locality, **compressor** = encoding, **indexer** = search, +**summarizer** = enrichment. They compose because they all speak the same +task vocabulary, and each is independently removable — delete its `compose` +entry and re-point the entry above it. ::: --- @@ -97,6 +102,7 @@ would make an interposer forward to itself. | Compressor | `clio_cte_compressor` | `562.0` | — | | Cache | `clio_cte_cache` | `563.0` | `cache::kCachePoolId` | | Indexer | `clio_cte_indexer` | `564.0` | `indexer::kIndexerPoolId` | +| Summarizer | `clio_cae_summarizer` | `401.0` | `summarizer::kSummarizerPoolId` | | CTE core | `clio_cte_core` | `512.0` | `core::kCtePoolId` | Each module's own verbs (`ReplicateBlob`, `FlushTag`, `ReindexScan`, …) are @@ -427,6 +433,97 @@ To enable it, uncomment its `compose` entry **and** re-point the indexer's --- +## Summarizer ChiMod (`clio_cae_summarizer`) + +**Enrichment.** Ships in the Context Assimilation Engine, not CTE, but it is +an interposer like the rest: it speaks the CTE core's task interface and +slots anywhere in the chain. It overrides exactly **one** verb — `PutBlob` — +and forwards everything else untouched. + +This logic used to live inside `clio_cae_core`'s `PutBlob` handler. It is now +its own pool so the assimilation entrypoint and the LLM enrichment can be +composed, scaled, and disabled independently. + +### Behavior + +On each `PutBlob` the module: + +1. **Forwards down `next_pool_id` first**, so the user's write completes and + acks with the same return code whatever the model does. +2. Resolves the blob's **tag name** (via `GetTagName` on the chain below; + `PutBlobTask` carries only the id) and matches it plus the blob name + against the configured rules, in order. +3. On a match, prompts `model` on `label_endpoint` with the rule's prompt + template followed by the blob payload, chunking when the payload exceeds + the context budget and concatenating the per-chunk responses. +4. Stores the result as **`{blob_name}_label`** in the same tag. + +Everything after step 1 is best-effort: a bad regex, an unknown prompt name, +an unreachable endpoint, or an empty response is logged and swallowed. **A +summarization failure never changes a `PutBlob` return code**, and the +original blob is always stored. + +Replica-addressed writes (`Context::replica_ != 0`) are skipped — the primary +write already flowed through here. + +### Configuration + +| Key | Default | Description | +|-----|---------|-------------| +| `next_pool_id` | *(none)* | Pool below. Null falls back to the CTE core. | +| `label_endpoint` | `""` | Ollama-compatible server base URL. The handler POSTs to `{label_endpoint}/api/generate`. | +| `label_prompts` | *(empty)* | Named prompt templates, keyed by name. | +| `label_matches` | *(empty)* | Ordered rules. Empty makes the pool a pure forwarder. | + +Each `label_matches` entry: + +| Field | Default | Description | +|-------|---------|-------------| +| `tag_re` | — | Regex matched with `regex_search` against the tag name. | +| `blob_re` | — | Regex matched with `regex_search` against the blob name. | +| `model` | — | Model name sent to the inference server. | +| `prompt` | — | Key into `label_prompts`. | +| `context_length` | `4096` | Ollama `num_ctx`. Also drives chunking. `0` disables chunking and accepts Ollama's ~2048 default, which silently truncates. | +| `num_predict` | `0` | Cap on response tokens; `0` = uncapped. With chunking the final summary is roughly `num_predict` x (#chunks). | + +```yaml +- mod_name: clio_cae_summarizer + pool_name: clio_cae_summarizer + pool_query: local + pool_id: "401.0" + next_pool_id: "512.0" + label_endpoint: "http://127.0.0.1:11434" + label_prompts: + summarize: "Summarize the following text in one short sentence." + label_matches: + - tag_re: ".*\\.txt$" + blob_re: ".*" + model: "gemma3:1b" + prompt: "summarize" + context_length: 4096 +``` + +### No summarize-the-summary loop + +The `{blob_name}_label` blob is written through `next_pool_id` — *below* this +container — so it never re-enters the handler. A rule with `blob_re: ".*"` is +safe. + +:::warning Inference is synchronous on the handling worker +The model call blocks the worker that owns the task (libcurl, easy interface). +A rule matching a hot write path serializes that path behind the model, so +scope `tag_re` / `blob_re` tightly. This is the one interposer whose overhead +is measured in seconds rather than microseconds. +::: + +:::info Build flag +Summarization needs libcurl and nlohmann/json. Without them the module still +builds and forwards — the inference client compiles to a stub that always +fails, so every rule is a no-op. +::: + +--- + ## Putting it together The full standard chain as shipped in the default `~/.clio/clio.yaml`. Note @@ -485,6 +582,15 @@ What a put through the chain top now does: And a read: **cache** serves the raw local copy — including over the zero-IPC SHM fast path — or falls through to the owner and re-populates. +:::note The summarizer is not in the default chain +It is opt-in and commented out in the shipped config, because every rule it +matches costs an LLM round-trip on the write path. It normally sits on the +**assimilation** path rather than the filesystem one: point +`clio_cae_core`'s `next_pool_id` at `401.0` and the summarizer's at the CTE +chain. Nothing stops you putting it in the filesystem chain instead — it +speaks the same vocabulary — but then every FUSE write pays for it. +::: + ### Trimming the chain Every layer is optional. Remove the entry and re-point the one above it: