dk 0.8.2: recall honours --top-k with a Context section, dk document, release hardening

.github/workflows/ci.yml

@@ -127,8 +127,12 @@ jobs:
         env:
           DAKERA_ROOT_API_KEY: test-integration-key
           DAKERA_AUTH_ENABLED: 'false'
+        # The v0.12.3+ images have no curl; `dakera healthcheck` probes the
+        # REST port instead. This check works on both, so moving the image to
+        # 0.12.3 once it is released is a one-line change. The v0.12.3 tests
+        # in tests/integration.rs skip themselves on an older server.
         options: >-
-          --health-cmd "curl -sf http://localhost:3000/health || exit 1"
+          --health-cmd "if command -v curl >/dev/null 2>&1; then curl -sf http://localhost:3000/health; else dakera healthcheck; fi"
           --health-interval 5s
           --health-timeout 3s
           --health-retries 10

CHANGELOG.md

@@ -5,6 +5,85 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+## [0.8.2] - 2026-10-22
+
+Support for Dakera server **v0.12.3**. Follows the engine's recall API correctness and security
+hardening work (Dakera-AI/dakera#917, #920, engine PR #922; merge this after it, as its merge tags
+v0.12.3). Compatible with v0.11.108 and v0.12.0 – v0.12.2 servers as before; `--neighborhood`,
+`--include-summaries`, the Context section's `context_of`, the unknown-field hint and `dk document` need v0.12.3.
+
+### Breaking
+
+- **`dk --format json|compact memory recall` prints one object,
+  `{"memories": [...], "context_memories": [...]}`, instead of an array of result rows** (approved
+  by the owner). Read the ranked results from `.memories` (for example
+  `dk -f json memory recall a q | jq '.memories[].id'`); the neighbourhood context is in
+  `.context_memories`. The old output could not be parsed as it was anyway: an `ℹ Found N memories`
+  line came before the JSON on stdout.
+
+### Fixed
+
+- **`dk memory recall` shows at most `--top-k` ranked results.** The server appended
+  session-neighbourhood context after the ranked results, so `--top-k 5` could print 7 rows. The
+  command now asks for `neighborhood_mode: "separate"` and prints the context in its own
+  **Context** section after the ranked table, each row saying which result it belongs to
+  (`#2 <id>`). A server that predates `neighborhood_mode` (v0.12.2 and older) appends its context
+  unmarked: the rows past `--top-k` are left out with a warning (under `--neighborhood inline` they
+  are shown as context).
+- **`dk memory recall` lists each result with the server's `rank_score`** (v0.12.3), the score
+  that follows the order of the results. `score` and `smart_score` can rise down the list, because
+  later ranking stages reorder the results after they are scored. Older servers send no
+  `rank_score`; their `smart_score` is listed as before.
+- **A `429` answer exits 6 (server busy), like a `503`, with the `Retry-After` hint.** Since v0.12.3
+  an overloaded server (a full inference or recall queue) answers `429` + `Retry-After` where it
+  answered `503`; `dk` exited 1 on it.
+- `dk --format json|compact memory recall` printed `ℹ Found N memories` before the JSON on
+  stdout; stdout is now one JSON document and notes go to stderr.
+- `dk keys get|delete|deactivate|rotate|usage` and `dk admin optimize|backup-delete` percent-encode
+  the id in the path; `dk keys *` errors carry the usual exit code and hint instead of the raw body.
+- `dk keys create --format json` prints the server's answer as JSON.
+- README: the config file example showed an `api_key` and `--key` that `dk` never read.
+
+### Added
+
+- **`dk document`** (Dakera v0.12.3, Dakera-AI/dakera#923): `import <file|->` turns a text,
+  Markdown, HTML, PDF or DOCX file into chunk memories of `--agent` (text formats through
+  `POST /v1/memory/documents`; PDF / DOCX, and text that is not UTF-8, uploaded as an attachment to
+  the agent's namespace and imported through `POST /v1/namespaces/{ns}/attachments/{ref}/document`),
+  with `--title`, `--document-id`, `--media-type` (else from the extension), `--tag`, `--session-id`,
+  `--type`, `--importance`, `--lang`, `--chunk-tokens`, `--overlap`, `--strategy structure|fixed`,
+  `--decompose`, `--wait` / `--timeout`; `job <id>`, `list` (`--limit`, `--cursor`), `show <id>`
+  (the chunks with pages and heading paths) and `delete <id>`, in table and JSON. On a server
+  before v0.12.3 the commands say it is needed. Shell completions include them.
+- `dk memory recall` cites document chunks: a `SOURCE` column (`Field Manual, p. 39-40 › Install`)
+  and `document` (`document_id`, `title`, `page_start` / `page_end`, `heading_path`,
+  `chunk_index`) in JSON, on ranked results and context.
+- `dk memory recall --neighborhood <separate|inline|off>` (default `separate`; `off` also sends
+  `neighborhood: false`, which older servers honour) and `--include-summaries[=false]` (sent only
+  when given).
+- A `MATCHED_TEXT` column when the server collapsed derived sentence sub-memories into a result
+  (`DAKERA_RECALL_COLLAPSE_DERIVED`).
+- `--format json` for `memory recall` carries `matched_text` / `matched_ids` on ranked results and
+  `context_of` / `context_of_rank` on context (see **Breaking** for the new shape).
+- A hint for the `422` (or `400`) "unknown field" refusal that the admin key, backup and
+  rotate-key routes answer from Dakera v0.12.3 on, naming the field.
+
+### Security
+
+- `~/.dakera/config.toml` is written `0600` (its directory `0700` when created), an existing file is
+  narrowed to `0600` on write, and a file readable by other users gets a warning on stderr.
+- `dk config` shows `DAKERA_API_KEY` masked (first 3 and last 4 characters) and the transport (TLS
+  or plain http); a password in the server URL is masked.
+- A warning on stderr when `DAKERA_API_KEY` is set and the server URL is plain `http://` to a host
+  other than the loopback interface.
+- Raw requests time out: 10 s to connect, 330 s between reads (longer than the 300 s
+  `encryption-rotate --wait-secs` may ask the server to wait).
+- Dependencies: rustls 0.23.46 (RUSTSEC-2026-0346, -0347, -0348, -0349), quinn-proto 0.11.19
+  (RUSTSEC-2026-0338 to -0342), chacha20 0.10.2 (0.10.1 was yanked), rustls-webpki 0.103.16
+  (RUSTSEC-2026-0382) and webpki-root-certs 1.0.10 (RUSTSEC-2026-0383).
+
 ## [0.8.1] - 2026-10-08
 
 Support for Dakera server **v0.12.2** (Dakera-AI/dakera#916). Compatible with v0.11.108,

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dakera-cli"
-version = "0.8.1"
+version = "0.8.2"
 edition = "2021"
 license = "MIT"
 description = "Command-line interface for Dakera AI Agent Memory Platform"

README.md

@@ -122,21 +122,26 @@ dk text search "user preferences" --namespace default
 
 ### Config file
 
-`dk init` creates `~/.dakera/config.toml`:
+`dk init` creates `~/.dakera/config.toml` (one profile per server):
 
 ```toml
-[server]
-url = "http://localhost:3000"
-api_key = "dk-mykey"
+active_profile = "default"
 
-[defaults]
-namespace = "default"
+[profiles.default]
+url = "http://localhost:3000"
+default_namespace = "default"
 ```
 
+The API key is never stored in it: `dk` reads it from `DAKERA_API_KEY` only. `dk` writes the file
+readable by its owner only (`0600`, in a `0700` directory) and warns on stderr when it finds the
+file readable by other users. `dk config` shows the key masked (`dk_…9876`), never in full, and
+whether the server URL uses TLS; a key sent over plain `http://` to a host other than the loopback
+interface gets a warning on stderr. TLS certificates are always verified (system trust store).
+
 ### Named profiles
 
 ```bash
-dk config profile add staging --url http://staging:3000 --key dk-staging-key
+dk config profile add staging --url https://staging.example.com:3000
 dk config profile use staging
 dk --profile staging namespace list
 ```
@@ -235,6 +240,31 @@ to stderr) and exits 5 if the job fails with a `400`, 6 on a `5xx`.
 
 ---
 
+### `dk document`
+
+Import a document as chunk memories of an agent (Dakera v0.12.3): each chunk keeps its heading path
+and page, so `dk memory recall` can cite it (a `SOURCE` column, `document` in JSON). Text, Markdown
+and HTML are sent as text; PDF and DOCX (and text that is not UTF-8) are uploaded as an attachment to
+the agent's namespace (`--namespace` to pick another) and imported from there, which needs
+`DAKERA_ATTACHMENTS` on the server. The format comes from the extension (`.txt`, `.md`, `.html`,
+`.pdf`, `.docx`; `-` reads stdin as text) unless `--media-type` says otherwise.
+
+```bash
+dk document import guide.md --agent docs-bot --wait
+dk document import manual.pdf --agent docs-bot --document-id manual-v3 --tag manual --wait
+dk document import notes.txt --agent docs-bot --chunk-tokens 300 --overlap 30 --strategy fixed
+cat page.html | dk document import - --agent docs-bot --media-type text/html --title "Release notes"
+dk document job job_17_3 --agent docs-bot      # Pending / Running (progress) / Completed / Failed
+dk document list --agent docs-bot --limit 20    # then --cursor <next_cursor>
+dk document show manual-v3 --agent docs-bot     # the document and its chunks (pages, headings)
+dk document delete manual-v3 --agent docs-bot   # every chunk memory of it
+```
+
+Re-importing the same `--document-id` replaces the document (unchanged chunks are kept as they are).
+Other flags: `--title`, `--session-id`, `--type`, `--importance`, `--lang`, `--decompose` (also
+derive sentence sub-memories per chunk), `--timeout` (with `--wait`, default 600 s). A server before
+v0.12.3 has no document routes: `dk` says so.
+
 ### `dk namespace`
 
 Manage namespaces.
@@ -255,8 +285,14 @@ Store, recall, search, and manage agent memories. This is the primary interface
 # Store a memory
 dk memory store my-agent "The user likes dark mode" --importance 0.8 --type semantic
 
-# Recall by semantic query
+# Recall by semantic query: at most --top-k ranked results, then a separate "Context"
+# section with the session neighbours of those results (which result each belongs to).
+# A MATCHED_TEXT column appears when the server collapsed sentence sub-memories into a result.
 dk memory recall my-agent "UI preferences" --top-k 10
+# --neighborhood separate (default) | inline | off; --include-summaries[=false]
+dk memory recall my-agent "UI preferences" --neighborhood off --include-summaries=false
+# JSON: {"memories": [...], "context_memories": [...]} (matched_text, context_of, context_of_rank)
+dk --format json memory recall my-agent "UI preferences" | jq '.memories[].id'
 
 # Full-text search within an agent's memories
 dk memory search my-agent "dark mode" --top-k 5
@@ -509,7 +545,23 @@ dk completion powershell
 
 ## Dakera v0.12 and compatibility
 
-`dk` 0.8 works against Dakera **v0.11.108, v0.12.0, v0.12.1 and v0.12.2** servers
+> **Breaking in dk 0.8.2:** `dk --format json|compact memory recall` prints one object,
+> `{"memories": [...], "context_memories": [...]}`, instead of an array of rows: read the ranked
+> results from `.memories`.
+
+The 0.8.2 additions need a **v0.12.3** server; older servers ignore them, as noted:
+
+| Command (0.8.2) | v0.12.2 and older | v0.12.3 |
+|---|---|---|
+| `dk memory recall` (`--neighborhood separate`, the default) | the server appends its context to the results unmarked: `dk` shows the first `--top-k` and warns how many it left out | at most `--top-k` ranked results, then the Context section with `context_of` |
+| `dk memory recall --include-summaries[=false]` | ignored | yes |
+| `MATCHED_TEXT` column | never shown | when the server collapses derived sentences (`DAKERA_RECALL_COLLAPSE_DERIVED`) |
+| the unknown-field hint (`keys create`, `admin backup-create`, `admin encryption-rotate`) | not needed: unknown fields are ignored | a misspelled field is a `422`, with a hint naming it |
+| `dk document ...` | no route: `dk` says it needs v0.12.3 | yes (`DAKERA_DOCUMENTS`, on by default; PDF / DOCX also need `DAKERA_ATTACHMENTS`) |
+| `SOURCE` column / `document` in `memory recall` | never shown | for results that are document chunks |
+| `dk admin` / `dk keys` commands | as before | as before; `POST /ops/shutdown` (no `dk` command) needs a `super_admin` key |
+
+`dk` 0.8.2 works against Dakera **v0.11.108 and v0.12.0 – v0.12.3** servers
 (it uses `dakera-client` 0.12). The new commands need the server release that has
 the route or field:
 
@@ -591,7 +643,7 @@ See the server's `docs/v0.12/UPGRADE.md` ("Going back to v0.11.108") for what
 | 3 | Not found |
 | 4 | Permission denied / authentication failure (`401`, `403`) |
 | 5 | Invalid input (`400`, `409`, `413`, `415`, `422`, or a bad argument) |
-| 6 | Server-side error (5xx, including `501` feature disabled and `503` busy/starting) |
+| 6 | Server-side error (5xx, including `501` feature disabled and `503` busy/starting), or `429` server overloaded (retry after its `Retry-After`) |
 
 Scripts can check `$?` after each command.
 

src/api.rs

@@ -85,6 +85,17 @@ impl ApiHttpError {
                 None => HINT_BUSY.to_string(),
             });
         }
+        // Since v0.12.3 the admin key, backup and rotate-key requests refuse a
+        // field they do not know instead of ignoring it.
+        if self.status == 422 || self.status == 400 {
+            if let Some(field) = unknown_field(&self.message) {
+                return Some(format!(
+                    "the server does not accept the field `{field}` on this route (admin key, \
+                     backup and encryption requests refuse unknown fields since v0.12.3); this \
+                     dk may be newer than the server, or the field is misspelled"
+                ));
+            }
+        }
         let pinned = self.details_contain("namespace: *");
         let super_admin = self.details_contain("required: super_admin");
         let hint = match (self.status, code) {
@@ -104,6 +115,13 @@ impl ApiHttpError {
     }
 }
 
+/// The field serde names in an "unknown field `x`, expected ..." refusal.
+pub fn unknown_field(message: &str) -> Option<&str> {
+    let start = message.find("unknown field `")? + "unknown field `".len();
+    let len = message[start..].find('`')?;
+    Some(&message[start..start + len])
+}
+
 const HINT_AUTH: &str = "authentication failed: set DAKERA_API_KEY to a valid, unexpired API key";
 const HINT_BUSY: &str = "the server is busy or starting; retry shortly";
 const HINT_PINNED: &str = "this API key is pinned to namespaces; since v0.12 such a key is \
@@ -467,6 +485,18 @@ mod tests {
         assert_eq!(e.message, "404 Not Found");
     }
 
+    #[test]
+    fn unknown_field_refusal_names_the_field() {
+        let error = "Failed to deserialize the JSON body into the target type: unknown field \
+                     `encrypted`, expected one of `name`, `backup_type`, `encrypt`";
+        let e = answer(422, "INVALID_REQUEST", error, None);
+        let hint = e.hint().unwrap();
+        assert!(hint.contains("`encrypted`"), "{hint}");
+        assert!(e.to_string().contains("hint:"));
+        let e = answer(400, "INVALID_REQUEST", "missing field `name`", None);
+        assert!(e.hint().is_none());
+    }
+
     #[test]
     fn segment_encodes_reserved_characters() {
         assert_eq!(segment("sha256:ab"), "sha256:ab");