Answer known questions locally; gate egress; approve every key use

README.md

@@ -323,6 +323,21 @@ agent.
   operator-only.
 - **Redaction** — keys, mnemonics, JWTs, bearer tokens and URL credentials are
   scrubbed from everything entering and leaving the agent (tx hashes stay visible).
+- **Egress gate** — the last check before any request reaches a model provider:
+  - Anything still carrying key material is withheld whole, never partly sent. This
+    covers PEM/OpenSSH keys, consensus keys, mnemonics, keystores, keyring entries and
+    API tokens.
+  - A message from you containing a mnemonic isn't sent at all.
+  - On validators (`agent.egress: strict`, the default) the model never sees raw shell,
+    file or log output. It gets a local summary instead: identifiers masked, repeats
+    folded.
+- **Key access** — every signature is approved on its own. The approval shows the
+  account, which key and where it lives, the decoded messages, the fee and why. It is
+  never auto-approved, in any mode. Remote (Telegram) approvals of a key use expire in
+  5 minutes.
+- **Local answers** — known questions ("is anyone jailed?", "how many peers?") and
+  known incidents with a playbook are answered by cometcli itself with no model call.
+  `/llm` asks the model anyway; `/route` shows why a prompt went where it did.
 - **Audit** — every tool call, shell command, prompt and transaction lands in
   `~/.cometcli/audit/YYYY-MM-DD.jsonl` (`cometcli audit` to tail it).
 - **Bounded execution** — every tool runs under a deadline; long requests pause
@@ -338,6 +353,7 @@ credentials       # API keys saved by `config set-key` (0600)
 settings.json     # permission rules, hooks, MCP servers (also per project: .cometcli/)
 COMET.md          # memory the agent always reads (also per project)
 kb/               # your own knowledge-base cases
+intents/          # your own known questions (answered without a model call)
 history/          # triage signal history per profile (30 days)
 incidents/        # incident records and postmortems per profile
 keys/             # ops keyring (file backend)

docs/COMMANDS.md

@@ -169,7 +169,17 @@ stop if tombstoned → fix the root cause → wait for sync → wait out the jai
 
 ### Signing transactions
 
-Two signers; when both can sign, you pick one first, then approve the tx (the approval shows which signer will sign):
+Every signature is a **key access** and is approved on its own.
+- **What the approval shows**: the account, which key and where it lives, the decoded
+  messages, the fee, chain and sequence, and the agent's stated reason.
+- **No shortcuts**: no rule, mode, autopilot or earlier approval covers it, and the next
+  signature asks again.
+- **Gas**: estimated with an unsigned transaction, so the key isn't touched before you
+  approve.
+- **Remote approvals**: in `watch`, a Telegram approval of a key use expires after
+  5 minutes.
+
+Two signers; when both can sign, you pick one first, then approve the key use:
 
 - **cometcli's keyring** — `signer.key` (import with `cometcli keys add --recover`).
 - **the node container's own keyring** — the key never leaves the node: cometcli builds
@@ -545,6 +555,32 @@ session started in this directory, `-r/--resume <id>` a specific one (an id pref
 `--model`, `--effort low|medium|high|xhigh|max`, `--max-tokens N`. Every conversation is
 saved after each turn to `~/.cometcli/sessions/` (0600, already redacted).
 
+### Local answers and the model
+
+Every prompt is classified on your machine first.
+
+| What you ask | Who answers |
+|---|---|
+| A known question: "is anyone jailed?", "validator count", "how many peers", "is my node synced", "missed blocks", "open proposals", "do I need to vote", "rewards", "balance", "upgrade plan", "slashing params" | cometcli runs the read-only tools and answers from a fixed template. No model call, no tokens. Repeats within 30s come from a cache. |
+| `/incident` where triage finds one clear root case that has a playbook | cometcli runs the playbook itself; changes and transactions still ask you. Today that's node down, jailed for downtime, halt-height left set, and container memory limit. A failed step hands over to the model with everything done so far. |
+| Anything else: why/how questions, actions, unknown problems | The model. For `/incident` it starts from the triage cometcli already ran. |
+
+`/llm <question>` asks the model anyway, `/route` explains the last decision, and
+`/cost` counts local answers next to tokens. To turn routing off, set
+`agent.router: off`. Add your own questions in `~/.cometcli/intents/*.yaml` (see
+[EXTENDING.md](EXTENDING.md#known-questions)).
+
+**What the model sees.** On validator profiles, `agent.egress: strict` is the default:
+- Output of `bash`, `read`, `grep`, `node.logs` and `fleet.exec` reaches the model only
+  as a local summary.
+- Addresses, IPs, hashes, hostnames and blobs are masked, and repeated lines are folded
+  with a count.
+- Errors are kept first when output is long.
+- Your screen and the audit log still show the raw output.
+
+RPC/sentry profiles and general mode default to `filtered` (raw output after redaction).
+In every mode, anything carrying key material is withheld whole.
+
 ### Agent session commands (TUI, REPL, web)
 
 | Command | Effect |
@@ -615,6 +651,8 @@ agent:
   autopilot: [local-change]      # optional; on-chain is rejected
   redact_hosts: [val.internal]   # masked before any text reaches the LLM
   redact_endpoints: true         # also mask this profile's endpoint/SSH hosts
+  egress: ""                     # strict | filtered — default strict on validators (raw output only as local summaries)
+  router: on                     # answer known questions and known incidents locally (off = always the model)
   no_stream: false               # set true for endpoints that mishandle streaming
   effort: medium                 # reasoning depth; mapped per provider (see below)
   max_tokens: 0                  # output cap per round; 0 = provider default

docs/EXTENDING.md

@@ -218,10 +218,61 @@ list of them:
     signals: {node.peers: 0, config.pex: false, logs.peer_net: 3}   # must match
 ```
 
+A case can also carry a **playbook**: the fix as steps cometcli runs itself on
+`/incident` when the case is the clear root cause (the only root, or the most severe), with
+no model call:
+
+```yaml
+  playbook:
+    - {do: node.logs, args: {lines: 40}, note: "the last lines before it stopped"}
+    - {do: node.service, args: {action: start}}
+    - {do: wait.until, args: {condition: synced, timeout: 1800}}
+    - {do: val.unjail, when: "val.jailed == true", note: "asks you"}
+    - {do: bash, args: {command: "docker update --memory 4g {unit}"}}
+```
+
+- `do` is any tool.
+- `when` is a condition on the triage signals; the step is skipped if it doesn't hold.
+- String arguments may use `{unit}`, `{home}`, `{binary}` and `{container_home}` from the
+  profile.
+- Every step goes through the normal gates: shell commands are classified, changes ask,
+  and transactions ask for key access.
+- A declined step stops the playbook. A failed step hands the incident to the model with
+  what was done.
+
 Conditions are `<signal> <op> <value>` with `== != > >= < <= contains exists missing`;
 values are numbers, `true`/`false`, or strings. A signal triage couldn't collect never
 satisfies a condition (except `missing`). `cometcli node triage` prints every signal name
 and value — write conditions against those. Fix steps must start with `[read]`, `[change]`
 or `[tx]`; the agent still goes through the normal approvals for each. Ranking: more matched
 conditions first, then severity. The built-in suite checks every case's `test` fixture
 ranks it in the top 3 and that a healthy node matches nothing.
+
+## Known questions
+
+Questions cometcli answers without a model call live in an intent catalog. Built-ins ship
+in the binary (`internal/router/intents/`). Yours go in `~/.cometcli/intents/*.yaml`; the
+same `id` overrides a built-in.
+
+```yaml
+- id: sentry-peers
+  title: peers on the sentries
+  scope: node                       # node: one node | chain: any node of the chain
+  patterns: ['^sentry peers$']      # regexes over the lowercased prompt; a hit routes here
+  examples:                         # phrasings; a close enough prompt routes here
+    - how many peers do the sentries have
+  steps:                            # read-only tools only (observe/diagnose)
+    - {as: p, tool: node.peers}
+  answer: |                         # Go text/template; each step is .<as> with .Text .Data .Err
+    **{{.node}}** has {{get .p.Data "count"}} peers.
+    {{trim .p.Text}}
+  ttl: 30s                          # reuse the answer this long
+```
+
+Some prompts always go to the model, whatever matches:
+- open-ended ones: why, how do I, explain, should I, …
+- actions: unjail, restart, vote yes, …
+- multi-line prompts and long prompts
+
+If a step fails, the model answers instead. Template functions: `get`, `trim`, `lines`,
+`filter`, `count`.

internal/agent/commands.go

@@ -44,7 +44,9 @@ const CommandHelp = `  /mode [ops|accept-edits|readonly|bypass]  show or set the
   /reset, /clear                clear conversation, start a new audit session
   /status                       current setup: model, mode, scope, memory, MCP (+ live node snapshot in node mode)
   /compact [focus]              summarize the conversation to free context
-  /cost                         token usage for this session
+  /cost                         token usage for this session, and prompts answered locally
+  /llm <question>               ask the model even when cometcli knows the answer
+  /route                        why the last prompt was answered locally or sent to the model
   /effort [low|medium|high|xhigh|max|default]  show or set reasoning depth
   /permissions                  show permission rules
   /allow|/ask|/deny <rule>      add a rule for this session, e.g. /allow bash(docker logs:*)
@@ -274,12 +276,31 @@ func RunCommand(a *Agent, c *toolkit.Context, reg *toolkit.Registry, line string
 			return "compacted — the summary is attached to your next message", nil
 		}}, nil
 
+	case "/llm":
+		if err := needAgent(); err != nil {
+			return CmdResult{}, err
+		}
+		q := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), f[0]))
+		if q == "" {
+			return CmdResult{}, fmt.Errorf("usage: /llm <question> — ask the model even when cometcli knows the answer")
+		}
+		return CmdResult{Text: "/llm", Prompt: forceModel + q}, nil
+
+	case "/route":
+		if err := needAgent(); err != nil {
+			return CmdResult{}, err
+		}
+		return CmdResult{Text: a.RouteWhy()}, nil
+
 	case "/cost", "/usage":
 		if err := needAgent(); err != nil {
 			return CmdResult{}, err
 		}
 		u, last := a.Usage()
 		txt := fmt.Sprintf("input %s (cached %s) · output %s", humanTok(u.Input), humanTok(u.CacheRead), humanTok(u.Output))
+		if r := a.Routes(); r.Local+r.Model > 0 {
+			txt += fmt.Sprintf("\nprompts: %d answered locally (no tokens%s), %d sent to the model", r.Local+r.Cached, cachedNote(r.Cached), r.Model)
+		}
 		if last > 0 {
 			txt += fmt.Sprintf("\ncontext %s of %s — auto-compacts at %s", humanTok(last), humanTok(a.contextWindow()), humanTok(a.compactThreshold()))
 		}
@@ -484,6 +505,10 @@ func RunCommand(a *Agent, c *toolkit.Context, reg *toolkit.Registry, line string
 				prompt = "This is about one of the operator's nodes: call use_node with the right profile first.\n\n" + prompt
 			}
 		}
+		if name == "incident" && a.Node() && a.routerOn() {
+			// triage and known cases run locally first (runIncident)
+			prompt = incidentMarker + args
+		}
 		return CmdResult{Text: text, Prompt: prompt}, nil
 	}
 	return CmdResult{}, ErrUnknownCommand
@@ -548,3 +573,10 @@ func humanTok(n int) string {
 	}
 	return fmt.Sprint(n)
 }
+
+func cachedNote(n int) string {
+	if n == 0 {
+		return ""
+	}
+	return fmt.Sprintf(", %d from cache", n)
+}

internal/agent/compact.go

@@ -112,6 +112,7 @@ func (a *Agent) compactNow(ctx context.Context, instructions string, midTurn boo
 		Model: a.Model, System: a.system(), Messages: hist,
 		Tools: a.toolDefs(), MaxTok: 8192, Effort: a.Effort,
 	}
+	a.guard(req)
 	resp, err := a.Provider.Chat(ctx, req)
 	if err != nil {
 		return err

internal/agent/egress.go

@@ -0,0 +1,78 @@
+package agent
+
+import (
+	"strings"
+
+	"github.com/abhijitkrm/cometcli/internal/audit"
+	"github.com/abhijitkrm/cometcli/internal/egress"
+)
+
+// egressMode is how much raw output the model may see in the current
+// scope (it changes when the session moves onto a node).
+func (a *Agent) egressMode() egress.Mode {
+	p := a.Ctx.Profile
+	if p == nil {
+		return egress.ModeFor(a.conf.Egress, "", false)
+	}
+	setting := a.conf.Egress
+	if p.Agent.Egress != "" {
+		setting = p.Agent.Egress
+	}
+	return egress.ModeFor(setting, p.Role, true)
+}
+
+// guard is the last check before a request leaves the machine: any part
+// of it that still carries key material is replaced whole — in the
+// session history too, so it's never offered again.
+func (a *Agent) guard(req *Request) {
+	if f := egress.Scan(req.System); len(f) > 0 {
+		a.withheld("system prompt", f)
+		req.System = egress.Withheld(f)
+		a.sys = req.System
+	}
+	for i := range req.Messages {
+		m := &req.Messages[i]
+		if f := egress.Scan(m.Text); len(f) > 0 {
+			a.withheld(m.Role+" message", f)
+			m.Text = egress.Withheld(f)
+		}
+		if len(m.RawSteps) > 0 {
+			if f := egress.Scan(string(m.RawSteps)); len(f) > 0 {
+				a.withheld("raw model output", f)
+				m.RawSteps, m.RawProvider = nil, ""
+			}
+		}
+		for j := range m.Calls {
+			if f := egress.Scan(string(m.Calls[j].Args)); len(f) > 0 {
+				a.withheld("tool call arguments", f)
+				m.Calls[j].Args = []byte(`{}`)
+			}
+		}
+		if f := egress.Scan(string(m.JSONArgs)); len(f) > 0 {
+			m.JSONArgs = nil
+		}
+	}
+	// req.Messages aliases a.history: the withheld forms are now the history
+}
+
+func (a *Agent) withheld(what string, f []egress.Finding) {
+	a.emit(Event{Kind: EvNotice, Text: "withheld " + what + " from the model: it contained key material (" + egress.Kinds(f) + ")"})
+	if lg := a.Audit(); lg != nil {
+		_ = lg.Log(audit.KindEgress, a.profileName(), map[string]any{"what": what, "kinds": egress.Kinds(f)})
+	}
+}
+
+// purpose is the agent's latest stated reason (its text before the tool
+// call), shown when a key is about to be used.
+func (a *Agent) purpose() string {
+	for i := len(a.history) - 1; i >= 0; i-- {
+		m := a.history[i]
+		if m.Role == "user" {
+			return ""
+		}
+		if m.Role == "assistant" && strings.TrimSpace(m.Text) != "" {
+			return clip(strings.Join(strings.Fields(a.Redact.Text(m.Text)), " "), 300)
+		}
+	}
+	return ""
+}

internal/agent/egress_test.go

@@ -0,0 +1,80 @@
+package agent
+
+import (
+	"context"
+	"encoding/json"
+	"strings"
+	"testing"
+
+	"github.com/cosmos/go-bip39"
+
+	"github.com/abhijitkrm/cometcli/internal/toolkit"
+)
+
+// providerSaw is everything sent to the provider, as one string.
+func providerSaw(p *mockProvider) string { return strings.Join(p.sent, "\n") }
+
+func TestStrictProfilesSendSummariesNotRawOutput(t *testing.T) {
+	tool := stubTool{name: "node.logs", tier: toolkit.TierDiagnose, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+		return &toolkit.Result{Text: "ERR dial tcp 10.20.30.40:26656: i/o timeout\nERR dial tcp 10.20.30.41:26656: i/o timeout\n"}, nil
+	}}
+	prov := &mockProvider{responses: []*Response{
+		{Calls: []Call{{ID: "c1", Name: "node__logs", Args: json.RawMessage(`{}`)}}},
+		{Text: "ok", Done: true},
+	}}
+	a := newTestAgent(t, prov, tool) // a validator profile: strict by default
+	var shown string
+	a.OnEvent = func(e Event) {
+		if e.Kind == EvToolResult {
+			shown = e.Output
+		}
+	}
+	a.Run(context.Background(), "logs")
+	sent := providerSaw(prov)
+	if strings.Contains(sent, "10.20.30.4") || !strings.Contains(sent, "local summary") || !strings.Contains(sent, "(×2)") {
+		t.Fatalf("model should see only the local summary:\n%s", sent)
+	}
+	if !strings.Contains(shown, "10.20.30.40") {
+		t.Errorf("the operator's UI keeps the raw output, got %q", shown)
+	}
+}
+
+func TestKeyMaterialNeverReachesTheProvider(t *testing.T) {
+	ent, _ := bip39.NewEntropy(256)
+	mn, _ := bip39.NewMnemonic(ent)
+	tool := stubTool{name: "node.config", tier: toolkit.TierDiagnose, run: func(*toolkit.Context, toolkit.Args) (*toolkit.Result, error) {
+		// redaction doesn't know this shape; the gate must stop it
+		return &toolkit.Result{Text: "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQ\n-----END OPENSSH PRIVATE KEY-----"}, nil
+	}}
+	prov := &mockProvider{responses: []*Response{
+		{Calls: []Call{{ID: "c1", Name: "node__config", Args: json.RawMessage(`{}`)}}},
+		{Text: "ok", Done: true},
+	}}
+	a := newTestAgent(t, prov, tool)
+	a.conf.Egress = "filtered" // the gate applies in every mode
+	var notices []string
+	a.OnEvent = func(e Event) {
+		if e.Kind == EvNotice {
+			notices = append(notices, e.Text)
+		}
+	}
+	if _, err := a.Run(context.Background(), "show the deploy key"); err != nil {
+		t.Fatal(err)
+	}
+	sent := providerSaw(prov)
+	if strings.Contains(sent, "b3BlbnNzaC1rZXktdjEAAAAABG5vbmU") || !strings.Contains(sent, "withheld by cometcli") {
+		t.Fatalf("private key reached the provider:\n%s", sent)
+	}
+	if len(notices) == 0 {
+		t.Error("the operator should be told something was withheld")
+	}
+
+	// a mnemonic typed by the operator isn't sent at all
+	before := len(prov.sent)
+	if _, err := a.Run(context.Background(), "restore my key: "+mn); err == nil || !strings.Contains(err.Error(), "key material") {
+		t.Fatalf("want a refusal, got %v", err)
+	}
+	if len(prov.sent) != before || strings.Contains(providerSaw(prov), strings.Fields(mn)[5]+" "+strings.Fields(mn)[6]) {
+		t.Fatal("a message with a mnemonic must not be sent")
+	}
+}