Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"runtimeExecutable": "bash",
"runtimeArgs": [
"-c",
"cd /Users/architsharma/guardrails/.claude/worktrees/gap-analysis-failure-modes-2a3ba9/dashboard && AGENTFOX_API_URL=http://127.0.0.1:8081 NOMETRIA_API_URL=http://127.0.0.1:8081 NEXT_DIST_DIR=.next-dev PORT=3001 npm run dev -- -p 3001"
"cd /Users/architsharma/guardrails/.claude/worktrees/gap-analysis-failure-modes-2a3ba9/dashboard && AGENTFOX_API_URL=http://127.0.0.1:8081 NEXT_DIST_DIR=.next-dev PORT=3001 npm run dev -- -p 3001"
],
"port": 3001
}
Expand Down
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ dashboard/.npm-cache/
**/.env
**/.env.*
!**/.env.example
# Written by `agentfox init` when run from the repo root without an explicit
# --path — a local dev convenience file, not a repo artifact.
/nometria.toml

# Business, go-to-market and competitor material that must not reach a public
# repository: drafted grant and RFP packages, launch screenshots, post copy, a
Expand Down
5 changes: 1 addition & 4 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,10 +309,7 @@ URL a user typed (a spec, an MCP server, a probe target) goes through `core/outb
not open a database or import a client library.

**Settings names.** `AGENTFOX_*` settings, `agentfox.toml` / `[agentfox]` and `X-AgentFox-*`
headers are the only names written or documented. The pre-rename `NOMETRIA_*`, `nometria.toml`
/ `[nometria]` and `x-nometria-*` are still *read*, at lower precedence and with a startup
deprecation warning (`core/config.py`, `core/headers.py`), until the deployments that set
them are renamed (`docs/deployment/vercel-env-rename.md`).
headers are the only names read or written.

**Published numbers are bound.** [`benchmarks/claims.yaml`](benchmarks/claims.yaml) binds each
quoted figure (in this README, `benchmarks/`, `docs/`, website pages) to the result file it
Expand Down
35 changes: 29 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,9 @@ the file it came from.
`system_scope`. `chain.verify` accepts one key or several.
- The `verify_chain.py` in evidence packages accepts several comma-separated keys in
`AGENTFOX_AUDIT_KEY`, for a chain that spans a rotation.
- One-time migration bridge (this release only): when both `AGENTFOX_<KEY>` and
`NOMETRIA_<KEY>` are set and differ for `TOKEN_ENCRYPTION_KEY` or
`AUDIT_SIGNING_KEY`, the `NOMETRIA_` value is used as the previous key. See
docs/deployment/vercel-env-rename.md.
- One-time migration bridge: when both `AGENTFOX_<KEY>` and `NOMETRIA_<KEY>` were set
and differed for `TOKEN_ENCRYPTION_KEY` or `AUDIT_SIGNING_KEY`, the `NOMETRIA_` value
was used as the previous key. Removed again by rename stage B (below).

- **Capability packs.** A business use case or framework is one directory with a
`pack.yaml` (id, version, maturity, owners, compliance mappings, vocabulary) and its
Expand All @@ -57,7 +56,31 @@ the file it came from.

### Removed

- The `nometria` Python package shim and the `nometria` console script. Import from `agentfox` (e.g. `agentfox.frameworks.langgraph`) and run `agentfox`. The `NOMETRIA_*` environment variables, `nometria.toml` and `x-nometria-*` headers are still read.
- The `nometria` Python package shim and the `nometria` console script. Import from `agentfox` (e.g. `agentfox.frameworks.langgraph`) and run `agentfox`.
- **The last of the Nometria names (rename stage B). Breaking.** The `NOMETRIA_*`
environment fallback (settings, `NOMETRIA_CONFIG`, the CLI's `NOMETRIA_API_URL` /
`NOMETRIA_API_TOKEN` / `NOMETRIA_USER`, `NOMETRIA_AGENT`, `NOMETRIA_MCP_LOG_LEVEL`, the
evidence package's `NOMETRIA_AUDIT_KEY`, the dashboard's `NOMETRIA_*`, the demos'
`NOMETRIA_DEMO_MODEL` and `NOMETRIA_DATABASE_*`), the `nometria.toml` / `[nometria]`
config file, acceptance of `x-nometria-*` request headers, the `legacy names` doctor
line and the one-time `NOMETRIA_` key bridge are gone. Only `AGENTFOX_*`,
`agentfox.toml` / `[agentfox]` and `X-AgentFox-*` are read; an old name is ignored
without a warning. Key rotation (`*_PREVIOUS`) stays.
- **Breaking:** the dashboard session cookie is `agentfox_session` (was
`nometria_session`). Everyone is signed out once and signs in again.
- **Breaking:** the LangGraph governance state key is `__agentfox__` (was
`__nometria__`). Declare `__agentfox__` in your state schema. Graph state checkpointed
before the upgrade keeps its governance state under the old key, where it is no
longer read: an in-flight thread resumed across the upgrade starts a new trace.
- **Breaking:** OPA policy documents are pushed as `agentfox_<policy>` (were
`nometria_<policy>`). An OPA sidecar that kept documents pushed by an older version
holds both under the same package; restart it (pushed policies are in memory) or
`DELETE /v1/policies/nometria_<policy>` for each.
- The secrets detector's entity for AgentFox API and agent keys (`nom_api_…`,
`nom_agt_…`) is `SECRET.AGENTFOX_KEY` (was `SECRET.NOMETRIA_KEY`). The `SECRET.*`
policy rules match it unchanged; a rule naming the old entity must be updated. The
key prefixes themselves are unchanged, so issued keys keep working and keep being
detected.

- The hidden pre-consolidation CLI names. `agentfox --help` shows thirteen verbs, and
the old top-level names had kept running, hidden, with a "now called" hint. They no
Expand Down Expand Up @@ -116,7 +139,7 @@ the file it came from.
variable. `render.yaml`, `deploy/`, the docs and the plugins use only the new names.
The self-host blueprint now prompts for `AGENTFOX_AUDIT_SIGNING_KEY` instead of
generating it, so a blueprint sync can never rotate an existing deployment's key.
Renaming the hosted deployment: `docs/deployment/vercel-env-rename.md`.
Renaming the hosted deployment: `docs/deployment/vercel-env-rename.md` (since removed).
- The shipped policies and the control catalog moved from `policies_data/` and
`compliance_data/` into the capability packs (`packs/<id>/policies/`,
`packs/compliance/catalog/controls/`). `compliance_dir` and `policies_dir` still
Expand Down
4 changes: 2 additions & 2 deletions api/vendor/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# This directory was `*`, which is why the deployment rotted silently: the Vercel
# function installs `../vendor/agentfox-0.3.0-py3-none-any.whl` by name from
# api/requirements.txt, but that file was ignored, so no rebuild ever reached the
# repository. What was actually committed here was `nometria-0.1.0-py3-none-any.whl`
# — the package name from before the rename — force-added once and then frozen,
# repository. What was actually committed here was a wheel under the package's
# pre-rename name, force-added once and then frozen,
# while every later `uv build` landed only on the machine that ran it.
#
# Negating the one named wheel means a rebuild shows up in `git status` like any
Expand Down
Binary file modified api/vendor/agentfox-0.3.1-py3-none-any.whl
Binary file not shown.
2 changes: 1 addition & 1 deletion dashboard/app/(marketing)/privacy/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ export default function Privacy() {
</p>
<div className="mk-grid mk-grid-2" style={{ marginTop: 18 }}>
<div className="mk-card">
<p className="mk-label">nometria_session</p>
<p className="mk-label">agentfox_session</p>
<p className="mk-body" style={{ margin: "10px 0 0", fontSize: "var(--t-body)" }}>
The API token minted for you at sign-in, which is what the control
plane checks on every request. <span className="mk-mono">httpOnly</span>
Expand Down
6 changes: 3 additions & 3 deletions dashboard/app/api/auth/logout/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ const { POST } = await import("./route");
function logout(cookie?: string) {
return new NextRequest("http://localhost:3000/api/auth/logout", {
method: "POST",
headers: cookie ? { Cookie: `nometria_session=${cookie}` } : {},
headers: cookie ? { Cookie: `agentfox_session=${cookie}` } : {},
});
}

Expand All @@ -34,13 +34,13 @@ describe("sign out", () => {
expect(init.method).toBe("POST");
expect(init.headers.Authorization).toBe("Bearer nom_api_session123");
expect(res.headers.get("location")).toMatch(/\/login$/);
expect(res.headers.get("set-cookie") || "").toMatch(/nometria_session=;/);
expect(res.headers.get("set-cookie") || "").toMatch(/agentfox_session=;/);
});

it("still signs the browser out when the gateway is unreachable", async () => {
fetchMock.mockRejectedValueOnce(new Error("ECONNREFUSED"));
const res = await POST(logout("nom_api_session123"));
expect(res.headers.get("set-cookie") || "").toMatch(/nometria_session=;/);
expect(res.headers.get("set-cookie") || "").toMatch(/agentfox_session=;/);
});

it("makes no gateway call without a session", async () => {
Expand Down
2 changes: 1 addition & 1 deletion dashboard/app/api/auth/token/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ describe("sign in with an API token", () => {
expect(String(url)).toMatch(/\/api\/me$/);
expect(init.headers.Authorization).toBe("Bearer nom_api_good");
expect(res.headers.get("location")).toMatch(/\/app\/start$/);
expect(res.headers.get("set-cookie") || "").toMatch(/nometria_session=nom_api_good/);
expect(res.headers.get("set-cookie") || "").toMatch(/agentfox_session=nom_api_good/);
});

it("refuses a token the gateway rejects", async () => {
Expand Down
4 changes: 2 additions & 2 deletions dashboard/app/docs/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,10 @@ AGENTFOX_API_URL=http://127.0.0.1:8080 npm run dev`}</Code>
<Output>{`{"error":"GitHub sign-in is not configured (GITHUB_CLIENT_ID is unset)."}`}</Output>
<p>
For a local evaluation only, the session is the token itself, held in a cookie
named <code>nometria_session</code>. Open <code>http://localhost:3000/login</code>,
named <code>agentfox_session</code>. Open <code>http://localhost:3000/login</code>,
then in the browser console:
</p>
<Code lang="ts" title="browser console">{`document.cookie = "nometria_session=<the token from step 3>; path=/"; location.href = "/app";`}</Code>
<Code lang="ts" title="browser console">{`document.cookie = "agentfox_session=<the token from step 3>; path=/"; location.href = "/app";`}</Code>
<Callout kind="warning">
This is a workaround for a local instance, not a sign-in method. Anything with that
cookie acts as the token&apos;s user. Do not do it on a shared or public
Expand Down
6 changes: 3 additions & 3 deletions dashboard/app/docs/guides/langgraph/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def keep_latest(old: dict, new: dict) -> dict:
class State(TypedDict, total=False):
messages: Annotated[list, add_messages]
docs: str
__nometria__: Annotated[dict[str, Any], keep_latest] # AgentFox's governance state
__agentfox__: Annotated[dict[str, Any], keep_latest] # AgentFox's governance state


guard = AgentFoxGuard(
Expand Down Expand Up @@ -246,7 +246,7 @@ resumed: Ticket filed.`}</Output>
<h2>The governance key in your state</h2>
<p>
The guard returns its bookkeeping (trace id, last verdict, what retrieval nodes
read, the tools called and each step) under the state key <code>__nometria__</code>{" "}
read, the tools called and each step) under the state key <code>__agentfox__</code>{" "}
(exported as <code>STATE_KEY</code>). Declare it in your state, as in{" "}
<code>graph.py</code>:
</p>
Expand All @@ -265,7 +265,7 @@ resumed: Ticket filed.`}</Output>
<li><strong>A grant&apos;s <code>--limit</code> refuses with &quot;this call passed None&quot;</strong>: the tool node found no arguments. Pass <code>arguments=</code>, or put the model&apos;s tool call in <code>state[&quot;messages&quot;]</code>.</li>
<li><strong>A node refused with <code>capability.denied</code></strong>: grant the tool to the agent; the first run registers the agent so the grant can name it.</li>
<li><strong>A paused run cannot be resumed</strong>: compile the graph with a checkpointer (<code>InMemorySaver</code> for tests) and pass the same <code>thread_id</code>.</li>
<li><strong>Trace id missing from the result</strong>: declare <code>__nometria__</code> in the state, with a merging reducer.</li>
<li><strong>Trace id missing from the result</strong>: declare <code>__agentfox__</code> in the state, with a merging reducer.</li>
</ul>

<h2>Limits</h2>
Expand Down
5 changes: 0 additions & 5 deletions dashboard/app/docs/install/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -253,11 +253,6 @@ taint_scope = "session"`}</Code>
<li>
<code>AGENTFOX_*</code> environment variables;
</li>
<li>
the pre-rename <code>NOMETRIA_*</code> environment variables, deprecated: still
read, with a startup warning naming each one (<code>agentfox doctor</code> lists
every one still set);
</li>
<li>
the <code>[agentfox]</code> table of <code>$AGENTFOX_CONFIG</code> if set (it must
exist), otherwise of <code>./agentfox.toml</code> in the working directory;
Expand Down
5 changes: 1 addition & 4 deletions dashboard/app/docs/reference/api/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,7 @@ export default function Page() {

<Callout kind="note">
<p>
Headers are <code>X-AgentFox-*</code>. The pre-rename <code>X-Nometria-*</code>{" "}
request headers are still accepted, for clients not yet updated; where a request
carries both, <code>X-AgentFox-*</code> wins. Responses carry only{" "}
<code>X-AgentFox-*</code>.
Headers are <code>X-AgentFox-*</code>.
</p>
</Callout>

Expand Down
30 changes: 10 additions & 20 deletions dashboard/app/docs/reference/config/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -814,15 +814,15 @@ const GROUPS: { id: string; title: string; rows: Row[] }[] = [
}
];

const OUTSIDE: { name: string; legacy: string; what: string }[] = [
{ name: "AGENTFOX_STATE_DIR", legacy: "—", what: "Directory for the default database and evidence. It decides the defaults of database_url and evidence_dir, so it is not itself a setting." },
{ name: "AGENTFOX_CONFIG", legacy: "NOMETRIA_CONFIG", what: "Path to the TOML file to read. It must exist. none, off or - turns file loading off." },
{ name: "AGENTFOX_AGENT", legacy: "NOMETRIA_AGENT", what: "Agent slug for agentfox.auto() when none is passed. Then OTEL_SERVICE_NAME, SERVICE_NAME, APP_NAME, K_SERVICE, the script name, and finally default-agent." },
{ name: "AGENTFOX_API_URL, AGENTFOX_API_TOKEN, AGENTFOX_USER", legacy: "NOMETRIA_API_URL, NOMETRIA_API_TOKEN, NOMETRIA_USER", what: "Where agentfox scan --submit sends a redacted summary, and the credential it uses." },
{ name: "AGENTFOX_AUDIT_KEY", legacy: "NOMETRIA_AUDIT_KEY, then the *_AUDIT_SIGNING_KEY names", what: "Read by the verify_chain.py inside an evidence package to check checkpoint signatures." },
{ name: "AGENTFOX_MCP_LOG_LEVEL", legacy: "NOMETRIA_MCP_LOG_LEVEL", what: "Log level of agentfox serve mcp. Default WARNING." },
{ name: "CRON_SECRET", legacy: "—", what: "Accepted in addition to cron_secret by /api/internal/jobs/run (Vercel Cron sets it)." },
{ name: "JEV_API_KEY", legacy: "—", what: "Key for the hosted jev judgment tier. Without it that tier is unavailable." },
const OUTSIDE: { name: string; what: string }[] = [
{ name: "AGENTFOX_STATE_DIR", what: "Directory for the default database and evidence. It decides the defaults of database_url and evidence_dir, so it is not itself a setting." },
{ name: "AGENTFOX_CONFIG", what: "Path to the TOML file to read. It must exist. none, off or - turns file loading off." },
{ name: "AGENTFOX_AGENT", what: "Agent slug for agentfox.auto() when none is passed. Then OTEL_SERVICE_NAME, SERVICE_NAME, APP_NAME, K_SERVICE, the script name, and finally default-agent." },
{ name: "AGENTFOX_API_URL, AGENTFOX_API_TOKEN, AGENTFOX_USER", what: "Where agentfox scan --submit sends a redacted summary, and the credential it uses." },
{ name: "AGENTFOX_AUDIT_KEY", what: "Then AGENTFOX_AUDIT_SIGNING_KEY. Read by the verify_chain.py inside an evidence package to check checkpoint signatures." },
{ name: "AGENTFOX_MCP_LOG_LEVEL", what: "Log level of agentfox serve mcp. Default WARNING." },
{ name: "CRON_SECRET", what: "Accepted in addition to cron_secret by /api/internal/jobs/run (Vercel Cron sets it)." },
{ name: "JEV_API_KEY", what: "Key for the hosted jev judgment tier. Without it that tier is unavailable." },
];

const DASHBOARD: { name: string; def: string; what: string }[] = [
Expand Down Expand Up @@ -859,17 +859,9 @@ export default function Page() {
<li>
<code>AGENTFOX_&lt;KEY&gt;</code> in the environment.
</li>
<li>
<code>NOMETRIA_&lt;KEY&gt;</code>, the pre-rename name: deprecated, still read so
existing deployments keep working, with one startup warning naming each one in
use. <code>agentfox doctor</code> lists every one still set. Where both are set,{" "}
<code>AGENTFOX_</code> wins.
</li>
<li>
The <code>[agentfox]</code> table of the file named by <code>AGENTFOX_CONFIG</code>,
or of <code>./agentfox.toml</code> in the working directory. A pre-rename{" "}
<code>nometria.toml</code> or <code>[nometria]</code> table is still read, with the
same warning.
or of <code>./agentfox.toml</code> in the working directory.
</li>
<li>The default in the tables below.</li>
</ol>
Expand Down Expand Up @@ -945,7 +937,6 @@ python -c "from agentfox.core.config import Settings as S; s = S(); print(s.fail
<thead>
<tr>
<th>Variable</th>
<th>Deprecated name, still read</th>
<th>What it does</th>
</tr>
</thead>
Expand All @@ -955,7 +946,6 @@ python -c "from agentfox.core.config import Settings as S; s = S(); print(s.fail
<td>
<code>{v.name}</code>
</td>
<td>{v.legacy}</td>
<td>{v.what}</td>
</tr>
))}
Expand Down
2 changes: 1 addition & 1 deletion dashboard/app/docs/reference/detectors/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ const OPTIN: Row[] = [
const ENTITIES: Row[] = [
["INJECTION.*", "INSTRUCTION_OVERRIDE, INSTRUCTION_INJECTION, INSTRUCTION_LEAK, INSTRUCTION_PERSONA, PERSONA_OVERRIDE, SYSTEM_PROMPT_LEAK, JAILBREAK, COVERT_INSTRUCTION, EXFILTRATION, ROLE_DELIMITER, CONTROL_TOKENS, FAKE_SYSTEM_BLOCK, INSTRUCTION_IN_DATA, HIDDEN_CHARACTERS, HIDDEN_INSTRUCTION, ENCODED_PAYLOAD, OBFUSCATED_CONTENT (injection.heuristic); JAILBREAK (classifiers); SEMANTIC_SIMILARITY (injection.similarity); CLASSIFIER, UNUSUAL (Hub validators)"],
["PII.*", "EMAIL, IP_ADDRESS, CREDIT_CARD, IBAN, DATE_OF_BIRTH, US_SSN, US_PHONE, US_PASSPORT, US_MRN, UK_NINO, UK_NHS, EU_VAT, IN_AADHAAR, IN_PAN (pii.native); PERSON, LOCATION, DATE_TIME, US_DRIVER_LICENSE, MEDICAL_LICENSE, CRYPTO_WALLET (pii.presidio); PRESENT_UNLOCATED (pii.judgment: personal data present, location unknown); HUB"],
["SECRET.*", "OPENAI_KEY, ANTHROPIC_KEY, AWS_ACCESS_KEY, GITHUB_TOKEN, SLACK_TOKEN, GOOGLE_API_KEY, STRIPE_KEY, NOMETRIA_KEY, PRIVATE_KEY, JWT, CONNECTION_STRING, GENERIC (secrets.native); HUB"],
["SECRET.*", "OPENAI_KEY, ANTHROPIC_KEY, AWS_ACCESS_KEY, GITHUB_TOKEN, SLACK_TOKEN, GOOGLE_API_KEY, STRIPE_KEY, AGENTFOX_KEY, PRIVATE_KEY, JWT, CONNECTION_STRING, GENERIC (secrets.native); HUB"],
["SAFETY.*", "HARM, SELF_HARM, ILLICIT, HARASSMENT, EXTREMISM (safety.lexicon; SEXUAL is a category with no patterns yet); HARM (Granite, Llama Guard); TOXIC, NSFW, PROFANITY, DRUGS, BIAS, BANNED_TERM, LLAMA_GUARD, SHIELD_GEMMA (Hub validators)"],
["SCHEMA.*", "VIOLATION, UNPARSEABLE (schema.json); SQL_INVALID, JSON_INVALID (Hub validators)"],
["RAILS.BLOCKED", "A NeMo rail refused the content."],
Expand Down
2 changes: 1 addition & 1 deletion dashboard/app/docs/reference/python/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,7 @@ guard.tool_node(fn=None, *, tool: str, provenance=None, arguments=None, messages
</li>
<li>
Governance state (trace id, last verdict, what was retrieved, tools called) is written under the{" "}
<code>&quot;__nometria__&quot;</code> key of the graph state, so it survives a
<code>&quot;__agentfox__&quot;</code> key of the graph state, so it survives a
checkpoint. Add that key to your state schema.
</li>
<li>
Expand Down
Loading
Loading