lattice-server

The Lattice control plane server.

Responsibilities:

Admin login, sessions, CSRF checks, and secure cookie defaults.
Scope and server allowlist authorization.
Node enrollment and outbound-agent APIs.
Operator-owned node metadata: name, role, sorted tags, and control-plane-only comments.
Fleet metrics and HostFacts inventory telemetry ingestion.
Server-only machine inventory profiles: vendor, region, encrypted console/detail links, cost, renewal cycles, and renewal reminders.
Batch task scheduling and result collection.
KV/static/Worker control APIs.
nftables plan and approval workflow with persisted per-node baseline inputs (public TCP/UDP, WireGuard TCP/UDP, interface, WireGuard CIDR).
NetPolicy APIs, reachability graph, rollback-protected egress nft apply, and ingress policy composition into the Network Guard input table.
Self-host DNS deployment intent CRUD with encrypted Cloudflare token storage, secret-free read views, CoreDNS/nft plan generation, rollback-protected apply, task-result status reconciliation, and Cloudflare hostname publication through the existing DDNS provider.
Proxy-core/subscription persistence foundation: encrypted Reality private keys, user UUID/password credentials, subscription tokens, redacted proto view contracts, JSON/bbolt store parity, and the first fail-closed sing-box vless+TCP+REALITY renderer, plus scoped CRUD/read APIs with secret-free views, a redacted reviewed plan endpoint, and secret-safe queue/apply through encrypted task scripts, sing-box check, atomic config swap, reload/restart, task-result status reconciliation, public subscription serving, audited subscription-token rotation, sing-box JSON plus Clash/Mihomo YAML subscription output, and baseline proxy usage rollup.
Operator-owned NodeGeo API and optional server-side GeoIP lookup for the dashboard Fleet Map.
Append-only audit events with a hash-chained WAL and sidecar head anchor.

Run Locally

LATTICE_ADMIN_PASSWORD='change-this-passphrase' \
LATTICE_ADMIN_USERNAME='admin' \
LATTICE_WEB_ROOT=../lattice-dashboard \
go run ./cmd/lattice-server

Open http://127.0.0.1:8088.

Readiness and runtime metrics are split by exposure risk. /readyz returns a small JSON readiness result for load balancers. /metrics is hidden unless a bearer token is configured, because it exposes fleet runtime counters:

LATTICE_METRICS_TOKEN='opaque-prometheus-token' \
go run ./cmd/lattice-server

curl -H 'Authorization: Bearer opaque-prometheus-token' \
  http://127.0.0.1:8088/metrics

Without the token, the server still logs its state.json writes once an hour, counted by the store method that asked for each one, for example state.json writes in the last 60m: 12 (UpdateMetrics 11, UpsertDDNSProfile 1). Writes to the bolt hot store are not in that count. An hour with no state.json writes logs nothing.

The server also keeps its own long-term history in metrics.db beside the state file: its process (CPU, RSS, Go heap, goroutines, GC pause, open files) and host as the container sees it (load, memory, the data volume, the sizes of the data files), state writes by caller, HTTP route groups, plugin calls by method with their processes' CPU and peak memory, and the metrics each node beats in. Points are kept at 1 minute for 48 hours, 5 minutes for 14 days, 1 hour for 90 days and 1 day for five years, and the number of series is capped (1024 by default, LATTICE_METRICS_MAX_SERIES to change it), so the file stops growing at a size the series count fixes. A full administrator reads it on the console's System page (/api/system/health, /api/system/series); a node's history is a node read (/api/nodes/history). It holds no secrets and no authority: deleting it loses history only, and backups may leave it out. A metrics.db the server cannot read (damaged, or written by a newer release before a rollback) is moved aside as metrics.db.unreadable-<unix time> and a fresh one started; the server never refuses to start over it.

The outbound probe is a separate program, lattice-probe, that runs beside the server and listens on a unix socket, /run/lattice-probe/probe.sock by default (LATTICE_PROBE_SOCKET to change it). vpn-core's Probe page reaches it through the host service latticenet.vpn-core/probe, which needs the vpn:probe scope and an unrestricted node allowlist; no proxy or vpncore grant implies it. Each operator may have 4 runs in flight and 120 in a rolling hour. A run the probe answered without using the network (a malformed request, a busy probe, an outbound that does not decode or create) does not count toward the 120. Every run is audited as vpncore.probe with the outbound types, the tested server's address, the targets, the outcome and a SHA-256 of the outbounds' canonical JSON, never the outbound itself. The server starts without the probe; a missing socket only makes it read as unavailable, with the reason, on the System page.

Operators can save up to 32 targets of their own beside the probe's built-in ones: a label, an https URL (port 443 or 1024 to 65535, no credentials, no fragment, no private address or local-only name), GET or HEAD, the statuses that pass (204 or 200-299,301), and optionally text the first 64 KiB of the body must contain. The probe service's targets method lists built-in and saved targets together, each marked builtin or custom, and says whether the running probe can measure saved ones. target_create, target_update and target_delete need vpncore:admin as well as vpn:probe; update and delete name the version they read and are refused with 409 when it moved. Each change is audited as vpncore.probe_target.<action> with the id and host only. A run still names targets by id: for each saved id the server sends the stored definition to the probe, which checks it again before fetching anything, and a run that brings definitions of its own is refused. A run that names a saved target is refused with 409 probe_outdated when the probe does not list the custom_targets capability, and its audit record adds each saved target's id, host[:port] and a short hash of its definition, never the URL's path or query.

Self-host DNS can optionally install a pinned CoreDNS executable during an approved selfdns apply. Leave these unset to keep the stricter precondition that coredns must already exist on the node:

LATTICE_COREDNS_BINARY_VERSION='1.12.4' \
LATTICE_COREDNS_BINARY_URL='https://example.com/releases/coredns-1.12.4-linux-amd64' \
LATTICE_COREDNS_BINARY_SHA256='<64 hex chars>' \
go run ./cmd/lattice-server

The URL must be HTTPS and point directly to an executable binary. The reviewed approval plan includes the version, URL, SHA-256, and fixed install path (/usr/local/bin/coredns); the node installs only after digest verification.

Fleet Map automatic node placement is enabled by default with the no-token https://ipwho.is/{ip} HTTPS JSON API. You do not need an IPInfo token for the normal Nezha-like auto-location flow. To disable external lookup entirely:

LATTICE_GEOIP_LOOKUP_URL=off \
go run ./cmd/lattice-server

To use a self-hosted or internal provider instead, set LATTICE_GEOIP_LOOKUP_URL to an HTTPS URL template containing {ip}. The response must be JSON with common country_code, region, city, latitude/longitude, and optional ASN/provider fields. Manual map coordinates work even when automatic lookup is disabled.

Audit WAL head custody can be automated with an HTTPS webhook. The server ships only a verified, locally anchored head; failed verification, a missing committed anchor, HTTP failures, or SSRF/egress-guard blocks are logged and do not block normal API traffic:

LATTICE_AUDIT_HEAD_WEBHOOK_URL='https://audit-archive.example.com/lattice/head' \
LATTICE_AUDIT_HEAD_WEBHOOK_TOKEN='opaque-bearer-token' \
LATTICE_AUDIT_HEAD_INTERVAL=15m \
go run ./cmd/lattice-server

The webhook URL must be HTTPS and must not contain userinfo, query, or fragment secrets. Outbound delivery uses the same guarded HTTP client as operator webhooks: local, private, link-local, metadata, special-use targets, and guarded redirects are blocked. The receiver is responsible for immutable retention and rollback alerts.

Build

From the organization workspace:

cd ../lattice
make build

Standalone builds require github.com/LatticeNet/lattice-sdk to be available at the version in go.mod; during local multi-repo development, use the lattice/go.work workspace.

Verify

What CI runs, in order (.github/workflows/ci.yml):

test -z "$(gofmt -l .)"
sh scripts/check-docker-defaults.sh
go vet ./...
go test -race -cover -timeout 25m ./...
gosec ./...
govulncheck ./...

The -timeout 25m is not decorative. internal/server alone runs well past go test's 10 minute default under -race -cover, so a plain go test ./... fails on a timeout that has nothing to do with the code under test.

Docker

The server image embeds lattice-dashboard and keeps runtime state under /var/lib/lattice.

Local multi-repo build:

cd ..
DOCKER_BUILDKIT=1 docker build \
  -f lattice-server/Dockerfile \
  --build-context lattice-sdk=./lattice-sdk \
  --build-context lattice-dashboard=./lattice-dashboard \
  -t lattice-server:local \
  ./lattice-server

Published image:

ghcr.io/latticenet/lattice-server

Image publication is tag-driven. During rapid alpha development, test deployments use alpha-0.2.1aN tags and only increment the trailing alpha number. The moving latest git tag publishes :latest, the moving alpha git tag publishes :alpha, and stable v* tags publish immutable version tags only after an explicit stable-release decision. Pushed image builds also publish a sha-<commit> tag, and the rolling alpha channel additionally publishes alpha-<commit>, so the GitHub Packages page has human-readable immutable entries instead of only raw digest rows. Source pushes to main run CI but do not publish a main image channel.

The image workflow writes OCI title, description, source, revision, license, and vendor metadata as both image labels and multi-arch manifest annotations. Build provenance/SBOM attestations are disabled to avoid extra untagged package versions. The package cleanup workflow prunes old untagged container package versions, protects child manifests referenced by every currently tagged image, and keeps the newest 10 stale untagged versions by default as a rollback buffer.

The container leaves LATTICE_MASTER_KEY_FILE unset by default so first boot can generate /var/lib/lattice/master.key automatically. Set it only when restoring or mounting a pre-existing key. The entrypoint chowns the mounted data directory to lattice:lattice when anything in it, the directory included, has another owner or group (a fresh bind mount, or a restore, docker cp or extract run as another user), then drops privileges to the lattice user. A tree lattice owns throughout is left alone, so an ordinary boot needs no CAP_CHOWN; without CAP_CHOWN, a tree that needs the repair stops the boot.

Manifest-v2 plugins require a writable private extraction cache, for example LATTICE_PLUGIN_BUNDLE_CACHE_DIR=/var/lib/lattice/plugin-bundles. Keep this separate from both the read-only plugin source mount (LATTICE_PLUGIN_DIR) and the executable working directories (LATTICE_PLUGIN_RUNTIME_DIR).

System plugin children can run as their own uid instead of the server's (design 30). Set LATTICE_PLUGIN_UID_BASE (1000 or higher) and optionally LATTICE_PLUGIN_UID_RANGE (default 1024), together with LATTICE_PLUGIN_RUNTIME_DIR: each plugin then runs as a uid from that range, with the same number as its gid and no supplementary groups, so it cannot read the server's 0600 files, open the probe socket, or signal the server or another plugin. The server refuses to start when the base is set without a runtime directory, when the range contains its own uid, one of its groups, or 65532 (lattice-probe), and when a file in the data directory or the master key is owned by a uid or group inside the range. With the base set, the runtime directory and every directory under it down to a plugin's generation are 0711 and the staged artifact is 0555, all owned by the server, and the runner refuses to exec an artifact when any of them has another mode or owner. A plugin's uid comes from a hash of its id, so it stays the same across restarts unless two ids collide, in which case the later one to load takes the next free uid; do not treat a plugin uid as a stable owner for files on disk.

Unset keeps plugins on the server's uid with today's staging: a 0700 working directory and artifact, and the runtime directory left at the mode it has.

The image gives /usr/local/bin/lattice-server the file capabilities cap_setuid,cap_setgid,cap_kill for this, so the container must not set no-new-privileges: with it the server starts with no capabilities and every plugin fails to start. File capabilities land on whoever execs the file, so the binary is 0750 root:lattice: the server's own identity can exec it and a plugin uid cannot, and the runner refuses to arm the drop from a capped binary that a plugin uid could exec. The image carries these capabilities whether or not the drop is on, and they narrow the server's reach only once the compose also sets cap_drop: [ALL] and adds back SETUID, SETGID and KILL (design 30 S2); until then a server compromise could use CAP_SETUID to become container root inside Docker's default capability set. scripts/check-image-isolation.sh IMAGE checks a built image for all of this.

Dashboard static serving is cache-aware: index.html, SPA fallback routes, and theme-init.js are served with Cache-Control: no-cache; content-hashed Vite assets under /assets/ are served with Cache-Control: public, max-age=31536000, immutable.

Use the compose file and deployment guide in the umbrella repository: lattice/compose/docker-compose.yml and lattice/docs/tutorials/docker-server.md.

Security Defaults

First-run username defaults to admin unless LATTICE_ADMIN_USERNAME is set. First-run password is random unless LATTICE_ADMIN_PASSWORD is set. After state contains any user, both variables remain bootstrap-only; rotate the current operator password with authenticated POST /api/auth/password.
GET /api/version returns the server build version, server commit/date, and the embedded dashboard ref exposed in the dashboard About page.
Password login sends username/password as JSON over HTTPS. Do not expose the dashboard over remote cleartext http://; use TLS, secure cookies, and HSTS in production.
Set LATTICE_REQUIRE_TOTP=1 or -require-totp to enforce TOTP for interactive operator sessions. Password/SSO login still creates a constrained session so an operator can open /settings/security, enroll, and activate TOTP; all other session-backed APIs return the stable mfa_required error until enrollment is complete. Bearer PAT automation is not an interactive session and is not gated by this policy.
Approvals are manual by default. LATTICE_APPROVAL_AUTO_RULES (or -approval-auto-rules) accepts a JSON array of opt-in auto-approve rules for trusted writers, e.g. [{"name":"linemeta-fleet","writer":"lattice-server","plugin":"singbox-linemeta","action_prefix":"apply-metadata","queue":true,"daily_cap":100}]. The first matching rule wins; queue selects approve-and-queue over approve-only, daily_cap bounds auto-approvals per UTC day (0 = uncapped), and automated decisions are audited as approval.auto_approve / approval.auto_skip. Malformed JSON is logged and ignored. The server starts fully manual. A rule that leaves plugin empty matches every plugin except substore-plan: a Sub-Store plan changes what existing identity holders receive, so a rule written for one writer or one action prefix never approves one, and only a rule that names "plugin":"substore-plan" does (the plugin id latticenet.sub-store is not the plan's plugin). Such a rule approves every Sub-Store plan any matching writer proposes, with no human reading the diff; prefer leaving these plans manual.
Passkeys (WebAuthn). Operators can register passkeys (platform authenticators such as Apple Passwords / iCloud Keychain, or roaming security keys) and sign in with them. Verification uses github.com/go-webauthn/webauthn, hand-rolling CBOR/COSE/attestation parsing is a security anti-pattern, and the server already carries vetted third-party auth deps (go-oidc). Attestation conveyance is none; discoverable (resident) keys and user verification are both required so the credential syncs and usernameless login works.
- Relying-party identity. The RP ID and origin are derived from the server's external URL (PublicURL / LATTICE_PUBLIC_URL, the same value that anchors OIDC, so there is no new config surface): RPID = host (no scheme/port), RPOrigin = scheme://host[:port]. For https://lattice.roobli.org that is RPID=lattice.roobli.org, origin https://lattice.roobli.org, so zero Apple-specific configuration. If PublicURL is unset, passkey endpoints fail closed with 503 (passkeys are not configured (server public URL unset)), exactly as the OIDC login handlers do. A prod RP ID will not match a dev origin, so local development must set LATTICE_PUBLIC_URL to the dev origin (e.g. http://localhost:5273); passkeys require the RP ID to match the browser origin, so use that dev origin directly rather than the prod host.
- Endpoints. Registration (interactive session): POST /api/security/webauthn/register/{begin,finish}. Management: GET /api/security/webauthn/credentials, POST /api/security/webauthn/credentials/{rename,delete}. Login (pre-auth, usernameless): POST /api/auth/webauthn/login/{begin,finish}. Adding or deleting a login-capable credential requires a fresh 2FA step-up grant when the account has TOTP enrolled (same step_up_grant verb used for secret reveals); rename does not.
- Login. A finished login resolves the user from the authenticator's user handle and issues the same session the password+TOTP path issues: a user-verified passkey satisfies possession + inherence, so no separate second factor is required. Challenges are short-lived (≤5 min), single-use, and IP-bound like the TOTP challenge.
- Sign counters. A zero or non-incrementing counter is not an error (synced passkeys, including Apple's, always report 0). Only a regression from a previously non-zero counter is logged as a possible-clone warning; login is never hard-failed on the counter alone.
- Apple Passwords. On Apple devices over HTTPS the passkey is saved to Apple Passwords and synced through iCloud Keychain automatically. There is no Apple-specific server setup beyond serving the dashboard over HTTPS at the RP-ID host.
Container images embed the dashboard commit pinned in dashboard.ref; update that file when intentionally rolling a new dashboard into the server image.
Management APIs are intended for localhost, WireGuard, or a hardened reverse proxy.
Agent APIs authenticate node tokens only through the Authorization: Bearer header; JSON body tokens are rejected so credentials do not enter request logs, traces, or failure captures. Successful node-token authentication updates token_last_used_at on the node record with a short write-throttle, giving operators lifecycle telemetry without turning every poll into a full state-file rewrite. Nodes may also set agent_source_allowlist with exact IPs or CIDR prefixes; the token is then accepted only from those sources. Direct deployments use the socket remote address, while CF-Connecting-IP / X-Forwarded-For are honored only when the server is explicitly configured with TrustProxy (LATTICE_TRUST_PROXY=1) and the immediate peer is a trusted proxy. By default the trusted set is loopback plus the private-use and unique-local ranges (127.0.0.0/8, ::1, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7), so a reverse proxy that reaches a containerized server through the Docker bridge gateway keeps per-client limits and audit source addresses. LATTICE_TRUSTED_PROXIES (CIDRs or addresses) replaces that set entirely, loopback included, and the server logs the effective set once at startup. X-Forwarded-For is read from the right, skipping trusted hops, so a client-supplied leftmost entry cannot choose the address; X-Real-IP is not read.
Agent HostFacts (OS, arch, cores, memory, platform, kernel, boot time) are advisory telemetry only. They are sanitized and clamped server-side and must not be used for authorization or policy decisions.
Node comments are operator-owned control-plane notes. They are returned to the dashboard, editable through POST /api/nodes/update, and never sent to the node-agent. Do not store secrets in comments and do not use comments as policy input.
Node tags are normalized on metadata writes: trimmed, deduplicated, and sorted for stable display and selector behavior. Tags can feed group smart selectors, but group membership remains a separate canonical model.
Internal IP/IPv6 fields are agent-reported informational telemetry. On many VPS providers the primary interface address is also the public address; when a dashboard needs a distinct LAN address it must treat equality with the public IP as "not separately reported" rather than as proof of private addressing.
Fleet Map GeoIP lookup defaults to the no-token ipwho.is HTTPS provider so nodes can be placed without extra setup. Set LATTICE_GEOIP_LOOKUP_URL=off to prevent the server from sending node public IPs to an external service, or set it to an internal HTTPS template containing {ip}. Automatically resolved NodeGeo is marked source=auto and manual saves are marked source=operator.
Node-agent update policies have two modes. Legacy policies may pin an explicit HTTPS binary URL and SHA-256 digest; the URL must not contain userinfo, query parameters, or fragments because the reviewed approval plan displays it. The dashboard's primary node detail UI uses the official-release mode by leaving binary_url and sha256 empty: the server resolves LATTICE_AGENT_RELEASE_REPO (default LatticeNet/lattice-node-agent), maps the node OS/arch to lattice-agent-linux-amd64 or lattice-agent-linux-arm64, reads the release SHA256SUMS (bounded to 512 KiB), and creates the reviewed update task with the concrete URL and digest. Server-controlled updates currently require Linux/systemd nodes because the reviewed task installs the binary and schedules a service restart; published darwin release artifacts remain manual-update artifacts. target_version=latest resolves to the latest stable v* GitHub release at plan time, explicitly skipping draft and prerelease alpha/beta/rc releases. Operators can still opt into a test node-agent by entering its exact prerelease version and reviewing the bound plan. /api/nodes/agent-updates/releases exposes a read-only snapshot of the current stable latest tag and published checksums for dashboard guidance; the approval plan remains authoritative because it binds the concrete URL and SHA-256 server-side. Editing or deleting a policy closes pending and approved-without-active-task update approvals for that node; active queued or leased update tasks remain in flight until their task result closes the approval. If a node already reports the current target before a pending update approval is applied, the scheduler closes the no-op approval as rejected instead of leaving stale host-mutation work in the inbox. Automatically closed update approvals include a plain-text rejection reason, including changed policy fields such as target version, SHA-256 digest, install path, or service name when they can be compared locally, so operators can re-plan without guessing which policy or node state changed. Approval and node-side exec/root-exec requirements still apply. Default install targets are treated as auto-detectable: the task script inspects the running agent parent process and systemd cgroup, then updates the currently executing lattice-agent path and restarts the detected service unit. This keeps current /opt/lattice/lattice-agent installs and older custom /opt/lattice/node-agent/lattice-agent layouts working. The task fails before downloading or installing if systemd, systemd-run, or the target unit is unavailable, so a successful task result means the binary was installed and systemd accepted the delayed restart unit. That success records last_applied_version; the live source of truth remains the next node heartbeat's reported agent_version. A plan rendered by this server also names two node-side steps, and the task script performs each only for an approval whose plan names it, so an approval planned earlier runs the script it was approved with. When the candidate's -compat-json lists sd-notify-v1, the script writes /etc/systemd/system/<service>.service.d/10-lattice-keepalive.conf (NotifyAccess=main and a runtime directory; the unit type is unchanged) so the new agent can arm its own watchdog. When it also lists health-marker-v1 and a previous binary exists, the script arms an update guard before it schedules the restart: a transient systemd timer that, 300 s later, keeps the new binary if /run/<service>/healthy holds the target version, leaves a binary that changed since the update alone, and otherwise restores the backup and restarts the service. A guard that cannot be armed restores the backup and fails the task, leaving the running agent untouched. The guard and the restart timers fire within a second (--timer-property=AccuracySec=1s) where systemd-run supports it (systemd 229 and later); an older systemd keeps its default one minute accuracy. The guard lives in /run, so a reboot before it fires cancels it and the node keeps the new binary.
Enroll and reconfigure commands install one pinned stable node-agent release: the installer is fetched at that tag and told to download the binary of the same tag. Both responses name it as agent_version. The installer always installs a binary, so a reconfigure on a node that runs another version (a newer prerelease or an older release) moves it to the pinned release, and the console says so next to the command.
Node reconfigure commands source both the canonical /opt/lattice/lattice-agent.env and legacy /opt/lattice/node-agent/agent.env before rerunning the installer. Operators can therefore reconfigure or upgrade old nodes without copying the node token again, while the installer still refuses node-id/token mismatches.
Browser Terminal uses scoped, in-memory server sessions and outbound node-agent polling. It is not inbound SSH, and the server does not store SSH keys. Operators need terminal:open; nodes must run lattice-agent with LATTICE_AGENT_ALLOW_TERMINAL=1. Session open/close events are audited, while live terminal I/O is kept in bounded process memory for the active session. The broker limits each node to four active sessions, expires unaccepted sessions after 10 minutes, expires idle sessions after 30 minutes, caps any session at 8 hours from open regardless of activity, forgives a disconnected viewer for 90 seconds before reaping, and prunes closed transcripts after 30 minutes. The 30 minute idle timer is a backstop: while a session is bridged the agent enforces real idle itself and closes the session.
MachineProfile cost/vendor/renewal data is server-only and is never sent to agents. Console/detail links are encrypted at rest and list APIs return only has_console_url / has_detail_url booleans.
API failures use a structured JSON envelope: {"error":{"code":"unauthorized","message":"invalid credentials","request_id":"req_..."}}. Every HTTP response includes X-Lattice-Request-ID; error envelopes carry the same id so dashboard, plugins and operators can correlate failures without parsing log text. Authorization denial audit events, authenticated allow audit events, login events, and agent task/event audits use the same value as correlation_id, covering node, task, token, KV/static, Worker, notification, monitor, DDNS, tunnel, and network approval changes. /api/audit remains backward-compatible as an array when called without query parameters; filtered calls return {events,total,limit,offset} and support action, decision, node_id, actor_id, token_id, scope, correlation_id, limit, and offset. limit defaults to 100 and is capped at 500 so dashboards and plugins do not accidentally fetch unbounded audit history. Personal access tokens with a non-global server_allowlist see only audit rows whose node_id is explicitly inside that allowlist; global audit rows are reserved for unrestricted operators. Server-side 5xx responses deliberately return generic public messages (internal server error or upstream service error) rather than raw provider, filesystem, database, or token-bearing error strings. Security-sensitive denials use stable business codes such as capability_denied, invalid_node_token, invalid_task_lease, and task_output_limit_exceeded. Approval workflows also expose stable conflict codes such as approval_stale for changed reviewed plans and agent_update_noop when an agent update plan is a no-op because the node already reports the target version.
nft baseline inputs are persisted per node and normalized before plan generation. The plan still becomes an approval before agent-side validation; actual firewall mutation remains behind network:apply.
NetPolicy state (/api/netpolicy, /api/netpolicy/graph) is server-validated operator intent. Writes and /api/netpolicy/plan require netpolicy:admin; list/graph require netpolicy:read; per-node PAT allowlists filter target nodes. The current nftpolicy apply path commits a dedicated inet lattice_policy output table for egress policy with a 60s rollback watchdog, control-plane selfcheck, IPv4/IPv6 control-plane domain named sets, operator-authored IPv4/IPv6 CIDR/node remotes, and egress domain remotes backed by node-filled nft sets refreshed through systemd or cron.d. Ingress deny/allow rules compose into Network Guard's single lattice_guard input render rather than a competing input table.
Approval-backed task results close failed approvals as rejected with a bounded plain-text reason, so the approvals inbox shows why a reviewed mutation did not apply instead of leaving it indefinitely approved.
Approval decisions always require network:apply; domain-specific host mutations also require their owning admin scope (dns:admin for selfdns, netpolicy:admin for nftpolicy, proxy:admin for proxycore, node:admin for agentupdate, and tunnel:admin for cftunnel) on the target node.
DNS deployment state (/api/dns/deployments) is server-owned intent for CoreDNS deployment. Writes require dns:admin on the target node, node existence is checked, Cloudflare tokens are write-only and encrypted at rest, and read views expose only has_credential. /api/dns/plan requires both dns:admin and same-node network:plan, renders a secret-free CoreDNS Corefile plus composed lattice_guard nft candidate into a pending selfdns approval, and queueing apply writes the reviewed artifacts, commits nft with rollback, manages lattice-selfdns.service, and updates deployment status from task results. /api/dns/publish reuses the existing Cloudflare DDNS provider server-side, never sends CF tokens to agents, records the last published A/AAAA values on the DNSDeployment, and is also triggered when the bound node's observed public IP changes. Service apply status (last_applied_at / last_error) is separate from hostname publication status (last_published_at / last_publish_error) so failures stay attributable to the right layer. Optional CoreDNS install is server-configured and plan-bound: the approval text contains the HTTPS URL and SHA-256, and the agent applies exactly that reviewed artifact metadata. A CoreDNS deployment may name allowed_clients (at most 64 CIDRs, no host bits, 0.0.0.0/0 and ::/0 refused), rendered as the acl plugin in every server block with one allow net per network and a final block, so other clients get REFUSED. A public deployment with a forward zone is refused unless it names them. The ACL limits who gets answers, not who reaches the port, and the plan says so; when the Corefile uses acl, the apply checks that the node's CoreDNS binary lists the plugin before it writes anything. /api/dns/measure-cert (dns:admin on the node plus monitor:admin on a token with no node allowlist) turns the DoT and DoH listeners an operator names on an external engine into server-evaluated tls checks at hostname:port, linking an authored tls check already on that endpoint instead of duplicating it, and links them to the deployment in the same write (cert_monitor_ids, which a save never writes). The deployment view lists the linked checks the caller may read (cert_checks) and the expiry the sheet shows (cert_expiry): the soonest one an enabled linked check measured, or the typed cert_not_after, with its source.
Proxy-core state currently exists as a persistence/model foundation plus a narrow server-side sing-box renderer, scoped CRUD/read APIs, a redacted reviewed plan endpoint, reviewed queue/apply, public subscription serving, audited subscription-token rotation, and baseline usage accounting. ProxyInbound.RealityPrivateKey and ProxyUser.UUID/Password/SubToken are encrypted at rest, and proto/read contracts expose only has_* presence booleans. internal/proxycore renders a canonical SHA-256-addressed sing-box vless+TCP+REALITY config from server-owned inbounds, profiles, and users; the artifact contains the REALITY private key and eligible VLESS UUIDs and must be treated as node-scoped secret material. The current JSON APIs return ProxyInboundView, ProxyUserView, and ProxyNodeProfileView shapes: global inbounds/users require unrestricted proxy:read/proxy:admin, while profiles are node-allowlist filtered. POST /api/proxy/nodes/{node_id}/plan stores a redacted review plan and binds the real rendered config SHA in the approval action; queue_apply:true re-renders the current config, rejects stale SHA, and queues a node-owned sh task that writes a same-directory candidate config, runs sing-box check -c, atomically swaps, and reloads or restarts the service. Because that queued script carries the real rendered proxy config, model.Task.Script is encrypted at rest in JSON and bbolt stores. Control-plane task views expose only script hash/size; only the authenticated owning node receives the script through the agent lease API. Future proxy APIs must not serialize the secret-bearing model structs or render artifacts directly. The public route is /sub/{slug}/{token}; both segments are regex-validated and a single-segment URL is rejected outright. It rate-limits before credential lookup, records only token SHA-256 hashes in audit metadata, and deliberately does not persist raw subscription tokens as map keys. Two things this paragraph used to claim are not true of the code and should not be relied on: the token lookup is a plain == string comparison that returns on first match, so it is neither constant-time nor a full scan (internal/store/subscription_share.go, SubscriptionShareByToken), and nothing detects duplicate tokens, so there is no fail-closed behaviour there. When no format is given the share's own DefaultFormat is used rather than a fixed default. The accepted values are format=base64, format=plain, format=sing-box (application/json client outbounds), and format=clash / format=clash-meta (text/yaml Mihomo proxies: list) for the supported VLESS+REALITY+TCP path. format is not the whole query surface: the route also accepts target (with platform as an alias, resolved against a 14-client whitelist), includeUnsupportedProxy, prettyYaml, and noFlow. Every one of them enters the cache key, which is why the whitelist is bounded. These bodies are derived from a secret-free VLESSRealityEndpoint projection; Clash/Mihomo YAML is emitted by a fixed-shape writer with quoted scalars, so no YAML dependency is introduced. ProxyInbound.Fingerprint is accepted only as a constrained safe token and is subscription metadata, not a secret. POST /api/proxy/users/rotate-sub-token returns the new subscription URL/path only in the explicit rotate response and uses LATTICE_PUBLIC_URL when configured instead of reflecting request Host. POST /api/agent/proxy-usage accepts low-trust node usage snapshots only through bearer node-token auth, filters counters to users eligible for the node's profile, treats the first snapshot as a baseline, advances usage monotonically under a dedicated mutex, and rejects malformed/negative input. GET /api/proxy/usage returns only secret-free counters/status for the dashboard.
Agent monitor results: POST /api/agent/monitor-result takes one result ({"node_id": ..., "result": {...}}) and POST /api/agent/monitor-results takes a batch ({"node_id": ..., "results": [...]}, 1 to 500 entries). Both need the node's bearer token and answer {"ok": true, "accepted": n, "duplicates": n, "dropped": [{"index", "monitor_id", "reason"}]}. A result is admitted only for an existing, enabled, non-tls monitor assigned to the posting node, with the node id taken from the token (a result that names another node is dropped as wrong_node) and the error text capped at 512 bytes, with the user info, query and fragment of every URL in it removed. An icmp result may carry sent (1 to 10), lost and rtts_ms (one per echo that came back); inconsistent packets are invalid, and the server derives success (an echo came back) and latency_ms (the median RTT) from them. Other kinds' packet fields are cleared. A result with a reason (a code such as icmp_not_permitted, destination_refused or agent_too_old) measured nothing: it is stored and shown, counts as neither a success nor a failure in rollups and streaks, and makes the path read unknown with that reason. It must carry the agent's own stamp, no more than 24 hours behind and no more than a minute ahead of the server's clock: a result without one is invalid and one outside that window is out_of_window. The server never restamps a result, because the stamp is what makes a result sent again a duplicate. A dropped reason (unknown_monitor, server_evaluated, disabled, not_assigned, out_of_window, invalid, wrong_node) is final, so the agent drops that result. The server logs drops at most once per node and reason every 15 minutes, with the stamp's offset for out_of_window, so a broken agent clock shows in the log. Results are judged in array order, so a buffered backlog goes oldest first. A result the pair already holds at the same instant counts as a duplicate, so a batch sent again after a lost response stores nothing twice and pages nothing twice, and on a 5xx nothing was stored. A page is decided after the write commits and waits in memory for the next sweep, so a crash in that window loses the page rather than sending it twice. An older server answers 404 on the batch path, which tells the agent to post one result per request.
NodeGeo state (GET/POST /api/nodes/geo) is operator-owned display metadata for the Fleet Map. Writes require node:admin on the target node, reads require node:read and are per-node allowlist-filtered, coordinates/country/ ASN are validated server-side, and update/clear actions are audited. Geo must not be used as node identity, authorization input, or nft compiler input.
PAT server allowlists are enforced against the actual node resources in request bodies, not only URL query parameters.
Node/task/monitor/DDNS/tunnel list APIs return only resources visible to the caller's scopes and server allowlist. GET /api/monitors carries each assigned node's newest result as latest (with fail_streak and since), filtered the same way, and GET /api/monitors/results returns the newest 500 results across nodes by default, or one node's with node_id, up to limit (at most 2000).
Latency probes: GET /api/monitors/latency (monitor:read) returns the probe configuration and what the control plane made of it against the fleet: each node's region, whether it is a source or target and why, the endpoint the port rule picked or why it found none, and every source to target pair. PUT /api/monitors/latency saves the configuration; it needs monitor:admin without a node allowlist, must name existing nodes and the version it was read at (a stale one gets 409), and is audited as monitor.latency.config. Until a save the defaults hold: probes on, every 60 s, from each node named cd-hs-sh to each node whose country is set and is not CN. The default source is found by name at every plan, so a node renamed to cd-hs-sh becomes a source within one sweep; a saved configuration names its sources by node id and ends that. The control plane turns the configuration into monitors marked managed_by: "latency", one per target and kind (mon_lat_<target> for tcp, mon_lati_<target> for icmp, mon_latb_<target> for the beacon) and assigned to the sources that use that kind. method (tcp, icmp, beacon or auto; empty reads as tcp) and source_methods (per-source pins) choose the kind: auto takes icmp where the source's agent runs it and the target has a public IPv4, else tcp; a pin never falls back, and a pair whose source cannot run its kind reads unknown with the reason (agent_too_old, icmp_not_permitted). A check of a kind no source uses any more stays, paused, with its history, which agents run like any other monitor; it re-plans every two minutes and after each save, writes the state file only when a generated monitor changed, and audits that change as monitor.latency.sync. A probe dials only a TCP line port the target already serves publicly, never a loopback listener, port 22, an sshd port or a knock-gated port, and only at the node's own public IPv4 or the provider edge it declares, never at an address or host name a line publishes. An edge given as a name is resolved by the control plane at each sweep and dialled by address, and only when every address it has is public, so a node cannot point the source's probe into the source's own network (the full rule is in internal/server/latency_probes.go and latency_edges.go). Generated monitors cannot be deleted through /api/monitors/delete (409) and their failures do not page. GET /api/monitors/latency/rollups returns p50 and p95 handshake time and loss for every pair over 1h (raw rows), 24h (five-minute rollups) and 7d (hourly rollups), with samples beside expected so a gap reads as unknown and percentiles and loss absent when nothing was heard. GET /api/monitors/latency/series?source=&target=&window= returns one pair's buckets (1 min, 5 min or 1 h), empty ones included. Both reads are narrowed to pairs whose two nodes the caller may read, and no probe result is ever audited. The series route stays for one release beside GET /api/monitors/series, which replaces it.
Monitoring paths (design 29): a path is one source probing one target, a (monitor, source node) pair. GET /api/monitors/paths?window= returns the matrix of every path the caller may read (both its source and its target node, and the check itself for an operator's check): sources, targets, and per path the state now (up, down after two failures in a row, or unknown with agent_offline, source_paused, not_probeable, no_data, agent_too_old, icmp_not_permitted, icmp_filtered (every echo lost on a path that has never had one back; once one has, all-loss runs read down and page) or a result's own reason), p50, p95, jitter (mean jump between consecutive successful runs), loss, samples beside expected, the same figures one window earlier (prev) and the newest run's stamp. Windows are 1h, 6h, 24h, 3d, 7d and 14d; 1h reads raw rows, 6h and 24h five-minute rollups, and 3d and longer hourly rollups, and each window's figures are cached until its next bucket closes. A tls check's path has the control plane as its source, an empty node id. GET /api/monitors/series returns one path's buckets, every step present (a gap has n 0 and expected above 0): ?source=&target_node=&window= joins every generated check from that source to that node by time and marks method_changes; ?monitor=&source=&window= reads one check. The step is the smallest multiple of the finest covering tier's resolution that keeps the series within 720 points and is no shorter than the interval: 1 h at one minute, 24 h at five, 3 d at ten, 7 d at fifteen, 14 d at thirty (hourly when the fine retention is under the window, or while the five-minute tier is still filling and the hourly one holds the older part). Both need monitor:read. Buckets carry sent and lost, counting probes (every echo of an icmp run, one per tcp or http run), and loss is lost / sent (fail / n in buckets written before the counts existed).
Checks (design 29 M2a, wire contract in design 29 D11): POST /api/monitors creates tcp, http, icmp and tls checks. icmp and tcp may name a node in target_node instead of a target; the server resolves it each time it hands the check to an agent (icmp: the node's public IPv4; tcp: its public line port by the latency port rule) and refuses the create when it cannot, and the caller needs monitor:read on that node. A literal target must be public: loopback, link-local, metadata, multicast, private, carrier-grade NAT, unspecified and broadcast addresses and localhost are refused. icmp takes an IPv4 literal or a node, packets 1 to 10 (default 5, 200 ms apart), a run that ends before the next one, and at most 2 packets a second per source and target across every enabled icmp check, generated ones included. http takes an absolute http or https URL without user info, expect_status (codes and ranges, default 200-399, stored canonical) and up to 8 headers; a header with secret must travel over https, its value is encrypted at rest, returned by no read (reads show {"name", "secret": true, "set": true}), named but never valued in audit records, sent only to the agents assigned the check, and bound to secrets_origin. GET /api/agent/monitors hands an agent only the kinds it reports in probe_methods on hello and heartbeat (an agent that reports none runs tcp and http), with node targets resolved; node reads carry the effective probe_methods and probe_refusals.
Measurement beacon (design 29 D3, wire contract in design 29 D12, wire format in the SDK's beacon package): an authenticated UDP responder an agent opens only while an applied node.beacon plan enables it. POST /api/nodes/beacon/plan (monitor:admin, node:admin and network:plan on the node, netguard:admin too when NetGuard manages it, and monitor:read on every source the plan names) takes node_id, action (enable, disable, change) and optional port (1024 to 65535; by default a random port in 40000 to 59999 assigned once per node), bind (an IPv4 literal; by default the node's primary interface address, so a NAT node binds its private address) and endpoint (the advertised IPv4 literal and port, a NAT mapping allowed; host names, loopback, link-local, multicast and 240.0.0.0/4 refused). It answers {approval, plan, findings}: the approval (plugin beacon, action node.beacon) carries the plan document, whose hash covers the settings, the limits, the source set (the path set's auto and beacon sources running the beacon client, and the sources of operator beacon checks) and, under NetGuard, the node's whole ruleset with the companion rules (UDP to the port from each source's public and WireGuard /32 and its observed egress, notrack in both directions). Findings refuse a port a sing-box inbound, sshd, a knock sequence or WireGuard holds and a bind on the WireGuard address, and warn on a mapped endpoint, no sources and a node NetGuard does not manage. Deciding it renders the document again and answers 409 stale when the bytes differ, needs the same scopes as planning plus network:apply, and, with a companion, must queue its apply task. The server mints the node's 32-byte key on the first enable, keeps it encrypted at rest, and hands each target its table and each source only its own path key per target in GET /api/agent/config (beacon); a source the plan did not approve reads firewall_pending (beacon_source_pending without NetGuard) until a change plan applies. POST /api/nodes/beacon/rotate (monitor:admin and node:admin, audited as beacon.rotate) moves the key to a new epoch; a source leaving the table does too. GET /api/nodes/beacon?node_id= (monitor:read) answers the beacon's state, applied settings, epoch, last authenticated request, the agent's report, the reachability verdict of the check one approved source runs after an apply (mon_beaconr_<node>), the exposure, firewall_pending and each source; no read and no audit record ever carries a key. Beacon checks (type: "beacon") name target_node only, resolve to the endpoint, and share icmp's packet rules and budget. The path set takes beacon as a method, and auto prefers the beacon for approved sources of a listening target, then icmp unless that pair reads icmp_filtered (then tcp, and it stays there), then tcp; generated beacon checks are mon_latb_<target>.
Check definitions (design 32 D1): one check with a source set and a target set. A definition holds a method (tcp, icmp, beacon, http or tls), sources and targets as selectors ({nodes, groups, all, exclude}, design 32 D5, resolved through the group resolver every time, so a node that joins a named group joins the check), addresses for external targets, and the fields every monitor it expands into carries. The control plane expands it into monitors marked managed_by: "check:<definition id>", one per (target, method), with opaque ids hashed from the method and the normalized target, so removing one target never hands another its history and a method change starts new series. Expanded monitors behave as operator checks (matrix rows keyed <source>~@<monitor id> and never marked managed, paging by the definition's policy, visibility, beacon sources, node targets resolved by the path set's port rule and sticky endpoint) except that a direct edit or delete answers 409 naming the definition. A target that leaves the set keeps its monitor, paused, with its history; a target node removed from the fleet loses it. GET /api/monitors/definitions (monitor:read) lists the definitions the caller may read with their path and paused-target counts, beside fleet_mesh, the path set listed as the built-in check named Fleet mesh with its own editor; ?id= returns one with its monitors and their stored runs. POST /api/monitors/definitions creates, POST /api/monitors/definitions/update edits with the version read (409 when stale; header edits are keep, set or delete as on a check), and POST /api/monitors/definitions/preview returns what the same body would write and measure: targets with the reason nothing can be dialled, every pair with the reason it would read unknown (including packet_budget and beacon_source_pending), and the monitors created, rewritten, paused and deleted. POST /api/monitors/definitions/delete and /forget name the monitors the operator confirmed (409 when they no longer match) and delete them with their history. A group or whole-fleet selector needs a token with no node allowlist; a restricted token may name explicit nodes it can read (sources it can administer), and sees only such definitions. Secret headers need explicit source nodes and stay bound to the origins they were saved for. Definitions resync on save, on the two-minute sweep, after a beacon approval, and after group edits, node edits and node removal; every changed definition is written in one state write and audited as monitor.check.sync. Each sync shares the per-source packet budget: pairs that already run keep their share and a pair that would push a source over it is paused, reading packet_budget, until there is room. A group a definition names cannot be deleted (409).
GET /api/monitors/settings (monitor:read, no node allowlist, since the figures cover the fleet) returns the history setting (fine_retention_days, 3 to 14, 7 by default; the hourly tier is fixed at 35 days and raw rows at 1440 per path), the paths recording history, and measured_bytes (bbolt page statistics of the monitor buckets) beside projected_bytes (every current path filling every tier at the setting). POST /api/monitors/settings/update saves it with the version it was read at (409 when stale), needs monitor:admin without a node allowlist, and is audited as monitor.settings.update; a lower retention starts the pruning sweep at once. The sweep otherwise runs hourly and deletes, for every pair including paused ones, the buckets and rows behind each tier's line in write transactions of at most 2000 deletes. bbolt reuses freed pages but never shrinks its file.
POST /api/monitors/update (monitor:admin on every source it names) edits an operator's check with the version it was read at (409 when stale) and only the fields it sends: name, interval_sec (10 s floor), timeout_sec, threshold_days (tls), alert (down or off), alert_after (failures in a row that page and read as down, 1 to 20, default 2; also accepted on create), enabled, assign_all, node_ids, paused_node_ids, packets (icmp), expect_status (http) and headers (http) as a list of {name, op, value, secret} with op keep (refused when the header is gone), set (needs a value) or delete; a header the list does not name is kept, so an edit that echoes a read neither wipes nor leaks a secret. target_node cannot change. A paused source keeps its history, is not handed the check and has its results refused as not_assigned. Changing type or target is refused with 400: that is a new check. Generated checks answer 409, as a delete does. Edits are audited as monitor.update. A check whose alert policy is off, like a generated latency check, and a disabled check open no incident and show no pending one; the incident sweep closes what such a check had open.
Control-plane task views expose script hash and byte size, not the full script body or agent-only lease credential.
Task read and run permissions are split: task:read lists task metadata and results, while task:run queues remote execution.
Task creation is validated at the control plane: interpreter allowlist (sh, bash, python3, node), timeout 1-600 seconds, output cap 1-256 KiB, and script body up to 64 KiB.
Agent task leases expose only execution fields, script body, limits, and lease_id; actor/token metadata and full target lists stay control-plane only.
Task result writes require the node token, the matching leased node, and the per-lease lease_id; stale, missing, cross-node, or output-over-limit results are rejected before storage.
Accepted task results are stored and returned through the control plane without the agent-only lease_id.
Operator-configured outbound webhooks and OIDC IdP calls use guarded HTTP clients that reject loopback, private, link-local, metadata, CGNAT, and documentation ranges. This applies to OIDC discovery, JWKS fetches, and token exchange, so an admin-supplied issuer cannot make the server reach internal services.
Plugin manifests require stable lowercase ids and non-empty duplicate-free capability lists before any plugin can be trusted by the control plane. Host-risk/system plugins can be verified with an operator trust policy: trusted publisher Ed25519 keys, artifact digest_sha256, and standard-base64 signature_ed25519 over the canonical Lattice plugin signing payload. Plugin installation/loading code should use the strict verifier path: decode manifest JSON with unknown fields rejected, verify artifact digest, then verify publisher signature when host-risk capabilities are present. Operators and dashboards can preflight a candidate plugin without installing it through POST /api/plugins/verify with scope plugin:verify. The endpoint accepts a manifest object and artifact_base64, applies the server-side trust policy, returns the manifest with signature_ed25519 stripped plus capability risk labels, and never writes the artifact to disk or registers it in /api/plugins.
Plugin host APIs are brokered through internal/plugin.Broker, which is built from a verified plugin.Loaded entry and checks the manifest's declared capabilities on every host call. The broker currently defines guarded facades for KV (kv:read/kv:write), notifications (notify:send), outbound HTTP (http:egress), plugin logs (log:write), and host-call audit events. It is a contract and enforcement point only; plugin execution/installation lifecycle is intentionally separate. The server-owned adapter (plugin_host.go) wires those facades to the real store, notification dispatcher, guarded outbound HTTP client, logger, and audit sink. Broker capability allow/deny decisions are written as plugin.host.* audit events with the plugin id, capability, decision, and correlation id. Plugin HTTP request and response bodies are capped at 256 KiB, and outbound HTTP uses the same SSRF/egress guard as webhooks.

Storage

The default server store is still the encrypted JSON state file plus the append-only hash-chained audit WAL (<state>.audit-wal) and a local sidecar head anchor (<state>.audit-anchor) that detects end-truncation on open and through /api/audit/verify. Configure LATTICE_AUDIT_HEAD_WEBHOOK_URL to periodically ship the verified anchored head to an HTTPS off-box archive.
internal/store.BoltStateStore is the Phase C bbolt foundation. It can import and export the full State, stores each top-level collection in its own bucket, reuses the existing AES-256-GCM secret encryption boundary, and now has record-level APIs for nodes, KV entries, audit events, static objects, Worker scripts, plugin lifecycle records, approvals, tasks, task results, monitors, monitor results, tunnels, users, PAT tokens, sessions, TOTP challenges, DDNS profiles, notification channels, machine profiles, nft inputs, DNS deployments, net policies, OIDC providers, OIDC identities, and OIDC auth states.
The local ops CLI can migrate the encrypted JSON file to bbolt and export bbolt back to encrypted JSON:
```
lattice-server migrate json-to-bolt \
  -json /var/lib/lattice/state.json \
  -bolt /var/lib/lattice/state.db

lattice-server migrate bolt-to-json \
  -bolt /var/lib/lattice/state.db \
  -json /var/lib/lattice/state.rollback.json
```
The CLI requires explicit -json and -bolt paths, refuses to overwrite targets unless -overwrite is set, reuses the normal master key source, and will not generate a new key during migration. If the key is not under the JSON state directory as master.key, pass -master-key-file or set LATTICE_MASTER_KEY_FILE.
The runtime path now has an opt-in bbolt hot-store sidecar for the highest churn domains:
```
LATTICE_RUNTIME_BOLT_HOT_STORE=/var/lib/lattice/state-hot.db lattice-server
```
When enabled, startup imports/merges existing JSON hot records into the sidecar, then audit events, interactive sessions, proxy users, proxy node profiles, and proxy usage snapshots are written at record level in bbolt instead of forcing a whole encrypted JSON rewrite. Monitor results live there too: one row per result and one latest record per monitor and node (with the failure streak the alert hold reads), 1440 rows per pair, and any JSON history is migrated once on the first enable. Each result is also counted, in the same transaction, into a five-minute bucket kept for the fine retention (7 days by default) and an hourly bucket kept 35 days for its pair (count, failures, latency extremes, a histogram and the jitter sum), which answer the windows the rows no longer reach. History reads run in a bolt read transaction outside the store lock. Without the sidecar the JSON file keeps 120 rows per pair and is rewritten only when a pair changes state, reaches its second failure in a row, or has not been written for five minutes. The in-memory read model remains unchanged and /readyz verifies the sidecar. The default remains the JSON state file so operators can canary the cutover per deployment.
The full runtime store is not bbolt-only yet. The next storage slice should add backup/restore drills and then migrate the remaining low-churn control plane collections behind an error-returning store interface.

Example plugin trust policy JSON:

{
  "allow_unsigned_host_risk": false,
  "trusted_publishers": {
    "latticenet": "standard-base64-of-32-byte-ed25519-public-key"
  }
}

Fail-closed by default: omitting allow_unsigned_host_risk (or setting it false) requires a trusted-publisher Ed25519 signature for every host-risk plugin. Set it true only for local development on a host you fully control.

Example plugin preflight request:

POST /api/plugins/verify
Authorization: Bearer <token with plugin:verify>
Content-Type: application/json

{
  "manifest": {
    "id": "latticenet.nft",
    "name": "nft Guard",
    "type": "system",
    "version": "0.1.0",
    "entrypoint": "system-go/latticenet-nft",
    "capabilities": ["network:plan"],
    "publisher": "latticenet",
    "digest_sha256": "hex-sha256-of-artifact",
    "signature_ed25519": "standard-base64-of-64-byte-ed25519-signature"
  },
  "artifact_base64": "standard-base64-artifact-bytes"
}

Successful response:

{
  "trusted": true,
  "artifact_sha256": "hex-sha256-of-artifact",
  "manifest": {
    "id": "latticenet.nft",
    "name": "nft Guard",
    "type": "system",
    "version": "0.1.0",
    "entrypoint": "system-go/latticenet-nft",
    "capabilities": ["network:plan"],
    "publisher": "latticenet",
    "digest_sha256": "hex-sha256-of-artifact"
  },
  "capabilities": [
    {"name": "network:plan", "risk": "host"}
  ]
}

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
.omx/state		.omx/state
cmd		cmd
internal		internal
scripts		scripts
tools		tools
.dockerignore		.dockerignore
.gitignore		.gitignore
DELETE-AUDIT-2026-08.md		DELETE-AUDIT-2026-08.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
SECURITY-REVIEW-CORE-SCOPES-2026-08.md		SECURITY-REVIEW-CORE-SCOPES-2026-08.md
dashboard.ref		dashboard.ref
docker-entrypoint.sh		docker-entrypoint.sh
go.mod		go.mod
go.sum		go.sum
refs_test.go		refs_test.go
sdk.ref		sdk.ref

Navigation Menu

Repository navigation

About

Repository files navigation

lattice-server

Run Locally

Build

Verify

Docker

Security Defaults

Storage

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

lattice-server

Run Locally

Build

Verify

Docker

Security Defaults

Storage

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages