Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@
# FANOUT_ROLLUP_INTERVAL=

# FANOUT_ROLLUP_SKIP_TO_LATEST; boolean
# RollupSkipToLatest, set once at boot, advances the service and edge rollup watermarks to the current max ingested timestamp. Existing history is not aggregated into those analytical rollups. Completed-batch trace caches still rebuild all retained history in bounded passes; uncached files remain immediately queryable. Useful for large pre-seeded datasets (benchmarks, restores); off in normal operation.
# Set once at boot, this advances the service and edge rollup watermarks to the current max ingested timestamp. Existing history is not aggregated into those analytical rollups. Completed-batch trace caches still rebuild all retained history in bounded passes; uncached files remain immediately queryable. Useful for large pre-seeded datasets (benchmarks, restores); off in normal operation.
# Default: false
# FANOUT_ROLLUP_SKIP_TO_LATEST=

Expand Down
7 changes: 2 additions & 5 deletions cmd/fanout-docgen/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ func renderAlertEnv(vars []field) []byte {
b.WriteString("---\n")
b.WriteString(`title: "Alert expression variables"` + "\n")
b.WriteString(`description: "Every value an alert rule expression can read, and what it means."` + "\n")
b.WriteString(`summary: "The variables available to a CEL alert rule, generated from the evaluation environment itself."` + "\n")
b.WriteString(`summary: "The variables available to a CEL alert rule."` + "\n")
b.WriteString("read_when:\n")
b.WriteString(` - "You are writing an alert rule and need the variable names."` + "\n")
b.WriteString(` - "A rule compiles but never fires, and you want to check what it is reading."` + "\n")
Expand All @@ -243,9 +243,7 @@ func renderAlertEnv(vars []field) []byte {
b.WriteString("{/* Generated by cmd/fanout-docgen from internal/alert. Edit the generator, not this page. */}\n\n")

b.WriteString("An alert rule's `expression` is [CEL](https://cel.dev) evaluated\n")
b.WriteString("against one service's rollup for the current window. These are the values it\n")
b.WriteString("can read — the names below match the variables declared and bound by the\n")
b.WriteString("evaluator.\n\n")
b.WriteString("against one service's rollup for the current window. It can read the following variables.\n\n")
b.WriteString("Nonempty expressions supplied when creating or replacing rules through the\n")
b.WriteString("API are type-checked against these declared variables. Unknown variables,\n")
b.WriteString("type mistakes, and non-boolean results are rejected before the rule is saved.\n\n")
Expand Down Expand Up @@ -556,7 +554,6 @@ func renderPage(name string, page groupPage, fields []field) []byte {
b.WriteString("Every setting below is read once at startup and requires a restart to change.\n")
b.WriteString("Built-in defaults are overridden by nonempty environment values.\n")
b.WriteString("Unknown `FANOUT_` names fail startup; platform-injected service names are tolerated.\n")
b.WriteString("Custom names use `FANOUT_` plus a clear name, adding the area only when it clarifies.\n")
b.WriteString("Vendor credentials and endpoints keep the vendor's documented names.\n\n")
if name == "ai" {
b.WriteString("Fanout also recognizes `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`,\n")
Expand Down
8 changes: 4 additions & 4 deletions cmd/fanout-docgen/mcptools.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ func renderMCPTools() ([]byte, error) {
b.WriteString("title: \"MCP tools\"\n")
b.WriteString("description: \"Every tool a Fanout instance exposes over MCP, with its inputs and whether it changes anything.\"\n")
fmt.Fprintf(&b,
"summary: \"%s served at /mcp, each with its inputs and mutation semantics, taken from the server's Apps-enabled tools/list answer.\"\n",
"summary: \"%s served at /mcp, each with its inputs and mutation semantics.\"\n",
count(len(tools), "tool"),
)
b.WriteString("read_when:\n")
Expand All @@ -55,7 +55,7 @@ func renderMCPTools() ([]byte, error) {
b.WriteString(" this page. */}\n\n")

fmt.Fprintf(&b,
"A Fanout instance serves MCP at `/mcp`. %s, listed below exactly as the\nserver reports them to an Apps-enabled client. Ordinary clients receive the %d model-visible tools; the three app-only helpers require Apps negotiation and are never offered to the model.\n\n",
"A Fanout instance serves MCP at `/mcp`. %s are available to an Apps-enabled client. Ordinary clients receive the %d model-visible tools; the three app-only helpers require Apps negotiation and are never offered to the model.\n\n",
count(len(tools), "tool"), modelVisible,
)

Expand Down Expand Up @@ -124,7 +124,7 @@ func renderMCPTools() ([]byte, error) {

for _, tool := range tools {
fmt.Fprintf(&b, "## %s\n\n", tool.Name)
fmt.Fprintf(&b, "**%s** — %s\n\n", mdx(tool.Title), mdx(tool.Description))
fmt.Fprintf(&b, "**%s**: %s\n\n", mdx(tool.Title), mdx(tool.Description))
fmt.Fprintf(&b, "%s\n\n", effectProse(tool))
if tool.AppOnly {
b.WriteString("**Visibility:** app only; no resource attachment.\n\n")
Expand Down Expand Up @@ -172,7 +172,7 @@ func renderMCPTools() ([]byte, error) {
b.WriteString("\n")
}
b.WriteString("## Authentication\n\n")
b.WriteString("MCP uses OAuth tokens bound to the instance’s resource URI.\n[Connect an agent](/docs/guides/mcp#scopes) documents protocol\nrequirements and delegated scopes. Dashboard tools operate on the authenticated\nuser’s own dashboards and require the account capability\n`dashboards:manage-own`; see [roles](/docs/reference/roles).\n\n## Other interfaces\n\nUse OTLP to write telemetry. Configuration uses environment variables, and user\nand alert management use the [HTTP API](/docs/reference/api).\n")
b.WriteString("MCP uses OAuth tokens bound to the instance's resource URI.\n[Connect an agent](/docs/guides/mcp#scopes) documents protocol\nrequirements and delegated scopes. Dashboard tools operate on the authenticated\nuser's own dashboards and require the account capability\n`dashboards:manage-own`; see [roles](/docs/reference/roles).\n\n## Other interfaces\n\nUse OTLP to write telemetry. Configuration uses environment variables, and user\nand alert management use the [HTTP API](/docs/reference/api).\n")

toolCount = tools
return []byte(b.String()), nil
Expand Down
12 changes: 5 additions & 7 deletions cmd/fanout-docgen/roles.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ func renderRoles() ([]byte, error) {
b.WriteString("---\n")
b.WriteString("title: \"Roles\"\n")
b.WriteString("description: \"The three Fanout roles and the capabilities each one carries.\"\n")
b.WriteString("summary: \"viewer, operator and admin, and exactly what each is permitted to do — taken from the authorization table the server enforces.\"\n")
b.WriteString("summary: \"viewer, operator and admin, and what each is permitted to do.\"\n")
b.WriteString("read_when:\n")
b.WriteString(" - \"You are deciding which role to give someone.\"\n")
b.WriteString(" - \"You are mapping identity-provider groups onto Fanout roles.\"\n")
Expand All @@ -83,7 +83,7 @@ func renderRoles() ([]byte, error) {

b.WriteString("{/* Generated by cmd/fanout-docgen from internal/api. Edit the generator, not this page. */}\n\n")

b.WriteString("Each user has one fixed role: `viewer`, `operator` or `admin`. The table is\ngenerated from the capability map enforced by the authorization middleware.\n\n")
b.WriteString("Each user has one fixed role: `viewer`, `operator` or `admin`.\n\n")

b.WriteString("| Capability |")
for _, role := range roleOrder {
Expand All @@ -109,7 +109,7 @@ func renderRoles() ([]byte, error) {

b.WriteString("\n## What each capability permits\n\n")
for _, capability := range caps {
fmt.Fprintf(&b, "- `%s` — %s\n", capability, capabilityNotes[capability])
fmt.Fprintf(&b, "- `%s`: %s\n", capability, capabilityNotes[capability])
}

b.WriteString("\n## Who assigns roles\n\n")
Expand All @@ -120,10 +120,8 @@ func renderRoles() ([]byte, error) {
b.WriteString("[Authentication modes](/docs/concepts/authentication) covers the details,\n")
b.WriteString("including why the last active administrator is never demoted.\n\n")

b.WriteString(":::caution[A role is not a data boundary]\n")
b.WriteString("Roles gate capabilities, not rows. Every role that can read telemetry can read\n")
b.WriteString("all of it — namespaces scope queries, not access. An instance is one trust\n")
b.WriteString("domain.\n")
b.WriteString(":::caution[Telemetry access]\n")
b.WriteString("See [namespace access limits](/docs/reference/capabilities#operating-limits).\n")
b.WriteString(":::\n")

return []byte(b.String()), nil
Expand Down
22 changes: 11 additions & 11 deletions cmd/fanout-docgen/routes.go
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ func renderRoutes(routes []api.RouteDoc) []byte {
b.WriteString("---\n")
b.WriteString("title: \"HTTP routes\"\n")
b.WriteString("description: \"Every HTTP route the server registers and what guards each one. Mostly the browser client's backend rather than a public API.\"\n")
b.WriteString("summary: \"Every route the server registers — application, operational, protocol and the SPA catch-all — with the authorization requirement for each, taken from the middleware that enforces them.\"\n")
b.WriteString("summary: \"Application, operational and protocol routes and the SPA catch-all, with the authorization requirement for each.\"\n")
b.WriteString("read_when:\n")
b.WriteString(" - \"You are auditing what an instance exposes and what guards each route.\"\n")
b.WriteString(" - \"A request came back 401 or 403 and you want to know which capability it wanted.\"\n")
Expand All @@ -251,13 +251,13 @@ func renderRoutes(routes []api.RouteDoc) []byte {

b.WriteString("## Conditional routes\n\nThe table includes every route the binary can register. These routes depend on\nconfiguration:\n\n")

b.WriteString("- `/debug/pprof/*` — only when `FANOUT_PPROF_ENABLED` is true, which is\n")
b.WriteString("- `/debug/pprof/*`: only when `FANOUT_PPROF_ENABLED` is true, which is\n")
b.WriteString(" **not** the default. Enabling it also turns on mutex and block sampling.\n")
b.WriteString("- `/mcp`, `/api/mcp`, `/api/auth/oauth/*`, `/oauth/*` and `/.well-known/*` — only when\n")
b.WriteString("- `/mcp`, `/api/mcp`, `/api/auth/oauth/*`, `/oauth/*` and `/.well-known/*`: only when\n")
b.WriteString(" `FANOUT_MCP_ENABLED` is true, which **is** the default.\n")
b.WriteString("- `/api/agent/*` — only when `FANOUT_AI_PROVIDER` selects `anthropic` or\n")
b.WriteString("- `/api/agent/*`: only when `FANOUT_AI_PROVIDER` selects `anthropic` or\n")
b.WriteString(" `openai`, with that provider's required key. An empty provider disables chat.\n")
b.WriteString("- `/api/auth/oidc/start` and `/api/auth/oidc/callback` — only in\n")
b.WriteString("- `/api/auth/oidc/start` and `/api/auth/oidc/callback`: only in\n")
b.WriteString(" `FANOUT_AUTH_MODE=oidc`.\n\n")
b.WriteString("Everything else is always registered. A path shown with `:name` or `*` is the\n")
b.WriteString("pattern Echo matches on, not a literal URL.\n")
Expand All @@ -267,7 +267,7 @@ func renderRoutes(routes []api.RouteDoc) []byte {

b.WriteString("## Errors\n\nApplication API errors, including unknown routes and wrong methods, return objects shaped like\n`{\"code\":\"bad_request\",\"message\":\"Safe explanation\"}`. Codes are stable\nsnake_case identifiers; clients branch on `code` rather than message text.\nInvalid dashboard or panel specs also include `problems` with field paths,\nmessages and optional hints. Every error response uses `Cache-Control: no-store`;\nHEAD errors have no body. Server failures return a short, safe message; the\nunderlying error is logged and never returned.\n\nDefault codes by HTTP status:\n\n| Status | Code |\n|---|---|\n| 400 | `bad_request` |\n| 401 | `unauthorized` |\n| 403 | `forbidden` |\n| 404 | `not_found` |\n| 405 | `method_not_allowed` |\n| 409 | `conflict` |\n| 413 | `payload_too_large` |\n| 415 | `unsupported_media_type` |\n| 422 | `unprocessable` |\n| 429 | `rate_limited` |\n| 500 | `internal` |\n| 502 | `bad_gateway` |\n| 503 | `unavailable` |\n| 504 | `timeout` |\n\nOther 4xx statuses use `bad_request`; other 5xx statuses use `internal`.\nSpecific conditions use `already_current`, `dashboard_version_not_found`,\n`dashboard_version_conflict`, `invalid_spec` and `last_active_administrator`.\nTool failures carry `error: {code, message}` inside the tool result.\nPanel execution results keep their own status and error fields.\nOAuth, OTLP/gRPC, MCP JSON-RPC and AG-UI run errors keep their protocol spelling.\nOperational responses at `/healthz`, `/readyz` and `/metrics` keep their bodies.\n\n")

b.WriteString("The **Requires** column comes from the authorization middleware for each method\nand path. [Roles](/docs/reference/roles) maps capabilities to roles.\n\n")
b.WriteString("[Roles](/docs/reference/roles) maps capabilities to roles.\n\n")

b.WriteString("| Method | Path | Requires |\n")
b.WriteString("|---|---|---|\n")
Expand All @@ -276,13 +276,13 @@ func renderRoutes(routes []api.RouteDoc) []byte {
}

b.WriteString("\n## What the requirements mean\n\n")
b.WriteString("- **none** — reachable without a credential.\n")
b.WriteString("- **none**: reachable without a credential.\n")

b.WriteString("- **any signed-in user** — a session, but no particular capability.\n")
b.WriteString("- **protocol handshake** — the OAuth and MCP endpoints, which authenticate\n")
b.WriteString("- **any signed-in user**: a session, but no particular capability.\n")
b.WriteString("- **protocol handshake**: the OAuth and MCP endpoints, which authenticate\n")
b.WriteString(" as part of their own protocol rather than through the middleware.\n")
b.WriteString("- **a named capability** — the caller's role must carry it.\n")
b.WriteString("- **or a service credential** — additionally reachable without a browser\n")
b.WriteString("- **a named capability**: the caller's role must carry it.\n")
b.WriteString("- **or a service credential**: additionally reachable without a browser\n")
b.WriteString(" session, which is how a scraper reaches the metrics endpoint.\n\n")
b.WriteString("`Any` registers every HTTP method with the same authorization requirement.\n`GET /*` serves the single-page application for browser paths. Unknown `/api/*`\nand OTLP paths return errors.\n")

Expand Down
2 changes: 1 addition & 1 deletion internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ type Config struct {
// MaintenanceInterval controls Parquet retention and compaction, and
// query-cache checkpointing.
MaintenanceInterval time.Duration `group:"storage" env:"FANOUT_MAINTENANCE_INTERVAL" default:"1h"`
// RollupSkipToLatest, set once at boot, advances the service and edge rollup
// Set once at boot, this advances the service and edge rollup
// watermarks to the current max ingested timestamp. Existing history is not
// aggregated into those analytical rollups. Completed-batch trace caches
// still rebuild all retained history in bounded passes;
Expand Down
4 changes: 1 addition & 3 deletions site/src/content/docs/docs/concepts/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,7 @@ the UI. Monitor the instance from outside its host.

## Deployment scope

Fanout serves one trust domain on one machine. Namespaces filter telemetry
queries; every telemetry reader can access all namespaces. Use separate
instances for mutually untrusted teams.
Fanout runs on one machine; see [trust domains and namespace access](/docs/reference/capabilities#operating-limits).

Workloads requiring horizontal ingest, replication or an object-storage tier
require a backend that provides those capabilities.
9 changes: 4 additions & 5 deletions site/src/content/docs/docs/concepts/authentication.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Authentication modes
description: Why Fanout has two sign-in modes, what each one owns, and where the boundary between them sits.
summary: Local codes versus an identity provider — who owns roles in each, and what happens on every login.
summary: Local codes versus an identity provider, who owns roles in each, and what happens on every login.
read_when:
- You are choosing between local sign-in and OIDC.
- You expect a provider group change to take effect and want to know when it does.
Expand Down Expand Up @@ -32,7 +32,7 @@ an email address.
## OIDC

An identity provider handles sign-in. Group and domain allowlists are evaluated
on every login, including for linked identities. Removing a user’s eligibility
on every login, including for linked identities. Removing a user's eligibility
blocks their next login; it does not revoke a browser session already issued.
Suspending the account in Fanout blocks access and revokes its sessions.

Expand Down Expand Up @@ -75,6 +75,5 @@ sessions. The last active administrator cannot be demoted, suspended or deleted.

## Access boundaries

Every telemetry reader can query every namespace. Namespaces filter queries;
separate trust domains require separate instances. Dashboard access is scoped
to the owner for every role, including administrators.
See [namespace access limits](/docs/reference/capabilities#operating-limits).
Dashboard access is scoped to the owner for every role, including administrators.
21 changes: 11 additions & 10 deletions site/src/content/docs/docs/concepts/storage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ status: preview
Fanout retains telemetry as ordinary Parquet files in self-contained atomic
batch directories. Persistent sidecars index traces, and DuckDB handles SQL,
broad scans, and rebuildable analytical rollups in the same process.
Application state—users, sessions, dashboards, alert rules, and agent
history—lives in SQLite.
Application state (users, sessions, dashboards, alert rules, and agent
history) lives in SQLite.

| Component | Purpose |
|---|---|
Expand Down Expand Up @@ -103,13 +103,14 @@ compiled into the binary. It downloads no extensions at runtime. Native source
builds use `scripts/with-duckdb.sh`, which checks the platform archive and uses
the matching engine headers; the ordinary `just` build and test recipes invoke
it automatically. DuckDB 2's asynchronous I/O uses its default memory-governed
read-ahead rather than an unlimited queue. Rooted service traversal uses keyed
read-ahead, which bounds the I/O queue. Rooted service traversal uses keyed
recursion with explicit hop, accumulated-node, and execution-time limits.

Offline verification decodes every Parquet column with the native engine and
separately checks physical schemas, span ordering, row counts, and trace-index
ranges. It reuses one bounded in-memory engine and creates no spill files.
Discarded hashes force decoding; they do not establish a stored checksum.
Verification computes and discards hashes to force decoding, without storing
a checksum.
Verification failures such as memory exhaustion or cancellation never authorize
quarantine.

Expand Down Expand Up @@ -158,9 +159,9 @@ active ID even before cleanup; they cannot invalidate unrelated traces.
Compaction derives the output trace contributions from complete cached inputs. If an input
is not yet cached, the output remains directly queryable until its own backfill.

The private cache stores per-batch trace parts and one
incremental candidate per trace. It does not copy every span or log timestamp
or maintain global scope indexes. Complete cached compaction replacements
Trace data in the private cache consists of per-batch trace parts and one
incremental candidate per trace. Timestamps are aggregated by batch and trace;
scope lookups use those parts and candidates. Complete cached compaction replacements
preserve candidate bounds without recomputation. Retention and incomplete
replacements recompute only affected trace IDs from active contributions.
A semantic version covers the cache schema and aggregation rules; a version
Expand All @@ -172,9 +173,9 @@ Service-version observations populate their analytical rollup on a separate
worker, so an analytical-gate wait cannot delay the completed-batch worker.
Completed-batch and analytical writers use separate gates and two connections
outside the read pool. They modify disjoint tables; maintenance holds both gates
for checkpointing. Trace reads avoid materializing every retained candidate into
a temporary relation for each request. A snapshot-local ambiguity probe keeps
unused batch-parts aggregation out of single-scope read plans.
for checkpointing. Trace reads query retained candidates directly. A
snapshot-local ambiguity probe includes batch-parts aggregation only when
the requested scope needs it.

Narrow file selections use explicit lists from footer bounds. Broad selections
use a glob with a captured-batch-ID filter, which excludes files published after
Expand Down
Loading
Loading