Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@
session links out of tracked files, commit messages, and pull request descriptions.
- Keep local agent caches, indexes, evaluation records, and browser evidence out
of both Git and the Docker build context. Git ignores do not exclude Docker inputs.
- Design proposals live in `docs/proposals/NNNN-slug.md` and decision records
in `docs/adr/NNNN-slug.md`. Agent working plans and scratch specs stay out of
the repository.

## Control database

Expand Down
10 changes: 10 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,18 @@ cmd/fanout/ process composition and the single entry point
internal/ ingest, storage, query, agent, MCP, auth, alerts
ui/host/ React browser host and shared MCP app renderer (build-time)
docs/diagrams/ archify JSON sources and exported dual-theme SVGs
docs/proposals/ numbered design proposals for significant changes
docs/adr/ numbered architecture decision records
```

## Design proposals and decisions

Write a [design proposal](docs/proposals/README.md) before building a
significant change to a capability, a public contract or a cross-cutting
design. Record a settled decision that should not be re-argued as an
[ADR](docs/adr/README.md). Agent working plans and scratch notes stay out of
the repository.

## Commits and pull requests

Write commit messages that explain why the change is needed, not only what it
Expand Down
3 changes: 2 additions & 1 deletion context7.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
"branch": "main",
"folders": ["site/src/content/docs", "docs", "README.md"],
"excludeFolders": [
"docs/specs",
"docs/proposals",
"docs/adr",
"docs/media",
"docs/diagrams"
],
Expand Down
26 changes: 26 additions & 0 deletions docs/adr/0001-environment-only-configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# 0001. Environment-only configuration

Status: Accepted
Date: 2026-10-30

## Context

Fanout runs as one binary on laptops, containers and hosted platforms. It
needs one configuration source that works the same way in each, and a way to
deliver secrets without writing them into configuration.

## Decision

Configuration comes only from the process environment, over built-in defaults.
`internal/config/config.go` is the authoritative schema. Custom names use the
`FANOUT_` prefix; vendor credentials and endpoints keep their vendor names
(`ANTHROPIC_API_KEY`, `OPENAI_BASE_URL`, …). Every secret also accepts a
`<NAME>_FILE` form. Unknown `FANOUT_` names fail startup. The binary does not
load dotenv files.

## Consequences

- One precedence rule and one place to read the schema.
- `.env.example` and the site reference are generated from the schema.
- Secrets can be mounted as files without appearing in the environment.
- A typo in a `FANOUT_` name fails fast instead of being ignored.
25 changes: 25 additions & 0 deletions docs/adr/0002-sqlite-goose-sqlc-control-state.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# 0002. SQLite with Goose and sqlc for control state

Status: Accepted
Date: 2026-10-30

## Context

Fanout needs a transactional store for users, sessions, dashboards, alerts and
settings, without adding a database server to operate.

## Decision

Control state lives in SQLite through `modernc.org/sqlite` and `database/sql`,
with sqlc-generated query bindings and timestamped Goose SQL migrations.
`internal/db/migrations` is the single schema source for both Goose and sqlc.
Published migrations are immutable and checksummed in
`internal/db/migrations_test.go`. PostgreSQL and other migration frameworks are
not used for control state.

## Consequences

- No external database to deploy.
- Schema changes always add a new migration and run `just db-gen`.
- A database with application tables must have a positive applied Goose
version before initialization; there is no schema fallback.
26 changes: 26 additions & 0 deletions docs/adr/0003-pinned-duckdb-and-batch-format-4.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# 0003. Pinned DuckDB engine and batch format 4

Status: Accepted
Date: 2026-10-30

## Context

Telemetry queries depend on exact engine behaviour: built-in functions, Parquet
VARIANT handling and nanosecond timestamps. A Go driver version does not
identify the native engine it bundles, and silent engine or format drift would
change query results.

## Decision

Fanout builds against one pinned DuckDB 2 engine (`internal/duckdb`, built with
`scripts/with-duckdb.sh`) with statically linked extensions and no dynamic
extension downloads. Telemetry is stored only as batch format 4 Parquet with
typed attribute maps. A schema change requires a new format version; there is
no legacy reader or format fallback.

## Consequences

- Query results are reproducible across platforms and releases.
- Changing the pin means verifying `SELECT version()` and reviewing every
changed built-in against the telemetry SQL blocklist.
- Data written in an unsupported format is rejected rather than reinterpreted.
26 changes: 26 additions & 0 deletions docs/adr/0004-no-legacy-paths.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# 0004. No legacy paths

Status: Accepted
Date: 2026-10-30

## Context

Keeping aliases, fallback routes, old readers or schema fallbacks next to their
replacements doubles the code to test and lets superseded behaviour linger.

## Decision

Superseded behaviour is removed in the same change that replaces it:

- Breaking naming changes update registrations, authorization classification,
clients, tests, metrics configuration and generated documentation together,
and remove superseded routes and handlers with no aliases or fallback.
- The control database has no legacy migration tracker or schema fallback.
- Telemetry storage accepts only the current batch format, with no legacy
reader or format fallback.

## Consequences

- One code path per behaviour, and less to review.
- A breaking change can require updating clients, or resetting data written in
an unsupported format.
22 changes: 22 additions & 0 deletions docs/adr/0005-calver-releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# 0005. CalVer releases

Status: Accepted
Date: 2026-10-30

## Context

Fanout ships frequently and its maturity is better described by date and stage
than by semantic version numbers.

## Decision

Releases use CalVer `YYYY.M.N[-alpha|-beta|-rc]`: full year, unpadded month,
and a release number within the month that starts at 0 and increments for
every published release. The suffix states maturity; GA has none. Git tags
carry a leading `v`; container tags do not. Published versions are immutable.

## Consequences

- A version tells when it was built and how mature it is.
- Product versions, client-contract versions and protocol versions are kept
separate; the product version never appears in API paths or tool names.
26 changes: 26 additions & 0 deletions docs/adr/0006-one-structured-query-path.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# 0006. One structured query path

Status: Accepted
Date: 2026-10-30

## Context

Telemetry is queried from dashboards, FanQL text, MCP tools and the agent, and
a visual builder is planned. Separate query paths would each need their own validation,
limits and SQL generation, and could diverge.

## Decision

Every surface produces the same validated structured query (`from`, `where`,
`measures`, `by`, `bucket`, `histogram`, `sort`, `limit`), which one engine path
validates, budgets and turns into SQL. FanQL is the text form of that query and
compiles to it; its filters become predicates inside the query, checked by the
same validation, and it never runs SQL of its own. Arbitrary read-only SQL remains a
separate, locked-down gate for what the structured query cannot express.

## Consequences

- One place enforces relations, functions and budgets.
- A FanQL query executes through the same engine path as the equivalent
dashboard panel.
- New query features are added once and become available everywhere.
23 changes: 23 additions & 0 deletions docs/adr/0007-hand-written-fanql-parser.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# 0007. A hand-written FanQL parser

Status: Accepted
Date: 2026-10-30

## Context

FanQL needs precise, recovering diagnostics with source spans and suggested
fixes, position-aware completion, and canonical printing, all fast enough to
run on every keystroke.

## Decision

FanQL uses a hand-written lexer and recursive-descent parser in Go, with no
parser-generator or grammar library. It recovers after errors so it can report
every independent problem in one pass.

## Consequences

- Diagnostics, completion and printing share one parser and one tree.
- No generated code or external grammar toolchain to maintain.
- Grammar changes are code changes, covered by the corpus, round-trip and fuzz
tests.
34 changes: 34 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Architecture decision records

An ADR records one decision that shapes the code, with its context and
consequences, in a page or less. [AGENTS.md](../../AGENTS.md) holds the binding
rules; an ADR records why a rule exists. Write one when a decision would
otherwise be re-argued. Supersede an ADR with a new one rather than editing its
decision.

ADRs are numbered in order: `NNNN-short-slug.md`.

## Template

```markdown
# NNNN. Title

Status: Accepted | Superseded by NNNN
Date: YYYY-MM-DD

## Context
## Decision
## Consequences
```

## Index

| # | Decision | Status |
|---|---|---|
| 0001 | [Environment-only configuration](0001-environment-only-configuration.md) | Accepted |
| 0002 | [SQLite with Goose and sqlc for control state](0002-sqlite-goose-sqlc-control-state.md) | Accepted |
| 0003 | [Pinned DuckDB engine and batch format 4](0003-pinned-duckdb-and-batch-format-4.md) | Accepted |
| 0004 | [No legacy paths](0004-no-legacy-paths.md) | Accepted |
| 0005 | [CalVer releases](0005-calver-releases.md) | Accepted |
| 0006 | [One structured query path](0006-one-structured-query-path.md) | Accepted |
| 0007 | [A hand-written FanQL parser](0007-hand-written-fanql-parser.md) | Accepted |
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Logged-in UI: shell rebuild, chat pane, native dashboard widgets, sign-in code step

Status: Implemented
Date: 2026-09-05
Status: approved design, pending implementation plan
Pull request: #226
Scope: `ui/host` (React host app), `ui/apps` (embedded MCP views, small changes), `ui/` shared modules. No Go changes are expected; the server widget allowlist and routes stay as they are.

## Why
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Agent-built dashboards

Status: draft for review · 2026-10-04 · branch `feat/agent-dashboards`
Status: Implemented
Date: 2026-10-04
Pull requests: #285, #286, #287, #295 (milestones 1 to 4)

## Goal

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Milestone 5: correct metrics and the system page

Status: draft for review · 2026-10-10 · branch `feat/agent-dashboards-m5`
Status: Implemented
Date: 2026-10-10
Pull request: #297

This extends `2026-10-04-agent-dashboards-design.md`. Milestones 1 to 4 are
This extends [0002-agent-dashboards.md](0002-agent-dashboards.md). Milestones 1 to 4 are
merged. Its success criteria S1 to S13 stay in force; this milestone adds S14
to S16, S20 and S22. Performance work (a scale benchmark, per-minute summaries
and metric query speed) is deferred to a later milestone.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Full UI without an LLM key

Design, 2026-10-29. Status: draft for review; section 0 (FanQL) is owner-approved as of 2026-10-29.
Status: Accepted
Date: 2026-10-29
Pull request: #410 (section 0, FanQL); sections 1 to 5 are not yet built

## Intent

Expand Down
40 changes: 40 additions & 0 deletions docs/proposals/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Design proposals

A proposal describes a significant change before it is built: what it does,
why, and what was considered instead. Write one when a change adds or changes
a product capability, a public contract (HTTP, MCP, configuration, storage
format), or a cross-cutting design. Small fixes and refactors do not need one.

Proposals are numbered in order: `NNNN-short-slug.md`. Keep the number and file
name when the status changes, so links stay stable.

## Template

```markdown
# Title

Status: Draft | Accepted | Implemented | Superseded by NNNN
Date: YYYY-MM-DD
Pull request: #NNN

## Summary
## Motivation
## Goals and non-goals
## Design
## Alternatives
## Risks
## Open questions
```

Earlier proposals predate the template and keep their original structure.

## Index

| # | Proposal | Status |
|---|---|---|
| 0001 | [Logged-in UI: shell, chat pane, native widgets, sign-in code step](0001-logged-in-ui-shell.md) | Implemented |
| 0002 | [Agent-built dashboards](0002-agent-dashboards.md) | Implemented |
| 0003 | [Milestone 5: correct metrics and the system page](0003-agent-dashboards-m5.md) | Implemented |
| 0004 | [Full UI without an LLM key, and FanQL](0004-fanql.md) | Accepted |

Settled decisions that a proposal leads to are recorded in [../adr](../adr/README.md).
2 changes: 1 addition & 1 deletion internal/fanql/latency_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ func latencyPercentile(samples []time.Duration, percentile float64) time.Duratio
return ordered[max(0, int(math.Ceil(float64(len(ordered))*percentile))-1)]
}

// Latency requirements: ../../docs/specs/2026-10-29-no-llm-ui-design.md,
// Latency requirements: ../../docs/proposals/0004-fanql.md,
// section 0 (especially 0.14 and 0.16). Tests bound work deterministically;
// benchmarks report the strict spec percentiles on an uninstrumented build.
func TestLatencyBudgets(t *testing.T) {
Expand Down
Loading