Skip to content

Correct documentation claims the first pass did not audit - #107

Merged
MaximumTrainer merged 1 commit into
mainfrom
docs/accuracy-pass-2
Sep 13, 2026
Merged

MaximumTrainer merged 1 commit into
mainfrom
docs/accuracy-pass-2

Conversation

@MaximumTrainer

Copy link
Copy Markdown
Owner

A second documentation pass, over the files the previous one (#106) changed only glancingly or not at all: README.md, the runbook's monitoring section, docs/journey-json.md, website/docs/test-journeys.md and the landing page.

Every correction below was found by checking the document against the source, not by reading it.

Monitoring was the worst of it

Six of the seven metrics the runbook told operators to alert on do not exist.

Runbook said Reality
waymark_session_started_total onboarding_sessions_started_total (label flowId)
waymark_session_completed_total onboarding_sessions_completed_total (label flowId)
waymark_session_abandoned_total No such metric. Nothing counts timeout abandonment
waymark_webhook_delivery_failed_total One counter onboarding_webhook_deliveries_total with a status label
waymark_webhook_delivery_retried_total Same — not a second metric
dotnet_duration_seconds Does not exist. UseHttpMetrics() is never called, so there is no request-duration series at all
process_cpu_seconds_total Correct

Alerts built from that table would never have fired — worse than no monitoring, because it reads as coverage. The section now lists what is actually emitted and, just as importantly, names what is not: no HTTP request metrics, no abandoned-session counter, and onboarding_active_sessions is a registered gauge that nothing ever sets, so it reads zero permanently.

It also promised a docs/grafana-dashboard.json that has never existed, hedged as "(if available)".

README

Three of the five NodeType ordinals were wrong. Documented as Form=0, Information=1, Logic=2, Redirect=3, DocumentUpload=4; actually Form=0, DocumentUpload=1, Redirect=2, Information=3, Logic=4. The column stores the ordinal with no value conversion, so anyone writing seed data or raw SQL against the documented numbers would create the wrong node type. (ConditionOperator and SessionStatus were both correct.)

Two tables were undocumented — AnalyticsEvents (the durable trail from #95) and SessionReadModels — now described, with a note on why neither appears in the ER diagram.

website/docs/test-journeys.md

Every submit example omitted the payload envelope the API binds, so none of the three walkthroughs would have worked as written. The API key shown (dev-key) was also not the one the app ships.

Landing page

  • Claimed journeys can be defined in "JSON or YAML" (twice). There is no YAML support anywhere in the backend.
  • Claimed document upload has "progress streaming via SSE". Progress comes from xhr.upload.onprogress; SSE carries session step progress, a different thing.

journey-json.md

Pointed at VisualJourneyCanvas.tsx for NODE_TYPE_STYLES, which lives in visualJourneyCanvasUtils.ts. The colour mapping it documents is correct.

Verification

Added two checks beyond the previous pass, both now passing:

  • Documented metric names match those registered in PrometheusMetricsService exactly — 0 documented-but-unregistered.
  • Documented enum ordinals match the source — 0 mismatches.

Plus the existing battery: all links resolve, every documented /api path maps to one of 27 real routes, both website pages parse with balanced nesting, all markdown tables well formed. Backend 354 passed / 3 skipped (broker tests; Docker down locally), frontend 175 passed.

Note

UseHttpMetrics() being absent is a genuine gap, not just a documentation one — there is no per-endpoint latency or error-rate telemetry. I documented the absence rather than adding the middleware, since that is a runtime change beyond a docs pass. Worth a follow-up issue if you want that telemetry.

🤖 Generated with Claude Code

A second pass over README.md, the runbook's monitoring section,
journey-json.md, test-journeys.md and the landing page - the files the
previous pass changed only glancingly or not at all. Each correction was
found by checking the document against the source rather than by reading.

Monitoring was the worst of it. Six of the seven metrics the runbook told
operators to alert on do not exist: the names are waymark_session_*, the
code registers onboarding_sessions_*; there is no abandoned-session
counter at all; webhook failures and retries are one counter with a status
label, not two metrics; and dotnet_duration_seconds needs UseHttpMetrics,
which is never called. Alerts built from that table would never have
fired, which is worse than no monitoring because it looks like coverage.
The section now lists what is actually emitted, and names what is not -
including onboarding_active_sessions, a gauge nothing ever sets.

README: three of the five NodeType ordinals were wrong (Information,
Logic, Redirect and DocumentUpload are 3, 4, 2 and 1, not 1, 2, 3 and 4).
The column stores the ordinal with no conversion, so anyone writing seed
data or raw SQL against the documented values would create the wrong node
type. AnalyticsEvents and SessionReadModels were undocumented; both are
now described.

test-journeys.md: every submit example omitted the payload envelope the
API binds, so none of the walkthroughs would have worked, and the API key
shown was not the one the app ships.

index.html claimed journeys can be defined in "JSON or YAML" - there is no
YAML support anywhere - and that document upload streams progress over
SSE, when progress comes from XHR events.

journey-json.md pointed at the wrong file for NODE_TYPE_STYLES.

Verified: documented metric names and enum ordinals now match the source
exactly, all links resolve, every documented /api path maps to a real
route, both pages parse, tables are well formed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@MaximumTrainer
MaximumTrainer merged commit 0e551ee into main Sep 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant