Refresh server smoke evidence

[raghubetina]

Summary

• replace stale claims that the server AnalysisRun and Compilation slices are still pending
• record the controlled installed-CLI, loopback Rails, and real Solid Queue smoke from a fresh Codex Skill invocation
• pin the evidence boundary to First Draft server 9e29606 and CLI 36f1292
• keep the local path explicitly unreleased, unpublished, unauthenticated, narrow, and non-production
• update server-backed eval wording and repository assertions to enforce the current boundary

Boundary

The demonstrated compiler path remains limited to one Entity using supported scalar Fields. This does not add Plan
GET or pull, a complete semantic analyzer, Publish, arbitrary application support, deployment, authentication, or
release evidence. The controlled success smoke did not exercise cancellation.

Verification

• sh script/check — 14 tests passed
• node script/check-cli-contract.mjs /Users/sandbox2/code/firstdraft/cli
• dependency install audit — 0 vulnerabilities

[raghubetina]

Technical review

A documentation PR whose entire content is claims, so verifying the claims is the review.

The pins are exact

sh script/check                              → 14 tests pass
node script/check-cli-contract.mjs <cli>     → exit 0
npm audit --audit-level=low                  → 0 vulnerabilities

Server baseline 9e29606 is not just some commit on main. It is precisely firstdraft/firstdraft#245's merge commit, and it contains #243 and #244, so it is exactly the revision at which the analysis and compilation lifecycle plus its HTTP transport all exist. CLI 36f1292 is still the tip of that repository's main, so the pin has not gone stale between PRs.

Picking the merge commit of the PR that completed the capability, rather than whatever main happened to be, is the difference between a pin that means something and a timestamp.

The stale claim is genuinely gone

The sentence I flagged when reviewing #9, which said the evals were "not evidence that the pending server AnalysisRun and Compilation lifecycle slices are merged," is removed rather than softened. What replaced it enumerates what is still absent instead of gesturing at it:

There is no Plan GET or pull operation, complete semantic analyzer, Publish action, arbitrary application generation, deployment workflow, or mobile-client generation.

and

That is narrow development evidence, not release, authentication, representative-user, deployment, or production evidence.

Naming the specific missing capabilities is more useful than a general disclaimer, because a reader can check each one.

Finding: one sentence joins a reproducible claim to an unreproducible one

The central new evidence reads:

A controlled local path through an installed CLI, loopback Rails, and real Solid Queue has been exercised, including a fresh Codex invocation of this Skill from a prose request through valid analysis and 151-file Movie application materialization.

Those are two different claims, and only one of them is checkable.

The CLI, Rails, and Solid Queue path is backed by a committed script. First Draft's evidence ledger links it directly:

The controlled CLI smoke pins the installed runtime tree from CLI commit 36f1292, runs loopback Rails with the real Solid Queue adapter, and materializes the bounded 151-file artifact.

Anyone can run that. Good.

The fresh Codex invocation is different. An agent reading this Skill and driving the workflow from a prose request is not what that script exercises, and nothing in this repository records it. There is no docs/ directory here and no field-report convention, so the claim exists only in the README and this PR body.

The word doing the damage is "including," which reads as though the agent session is part of the scripted exercise. It is the one genuinely new thing here and the one nobody can reproduce.

Two ways to fix it, both cheap. Split the sentence and cite script/compilation_http_cli_smoke for the reproducible half, so a reader knows where to look. Then either label the agent session as a one-off observation with its date and the revisions it ran against, or commit a short report the way firstdraft/firstdraft handled the comparable case in docs/solutions/2026-07-29-cli-api-foundation-plan-round-trip.md.

Worth raising because this is the same shape as the finding on #235, which led to a committed script and a corrected ledger, and the same shape as the one on firstdraft/cli#8, where "the packed executable against a real local endpoint" turned out to be two separate verifications. The pattern is a conjunction whose parts are each true and whose whole was never demonstrated.

None of this makes the PR wrong. The compiler path really is reproducible, the pins really are exact, and the boundary language is otherwise careful. It is one sentence that claims slightly more than the repository can show.

[raghubetina]

Lesson: the README that quietly became a lie

This PR does not add a feature. It deletes sentences that stopped being true.

A month ago the repository said the server side was pending. Then the server shipped, and nobody edited that sentence, because editing prose is not on anyone's checklist when they merge a feature. So the README sat there confidently telling readers something false.

That is documentation rot, and it is worth treating as a bug class rather than an inevitability.

Why it is worse than no documentation

Nobody trusts a blank page. Everybody trusts a confident paragraph.

A stale "this is not implemented yet" makes people build workarounds for a problem you solved. A stale "this is production ready" makes people ship on top of something that is not. Both cost more than silence would have.

And rot compounds. Once a reader catches one wrong statement, they stop believing the correct ones too, which is how a team ends up with a wiki nobody reads and knowledge that lives only in whoever has been there longest.

The trick: anchor claims to things that can be checked

Prose drifts because nothing verifies it. The fix is to write claims that are attached to something mechanical.

This PR does it with a commit hash:

The project-scoped AnalysisRun and Compilation transport is merged at server baseline `9e29606`.

Notice what that buys. "The server supports compilation" is a claim with no expiry and no way to check it. "It is merged at 9e29606" is a claim you can verify in ten seconds, and it tells you exactly which version the rest of the paragraph describes.

The pin is also chosen well. 9e29606 is the merge commit of the pull request that completed the capability, not whatever main happened to be that afternoon. A pin to a moving target is decoration.

Other anchors that work the same way:

• Link to the test or script that proves it. "The controlled CLI smoke [script link] materializes the 151-file artifact" survives, because if the script is deleted or renamed the link breaks.
• Link to code, not concepts. A path to a file gets caught by a link checker. "The importer handles this" does not.
• Put counts in the tests. A doc claiming 25 supported shapes is prose. An assertion of 25 fails when someone adds a twenty-sixth.

This repository runs a documentation link check in CI, which is exactly this idea mechanized: it cannot tell whether a sentence is true, but it can tell when the thing a sentence points at has moved.

Say what is not true, specifically

The replacement text here does something worth copying:

There is no Plan GET or pull operation, complete semantic analyzer, Publish action, arbitrary application generation, deployment workflow, or mobile-client generation.

Compare with the version most projects write: "this is experimental, some features are not yet supported."

The general disclaimer ages into meaninglessness, because it stays technically true forever and stops carrying information the day it is written. The specific list has a natural expiry: when Publish ships, someone deleting that item is doing an obvious edit rather than a judgment call.

Enumerate the gaps. It is more work up front and much less work later, and it makes staleness visible rather than invisible.

Separate what you ran from what anyone can run

The other habit worth taking from this review. There are two kinds of evidence and they should not share a sentence.

Reproducible: a committed script, a CI job, a test. Anybody can check it, and it keeps being checked after you stop paying attention.

Anecdotal: you ran it once on your machine and it worked. Real, useful, and it proves something happened one time under conditions nobody recorded.

Both belong in a status document. What causes trouble is joining them with "and" so a reader cannot tell which half is which. I flagged one sentence here that does exactly that: a scripted smoke and a manual agent session, presented as one exercise.

If you keep them apart, the anecdote is honest evidence. If you blend them, the reproducible half lends credibility the anecdote did not earn.

The practical version

You are not going to keep every document current by willpower. So:

• when you finish a feature, grep the docs for the feature name before you open the PR
• prefer claims that point at a commit, a file, or a test over claims that stand alone
• list the specific gaps instead of writing a general disclaimer
• keep "we ran this once" separate from "here is how you run it"
• when you find a stale sentence, fix it in that PR rather than filing a ticket

That last one is the whole discipline. This PR is somebody doing exactly that, and it is worth as much as the feature work it corrects.

[raghubetina]

Addressed the evidence finding in 1fba4a8:

• linked the committed CLI smoke for the reproducible Rails/Solid Queue path;
• separated the fresh-agent run as a one-off observation dated 2026-07-30 and pinned its Skill, server, and CLI revisions;
• explicitly stated that the observation is not a reproducible agent eval;
• refreshed the importer boundary for the now-landed Reference and Predicate slices and pinned the duplicated Skill summary in tests.

The Skill pin e24b438 is intentional: it is the exact current origin/main commit, so the provenance link is durable.

Verification:

• sh script/check — 14/14 tests pass
• node script/check-cli-contract.mjs /Users/sandbox2/code/firstdraft/cli — passes
• npm audit --omit=dev — 0 vulnerabilities
• git diff --check — passes