diff --git a/README.md b/README.md index fcf2dcc..47fdc1d 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,8 @@ built for agents that author and review Foundation Plans with their users. The package is not released yet. This repository contains the auditable command shell, local Foundation Plan initialization, subject identity generation, conditional whole-document push, and whole-graph analysis status -polling; release behavior will arrive in reviewed increments. +polling, plus explicit compilation and verified local artifact materialization; release behavior will arrive in +reviewed increments. ## Requirements @@ -90,23 +91,59 @@ to retry a bounded number of times because the command sends only `GET` requests inspect the API origin pinned in `.firstdraft/state.json`; an invalid server response instead requires reconciling the CLI and server contract. -Every handled failure from `plan init`, `plan subject-id`, `plan push`, or `plan status` writes exactly one JSON -object to standard error. Agents should branch on its stable `error` value, not on the human-readable `detail`: - -| Commands | `error` | Exit | Meaning | -| ---------------------------------------------------------- | ----------------------------- | ---: | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `plan init`, `plan subject-id`, `plan push`, `plan status` | `invalid_arguments` | 2 | The command syntax is invalid; nothing was written and no request was made. | -| `plan init` | `local_initialization_failed` | 1 | Local initialization failed. The directory may be incomplete; existing files were not overwritten. | -| `plan push` | `invalid_configuration` | 2 | The configured API origin is invalid or conflicts with pinned local state; no request was made. | -| `plan push`, `plan status` | `local_input_unreadable` | 1 | The required local Plan or private state could not be read; no request was made. | -| `plan status` | `project_not_pushed` | 1 | Local state is valid but has no pinned remote Project yet; run `plan push` first. | -| `plan push` | `request_outcome_unknown` | 1 | A sent request or its response could not be verified. Stop and reconcile before pushing again. The object includes `status` when one was received. | -| `plan status` | `status_unavailable` | 1 | The network request or response stream failed. The object includes `status` when headers were received; retry the GET a bounded number of times. | -| `plan status` | `invalid_server_response` | 1 | First Draft returned a response that does not satisfy the status contract. The object includes `status`; retrying unchanged will not repair the mismatch. | -| `plan push`, `plan status` | `server_rejected` | 1 | First Draft returned a validated rejection. The object includes `status` and a whitelisted `response` containing validated problem details or diagnostics. | -| `plan status --wait` | `analysis_changed` | 1 | A different current analysis appeared while polling. `current` contains its validated state; start a fresh wait to follow it explicitly. | -| `plan status --wait` | `wait_timed_out` | 1 | The two-minute wait ended while processing continued. `current` contains the last validated state; another wait is safe. | -| `plan push` | `local_state_not_saved` | 1 | The server accepted the Plan, but local state replacement failed. This is the only error that includes private `recovery_state`. | +## Compile a valid Plan + +After the current analysis is `valid`, choose an absent destination: + +```sh +firstdraft plan compile --output ../oscar-party +``` + +The command conditionally starts one Compilation using the complete Foundation Plan ETag saved by the last +successful push. It never reads an API origin from the current environment, retries an ambiguous `POST`, follows a +replacement Compilation, or overwrites an output path. It polls the accepted Compilation sequentially once per +second for at most ten minutes. A failed or cancelled Compilation is returned as a handled domain failure. + +On success, the CLI downloads the Compilation's deterministic artifact, verifies its media type, declared and actual +byte sizes, strong digest ETag, exact-byte SHA-256, canonical UTF-8 JSON envelope, provenance, metadata-only manifest +digest, portable paths, strict Base64 contents, file digests, modes, owners, and source-subject UUIDs. It writes +exclusively into a uniquely created sibling directory, verifies the complete tree, and atomically renames that +directory into the still-absent destination. On POSIX, generated directories, including the output root, are created +and verified at mode `0755` independent of the caller's umask; files use their artifact-declared `0644` or `0755` +mode. Windows cannot represent those POSIX permission distinctions, so the CLI still verifies the complete structure, +contents, and digests there without claiming exact mode bits. The success JSON names the pinned Project and +Compilation and reports the local output path, file count, and manifest digest; it never prints generated file +contents or private local state. + +Every handled failure from `plan init`, `plan subject-id`, `plan push`, `plan status`, or `plan compile` writes +exactly one JSON object to standard error. Agents should branch on its stable `error` value, not on the human-readable +`detail`: + +| Commands | `error` | Exit | Meaning | +| -------------------------------------------------------------------------- | -------------------------------- | ---: | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `plan init`, `plan subject-id`, `plan push`, `plan status`, `plan compile` | `invalid_arguments` | 2 | The command syntax is invalid; nothing was written and no request was made. | +| `plan init` | `local_initialization_failed` | 1 | Local initialization failed. The directory may be incomplete; existing files were not overwritten. | +| `plan push`, `plan compile` | `invalid_configuration` | 2 | API configuration or the saved ETag is incompatible with the requested command; no request was made. | +| `plan push`, `plan status`, `plan compile` | `local_input_unreadable` | 1 | The required local Plan or private state could not be read; no request was made. | +| `plan status`, `plan compile` | `project_not_pushed` | 1 | Local state is valid but has no pinned remote Project yet; run `plan push` first. | +| `plan push`, `plan compile` | `request_outcome_unknown` | 1 | A sent mutation or its response could not be verified. Stop and reconcile instead of retrying it automatically. | +| `plan status` | `status_unavailable` | 1 | The network request or response stream failed. The object includes `status` when headers were received; retry the GET a bounded number of times. | +| `plan status` | `invalid_server_response` | 1 | First Draft returned a response that does not satisfy the status contract. The object includes `status`; retrying unchanged will not repair the mismatch. | +| `plan push`, `plan status` | `server_rejected` | 1 | First Draft returned a validated rejection. The object includes `status` and a whitelisted `response` containing validated problem details or diagnostics. | +| `plan status --wait` | `analysis_changed` | 1 | A different current analysis appeared while polling. `current` contains its validated state; start a fresh wait to follow it explicitly. | +| `plan status --wait` | `wait_timed_out` | 1 | The two-minute wait ended while processing continued. `current` contains the last validated state; another wait is safe. | +| `plan push` | `local_state_not_saved` | 1 | The server accepted the Plan, but local state replacement failed. This is the only error that includes private `recovery_state`. | +| `plan compile` | `compilation_start_rejected` | 1 | First Draft rejected the conditional start; a validated problem may be included as `response`. | +| `plan compile` | `compilation_status_unavailable` | 1 | The first failed read stopped polling the pinned Compilation; a validated problem may be included. | +| `plan compile` | `invalid_compilation_status` | 1 | A status response violated the exact Compilation contract. | +| `plan compile` | `compilation_changed` | 1 | Compilation identity, immutable metadata, or lifecycle progression changed while polling. | +| `plan compile` | `compilation_wait_timed_out` | 1 | The ten-minute deadline ended; `current` contains the last validated status. | +| `plan compile` | `compilation_failed` | 1 | The pinned Compilation failed; `current` contains its validated failure. | +| `plan compile` | `compilation_cancelled` | 1 | The pinned Compilation was cancelled. | +| `plan compile` | `artifact_unavailable` | 1 | The artifact GET failed or was rejected before materialization. | +| `plan compile` | `invalid_artifact` | 1 | Artifact transport metadata, bytes, provenance, manifest, or files failed validation. | +| `plan compile` | `invalid_output_path` | 2 | The output is not absent beneath an existing real directory; no request was made. | +| `plan compile` | `materialization_failed` | 1 | After artifact validation, filesystem state changed or the verified tree could not be written and atomically published. | Handled failure output never includes command arguments, local Plan bytes, runtime paths, raw filesystem or network errors, or unvalidated response bodies. Optional fields inside a validated rejection diagnostic are omitted when diff --git a/scripts/check-pack.js b/scripts/check-pack.js index 864add2..0b32b5e 100644 --- a/scripts/check-pack.js +++ b/scripts/check-pack.js @@ -27,9 +27,11 @@ if (result.status !== 0) { "package.json", "src/api-response.js", "src/cli.js", + "src/commands/plan-compile.js", "src/commands/plan-init.js", "src/commands/plan-push.js", "src/commands/plan-status.js", + "src/compilation-artifact.js", "src/file-system.js", "src/plan-state.js", "src/uuid-v7.js", diff --git a/scripts/smoke-package.js b/scripts/smoke-package.js index 692d414..ef377d6 100644 --- a/scripts/smoke-package.js +++ b/scripts/smoke-package.js @@ -1,12 +1,16 @@ import assert from "node:assert/strict"; -import { spawnSync } from "node:child_process"; +import { spawn, spawnSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { once } from "node:events"; import { + lstatSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync, } from "node:fs"; +import { createServer } from "node:http"; import { tmpdir } from "node:os"; import path from "node:path"; @@ -168,6 +172,18 @@ try { detail: "Could not read valid local First Draft state. No network request was made. Run 'firstdraft plan init' if this directory is not initialized; otherwise repair the private state before retrying.", }); + + const invalidCompile = spawnPackedCli( + ["plan", "compile", "--canary-secret-option"], + installationDirectory, + ); + assertHandledFailure(invalidCompile, 2, { + error: "invalid_arguments", + detail: + "Invalid arguments. Run 'firstdraft plan compile --help' for usage.", + }); + + await exercisePackedCompilation(projectDirectory); } finally { rmSync(temporaryDirectory, { recursive: true, force: true }); } @@ -210,6 +226,217 @@ function spawnPackedCli(arguments_, cwd = process.cwd()) { }); } +/** + * @param {string[]} arguments_ + * @param {string} cwd + */ +async function spawnPackedCliAsync(arguments_, cwd) { + const child = spawn(process.execPath, [packedExecutable, ...arguments_], { + cwd, + stdio: ["ignore", "pipe", "pipe"], + }); + let stdout = ""; + let stderr = ""; + child.stdout.setEncoding("utf8"); + child.stderr.setEncoding("utf8"); + child.stdout.on("data", (chunk) => { + stdout += chunk; + }); + child.stderr.on("data", (chunk) => { + stderr += chunk; + }); + const [status, signal] = await once(child, "close"); + assert.equal(signal, null); + return { status, stdout, stderr }; +} + +/** @param {string} projectDirectory */ +async function exercisePackedCompilation(projectDirectory) { + const projectId = "01900000-0000-7000-8000-000000000901"; + const compilationId = "01900000-0000-7000-8000-000000000902"; + const analysisId = "01900000-0000-7000-8000-000000000903"; + const headSha256 = "1".repeat(64); + const statusPath = `/v1/projects/${projectId}/compilations/${compilationId}`; + const artifactPath = `${statusPath}/artifact`; + const compilerRelease = "foundation-plan-rails/compiler-scalar-2026-07"; + const target = { id: "rails", profile: "rails-sketch/2026-07" }; + const contents = Buffer.from("class Movie < ApplicationRecord\nend\n"); + const file = { + path: "app/models/movie.rb", + sha256: sha256(contents), + mode: 0o644, + owner: "renderer:model", + source_subject_uuids: [], + contents_base64: contents.toString("base64"), + }; + const metadata = { + path: file.path, + sha256: file.sha256, + mode: file.mode, + owner: file.owner, + source_subject_uuids: file.source_subject_uuids, + }; + const manifestSha256 = sha256( + Buffer.from(JSON.stringify({ files: [metadata] })), + ); + const artifact = Buffer.from( + JSON.stringify({ + format: "firstdraft.compilation-artifact/1", + provenance: { + compilation_id: compilationId, + project_id: projectId, + graph_version: 1, + head_source_sha256: headSha256, + foundation_plan: { + format: "firstdraft.foundation-plan.sketch/0.19", + sha256: "2".repeat(64), + }, + analysis: { + id: analysisId, + release: "foundation-plan-rails/scalar-2026-07", + }, + compiler_release: compilerRelease, + target, + core: { + repository: "firstdraft/foundation-rails-core", + revision: "3".repeat(40), + sha256: "4".repeat(64), + }, + }, + manifest_sha256: manifestSha256, + files: [file], + }), + ); + const artifactSha256 = sha256(artifact); + const compilation = { + project: { id: projectId, graph_version: 1 }, + compilation: { + id: compilationId, + analysis_run_id: analysisId, + graph_version: 1, + status: "succeeded", + compiler_release: compilerRelease, + target, + status_path: statusPath, + cancel_path: `${statusPath}/cancel`, + artifact: { + path: artifactPath, + sha256: artifactSha256, + media_type: "application/vnd.firstdraft.compilation-artifact+json", + byte_size: artifact.byteLength, + }, + failure: null, + created_at: "2026-07-30T12:00:00.000Z", + started_at: "2026-07-30T12:00:01.000Z", + completed_at: "2026-07-30T12:00:02.000Z", + }, + }; + let startRequestSeen = false; + let artifactRequestSeen = false; + const server = createServer(async (request, response) => { + const chunks = []; + for await (const chunk of request) chunks.push(Buffer.from(chunk)); + const requestBody = Buffer.concat(chunks); + + if ( + request.method === "POST" && + request.url === `/v1/projects/${projectId}/compilations` + ) { + assert.equal(request.headers["if-match"], `"sha256:${headSha256}"`); + assert.equal(requestBody.byteLength, 0); + startRequestSeen = true; + respondJson(response, 202, compilation, { Location: statusPath }); + return; + } + if (request.method === "GET" && request.url === artifactPath) { + artifactRequestSeen = true; + response.writeHead(200, { + "Content-Type": "application/vnd.firstdraft.compilation-artifact+json", + "Content-Length": artifact.byteLength, + ETag: `"sha256:${artifactSha256}"`, + }); + response.end(artifact); + return; + } + + response.writeHead(404).end(); + }); + + try { + await new Promise((/** @type {(value?: void) => void} */ resolve) => { + server.listen(0, "127.0.0.1", () => { + resolve(); + }); + }); + const address = server.address(); + assert(address && typeof address === "object"); + const apiUrl = `http://127.0.0.1:${address.port}`; + writeFileSync( + path.join(projectDirectory, ".firstdraft", "state.json"), + `${JSON.stringify( + { + format: "firstdraft.cli-state/1", + project_id: projectId, + api_url: apiUrl, + foundation_plan_etag: `"sha256:${headSha256}"`, + }, + null, + 2, + )}\n`, + { mode: 0o600 }, + ); + const output = path.join(projectDirectory, "generated"); + const execution = await spawnPackedCliAsync( + ["plan", "compile", "--output", output], + projectDirectory, + ); + + assert.equal(execution.status, 0); + assert.equal(execution.stderr, ""); + assert.equal(JSON.parse(execution.stdout).output.path, output); + assert.equal( + readFileSync(path.join(output, "app/models/movie.rb"), "utf8"), + contents.toString("utf8"), + ); + if (process.platform !== "win32") { + assert.equal( + lstatSync(path.join(output, "app/models/movie.rb")).mode & 0o777, + 0o644, + ); + } + assert.equal(startRequestSeen, true); + assert.equal(artifactRequestSeen, true); + } finally { + await new Promise( + ( + /** @type {(value?: void) => void} */ resolve, + /** @type {(error: Error) => void} */ reject, + ) => { + server.close((error) => { + if (error) reject(error); + else resolve(); + }); + }, + ); + } +} + +/** @param {import("node:http").ServerResponse} response @param {number} status @param {unknown} body @param {Record} [headers] */ +function respondJson(response, status, body, headers = {}) { + const source = Buffer.from(JSON.stringify(body)); + response.writeHead(status, { + "Content-Type": "application/json", + "Content-Length": source.byteLength, + ...headers, + }); + response.end(source); +} + +/** @param {Buffer} value */ +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + /** * @param {ReturnType} execution * @param {number} status diff --git a/src/api-response.js b/src/api-response.js index 636e7b5..4ebc5a7 100644 --- a/src/api-response.js +++ b/src/api-response.js @@ -95,13 +95,19 @@ export function isDiagnostic(value) { ); } -/** @param {Response} response */ -async function readResponseBytes(response) { +/** + * @param {Response} response + * @param {number} [maximumBytes] + */ +export async function readResponseBytes( + response, + maximumBytes = MAX_RESPONSE_BYTES, +) { const declaredLength = response.headers.get("content-length"); if ( declaredLength !== null && /^\d+$/.test(declaredLength) && - Number(declaredLength) > MAX_RESPONSE_BYTES + Number(declaredLength) > maximumBytes ) { if (response.body !== null) { await response.body.cancel().catch(() => undefined); @@ -124,7 +130,7 @@ async function readResponseBytes(response) { if (done) break; byteLength += value.byteLength; - if (byteLength > MAX_RESPONSE_BYTES) { + if (byteLength > maximumBytes) { await reader.cancel().catch(() => undefined); throw new FirstDraftProtocolError( "The First Draft response is too large.", diff --git a/src/cli.js b/src/cli.js index 90e97cd..dcd15b0 100644 --- a/src/cli.js +++ b/src/cli.js @@ -1,5 +1,23 @@ import { parseArgs } from "node:util"; +import { + CompilationArtifactInvalidError, + CompilationArtifactResponseInvalidError, + CompilationArtifactUnavailableError, + CompilationCancelledError, + CompilationChangedError, + CompilationFailedError, + CompilationLocalStateError, + CompilationMaterializationError, + CompilationNotPushedError, + CompilationOutputPathError, + CompilationRequestOutcomeUnknownError, + CompilationStartRejectedError, + CompilationStatusInvalidError, + CompilationStatusUnavailableError, + CompilationTimeoutError, + compilePlan, +} from "./commands/plan-compile.js"; import { initializePlan } from "./commands/plan-init.js"; import { PlanPushConfigurationError, @@ -43,6 +61,7 @@ Commands: subject-id Generate a UUIDv7 for a new Plan subject push Send the local Foundation Plan to First Draft status Read the current whole-graph analysis status + compile Compile the accepted Plan into a new local directory Options: -h, --help Show help @@ -76,6 +95,20 @@ The command uses only the API origin pinned by a successful plan push. Without --wait, it makes exactly one status request. `; +const PLAN_COMPILE_HELP = `First Draft CLI + +Usage: + firstdraft plan compile --output + +Options: + --output Materialize the generated application here + -h, --help Show help + +The command starts one compilation of the exact Plan ETag pinned by the +last successful push, waits up to ten minutes, validates the complete +artifact, and atomically renames it into an absent output path. +`; + const PLAN_SUBJECT_ID_HELP = `First Draft CLI Usage: @@ -137,6 +170,38 @@ const PLAN_STATUS_CHANGED_DETAIL = "The current analysis changed while waiting. Run 'firstdraft plan status --wait' again to follow the latest analysis."; const PLAN_STATUS_TIMEOUT_DETAIL = "The current analysis is still processing after the bounded wait. Run 'firstdraft plan status --wait' again to continue waiting."; +const PLAN_COMPILE_INVALID_ARGUMENTS_DETAIL = + "Invalid arguments. Run 'firstdraft plan compile --help' for usage."; +const PLAN_COMPILE_LOCAL_INPUT_UNREADABLE_DETAIL = + "Could not read valid local First Draft state. No network request was made. Run 'firstdraft plan push' before compiling."; +const PLAN_COMPILE_INCOMPATIBLE_STATE_DETAIL = + "The saved Foundation Plan ETag is incompatible with compilation. No network request was made; reconcile the CLI and server contract."; +const PLAN_COMPILE_NOT_PUSHED_DETAIL = + "The local Foundation Plan has not been pushed successfully. Run 'firstdraft plan push' before compiling."; +const PLAN_COMPILE_REQUEST_OUTCOME_UNKNOWN_DETAIL = + "The compilation may have started, but the response could not be verified. Do not start another compilation until the current Project is reconciled."; +const PLAN_COMPILE_START_REJECTED_DETAIL = + "First Draft rejected the compilation start request."; +const PLAN_COMPILE_STATUS_UNAVAILABLE_DETAIL = + "Could not read the pinned compilation status. The command stopped without following or starting another Compilation."; +const PLAN_COMPILE_STATUS_INVALID_DETAIL = + "First Draft returned an invalid compilation status response. Retrying unchanged will not repair this protocol mismatch."; +const PLAN_COMPILE_CHANGED_DETAIL = + "The pinned Compilation changed while being polled. The command stopped without downloading an artifact."; +const PLAN_COMPILE_TIMEOUT_DETAIL = + "The pinned Compilation is still running after the bounded ten-minute wait."; +const PLAN_COMPILE_FAILED_DETAIL = + "The pinned Compilation failed. No artifact was downloaded or materialized."; +const PLAN_COMPILE_CANCELLED_DETAIL = + "The pinned Compilation was cancelled. No artifact was downloaded or materialized."; +const PLAN_COMPILE_ARTIFACT_UNAVAILABLE_DETAIL = + "Could not download the pinned Compilation artifact. No files were materialized."; +const PLAN_COMPILE_ARTIFACT_INVALID_DETAIL = + "The downloaded Compilation artifact did not satisfy the integrity contract. No files were materialized."; +const PLAN_COMPILE_MATERIALIZATION_FAILED_DETAIL = + "The validated Compilation artifact could not be materialized at the requested absent output path."; +const PLAN_COMPILE_INVALID_OUTPUT_PATH_DETAIL = + "The compilation output path must be absent beneath an existing real directory. No network request was made."; const PLAN_SUBJECT_ID_INVALID_ARGUMENTS_DETAIL = "Invalid arguments. Run 'firstdraft plan subject-id --help' for usage."; @@ -161,6 +226,8 @@ const PLAN_SUBJECT_ID_INVALID_ARGUMENTS_DETAIL = * @property {(timeoutMs?: number) => AbortSignal} [createRequestSignal] * @property {(delayMs: number) => Promise} [planStatusSleep] * @property {() => number} [planStatusNow] + * @property {(delayMs: number) => Promise} [planCompileSleep] + * @property {() => number} [planCompileNow] * @property {string} [apiUrl] */ @@ -179,6 +246,8 @@ const PLAN_SUBJECT_ID_INVALID_ARGUMENTS_DETAIL = * @property {(timeoutMs?: number) => AbortSignal} [createRequestSignal] * @property {(delayMs: number) => Promise} [planStatusSleep] * @property {() => number} [planStatusNow] + * @property {(delayMs: number) => Promise} [planCompileSleep] + * @property {() => number} [planCompileNow] * @property {string} [apiUrl] */ @@ -202,6 +271,8 @@ export async function run({ createRequestSignal, planStatusSleep, planStatusNow, + planCompileSleep, + planCompileNow, apiUrl = process.env.FIRSTDRAFT_API_URL, }) { if (argv[0] === "plan") { @@ -220,6 +291,8 @@ export async function run({ createRequestSignal, planStatusSleep, planStatusNow, + planCompileSleep, + planCompileNow, apiUrl, }); } @@ -286,6 +359,8 @@ async function runPlan({ createRequestSignal, planStatusSleep, planStatusNow, + planCompileSleep, + planCompileNow, apiUrl, }) { if (argv[0] === "init") { @@ -327,6 +402,20 @@ async function runPlan({ }); } + if (argv[0] === "compile") { + return runPlanCompile({ + argv: argv.slice(1), + stdout, + stderr, + cwd: cwd ?? getCwd(), + fetchFunction, + planPushFileSystem, + createRequestSignal, + planCompileSleep, + planCompileNow, + }); + } + if (argv[0] === "subject-id") { return runPlanSubjectId({ argv: argv.slice(1), @@ -644,6 +733,215 @@ async function runPlanStatus({ return 0; } +/** + * @param {Pick} options + */ +async function runPlanCompile({ + argv, + stdout, + stderr, + cwd, + fetchFunction, + planPushFileSystem, + createRequestSignal, + planCompileSleep, + planCompileNow, +}) { + const parsed = parseArguments(() => + parseArgs({ + args: [...argv], + options: { + output: { type: "string" }, + help: { type: "boolean", short: "h" }, + }, + allowPositionals: false, + strict: true, + tokens: true, + }), + ); + + if (!parsed || repeatedValueOption(parsed.tokens)) { + writeJson(stderr, { + error: "invalid_arguments", + detail: PLAN_COMPILE_INVALID_ARGUMENTS_DETAIL, + }); + return 2; + } + + if (parsed.values.help) { + stdout.write(PLAN_COMPILE_HELP); + return 0; + } + + if ( + typeof parsed.values.output !== "string" || + parsed.values.output.length === 0 || + parsed.values.output.includes("\0") + ) { + writeJson(stderr, { + error: "invalid_arguments", + detail: PLAN_COMPILE_INVALID_ARGUMENTS_DETAIL, + }); + return 2; + } + + let result; + try { + result = await compilePlan({ + cwd, + output: parsed.values.output, + fetchFunction, + fileSystem: planPushFileSystem, + createRequestSignal, + sleep: planCompileSleep, + now: planCompileNow, + }); + } catch (error) { + if (error instanceof PlanPushLocalError) { + writeJson(stderr, { + error: "local_input_unreadable", + detail: PLAN_COMPILE_LOCAL_INPUT_UNREADABLE_DETAIL, + }); + return 1; + } + + if (error instanceof CompilationLocalStateError) { + writeJson(stderr, { + error: "invalid_configuration", + detail: PLAN_COMPILE_INCOMPATIBLE_STATE_DETAIL, + }); + return 2; + } + + if (error instanceof CompilationNotPushedError) { + writeJson(stderr, { + error: "project_not_pushed", + detail: PLAN_COMPILE_NOT_PUSHED_DETAIL, + }); + return 1; + } + + if (error instanceof CompilationRequestOutcomeUnknownError) { + writeJson(stderr, { + error: "request_outcome_unknown", + detail: PLAN_COMPILE_REQUEST_OUTCOME_UNKNOWN_DETAIL, + ...(typeof error.status === "number" ? { status: error.status } : {}), + }); + return 1; + } + + if (error instanceof CompilationStartRejectedError) { + writeJson(stderr, { + error: "compilation_start_rejected", + detail: PLAN_COMPILE_START_REJECTED_DETAIL, + status: error.status, + ...(error.response ? { response: error.response } : {}), + }); + return 1; + } + + if (error instanceof CompilationStatusUnavailableError) { + writeJson(stderr, { + error: "compilation_status_unavailable", + detail: PLAN_COMPILE_STATUS_UNAVAILABLE_DETAIL, + ...(typeof error.status === "number" ? { status: error.status } : {}), + ...(error.response ? { response: error.response } : {}), + }); + return 1; + } + + if (error instanceof CompilationStatusInvalidError) { + writeJson(stderr, { + error: "invalid_compilation_status", + detail: PLAN_COMPILE_STATUS_INVALID_DETAIL, + status: error.status, + }); + return 1; + } + + if (error instanceof CompilationChangedError) { + writeJson(stderr, { + error: "compilation_changed", + detail: PLAN_COMPILE_CHANGED_DETAIL, + current: error.current, + }); + return 1; + } + + if (error instanceof CompilationTimeoutError) { + writeJson(stderr, { + error: "compilation_wait_timed_out", + detail: PLAN_COMPILE_TIMEOUT_DETAIL, + current: error.current, + }); + return 1; + } + + if (error instanceof CompilationFailedError) { + writeJson(stderr, { + error: "compilation_failed", + detail: PLAN_COMPILE_FAILED_DETAIL, + current: error.current, + }); + return 1; + } + + if (error instanceof CompilationCancelledError) { + writeJson(stderr, { + error: "compilation_cancelled", + detail: PLAN_COMPILE_CANCELLED_DETAIL, + current: error.current, + }); + return 1; + } + + if (error instanceof CompilationArtifactUnavailableError) { + writeJson(stderr, { + error: "artifact_unavailable", + detail: PLAN_COMPILE_ARTIFACT_UNAVAILABLE_DETAIL, + ...(typeof error.status === "number" ? { status: error.status } : {}), + ...(error.response ? { response: error.response } : {}), + }); + return 1; + } + + if ( + error instanceof CompilationArtifactResponseInvalidError || + error instanceof CompilationArtifactInvalidError + ) { + writeJson(stderr, { + error: "invalid_artifact", + detail: PLAN_COMPILE_ARTIFACT_INVALID_DETAIL, + ...(error instanceof CompilationArtifactResponseInvalidError + ? { status: error.status } + : {}), + }); + return 1; + } + + if (error instanceof CompilationOutputPathError) { + writeJson(stderr, { + error: "invalid_output_path", + detail: PLAN_COMPILE_INVALID_OUTPUT_PATH_DETAIL, + }); + return 2; + } + + if (error instanceof CompilationMaterializationError) { + writeJson(stderr, { + error: "materialization_failed", + detail: PLAN_COMPILE_MATERIALIZATION_FAILED_DETAIL, + }); + return 1; + } + + throw error; + } + + writeJson(stdout, result); + return 0; +} + /** * @param {Pick} options */ diff --git a/src/commands/plan-compile.js b/src/commands/plan-compile.js new file mode 100644 index 0000000..bf2d083 --- /dev/null +++ b/src/commands/plan-compile.js @@ -0,0 +1,841 @@ +import { createHash } from "node:crypto"; + +import { + ARTIFACT_MEDIA_TYPE, + CompilationArtifactInvalidError, + CompilationMaterializationError, + CompilationOutputPathError, + MAX_ARTIFACT_BYTES, + materializeCompilationArtifact, + parseCompilationArtifact, + resolveOutputTarget, +} from "../compilation-artifact.js"; +import { + FirstDraftNetworkError, + FirstDraftProtocolError, + isProblemBody, + readResponseBody, + readResponseBytes, + responseMediaType, + sendRequest, +} from "../api-response.js"; +import { isUuidV7, readPlanState } from "../plan-state.js"; + +const REQUEST_TIMEOUT_MS = 30_000; +const WAIT_TIMEOUT_MS = 10 * 60_000; +const POLL_INTERVAL_MS = 1_000; +const ALL_STATUSES = new Set([ + "queued", + "running", + "succeeded", + "failed", + "cancelled", +]); +const TERMINAL_STATUSES = new Set(["succeeded", "failed", "cancelled"]); +const RESPONSE_KEYS = ["project", "compilation"]; +const PROJECT_KEYS = ["id", "graph_version"]; +const COMPILATION_KEYS = [ + "id", + "analysis_run_id", + "graph_version", + "status", + "compiler_release", + "target", + "status_path", + "cancel_path", + "artifact", + "failure", + "created_at", + "started_at", + "completed_at", +]; +const TARGET_KEYS = ["id", "profile"]; +const ARTIFACT_KEYS = ["path", "sha256", "media_type", "byte_size"]; +const FAILURE_KEYS = ["phase", "code", "message"]; +const SHA256_PATTERN = /^[0-9a-f]{64}$/; +const HEAD_ETAG_PATTERN = /^"sha256:([0-9a-f]{64})"$/; +const RELEASE_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:/-]*$/; +const MAX_IDENTIFIER_BYTES = 256; +const MAX_FAILURE_MESSAGE_BYTES = 4_096; + +export class CompilationNotPushedError extends Error {} +export class CompilationLocalStateError extends Error {} + +export class CompilationRequestOutcomeUnknownError extends Error { + /** @param {number | undefined} status */ + constructor(status) { + super("The compilation start request outcome is unknown."); + this.status = status; + } +} + +export class CompilationStartRejectedError extends Error { + /** + * @param {number} status + * @param {Record | null} response + */ + constructor(status, response) { + super("First Draft rejected the compilation start request."); + this.status = status; + this.response = response; + } +} + +export class CompilationStatusUnavailableError extends Error { + /** + * @param {number | undefined} status + * @param {Record | null} [response] + */ + constructor(status, response = null) { + super("The compilation status is unavailable."); + this.status = status; + this.response = response; + } +} + +export class CompilationStatusInvalidError extends Error { + /** @param {number} status */ + constructor(status) { + super("The compilation status response is invalid."); + this.status = status; + } +} + +export class CompilationChangedError extends Error { + /** @param {CompilationResponse} current */ + constructor(current) { + super("The pinned compilation changed while it was being polled."); + this.current = current; + } +} + +export class CompilationTimeoutError extends Error { + /** @param {CompilationResponse} current */ + constructor(current) { + super("The pinned compilation did not finish before the wait deadline."); + this.current = current; + } +} + +export class CompilationFailedError extends Error { + /** @param {CompilationResponse} current */ + constructor(current) { + super("The compilation failed."); + this.current = current; + } +} + +export class CompilationCancelledError extends Error { + /** @param {CompilationResponse} current */ + constructor(current) { + super("The compilation was cancelled."); + this.current = current; + } +} + +export class CompilationArtifactUnavailableError extends Error { + /** + * @param {number | undefined} status + * @param {Record | null} [response] + */ + constructor(status, response = null) { + super("The compilation artifact is unavailable."); + this.status = status; + this.response = response; + } +} + +export class CompilationArtifactResponseInvalidError extends Error { + /** @param {number} status */ + constructor(status) { + super("The compilation artifact response is invalid."); + this.status = status; + } +} + +/** + * @typedef {object} CompilationResponse + * @property {{id: string, graph_version: number}} project + * @property {{ + * id: string, + * analysis_run_id: string, + * graph_version: number, + * status: string, + * compiler_release: string, + * target: {id: string, profile: string}, + * status_path: string, + * cancel_path: string, + * artifact: null | {path: string, sha256: string, media_type: string, byte_size: number}, + * failure: null | {phase: string, code: string, message: string}, + * created_at: string, + * started_at: string | null, + * completed_at: string | null + * }} compilation + */ + +/** + * @typedef {object} CompilePlanOptions + * @property {string} cwd + * @property {string} output + * @property {typeof globalThis.fetch} [fetchFunction] + * @property {import("../plan-state.js").PlanStateFileSystem} [fileSystem] + * @property {(timeoutMs: number) => AbortSignal} [createRequestSignal] + * @property {(delayMs: number) => Promise} [sleep] + * @property {() => number} [now] + */ + +/** + * @param {CompilePlanOptions} options + */ +export async function compilePlan({ + cwd, + output, + fetchFunction = globalThis.fetch, + fileSystem, + createRequestSignal = (timeoutMs) => AbortSignal.timeout(timeoutMs), + sleep = sleepFor, + now = Date.now, +}) { + const state = readPlanState({ cwd, fileSystem }); + if (state.api_url === undefined || state.foundation_plan_etag === undefined) { + throw new CompilationNotPushedError( + "The local Foundation Plan has not been pushed.", + ); + } + + const headEtagMatch = HEAD_ETAG_PATTERN.exec(state.foundation_plan_etag); + if (headEtagMatch === null) { + throw new CompilationLocalStateError( + "The saved Foundation Plan ETag cannot identify its accepted source.", + ); + } + const headSourceSha256 = headEtagMatch[1] ?? ""; + const outputTarget = resolveOutputTarget({ cwd, output }); + const endpoint = new URL( + `/v1/projects/${state.project_id}/compilations`, + state.api_url, + ); + + const initial = await startCompilation({ + endpoint, + projectId: state.project_id, + etag: state.foundation_plan_etag, + fetchFunction, + createRequestSignal, + }); + const deadline = now() + WAIT_TIMEOUT_MS; + let current = initial; + + while (!TERMINAL_STATUSES.has(current.compilation.status)) { + const remaining = deadline - now(); + if (remaining <= 0) throw new CompilationTimeoutError(current); + + await sleep(Math.min(POLL_INTERVAL_MS, remaining)); + if (now() >= deadline) throw new CompilationTimeoutError(current); + + const next = await readCompilationStatus({ + apiUrl: state.api_url, + path: initial.compilation.status_path, + projectId: state.project_id, + fetchFunction, + createRequestSignal, + requestTimeout: Math.max( + 1, + Math.min(REQUEST_TIMEOUT_MS, deadline - now()), + ), + }); + if (!sameCompilation(initial, next) || !validTransition(current, next)) { + throw new CompilationChangedError(next); + } + current = next; + } + + if (current.compilation.status === "failed") { + throw new CompilationFailedError(current); + } + if (current.compilation.status === "cancelled") { + throw new CompilationCancelledError(current); + } + + const artifactMetadata = + /** @type {{path: string, sha256: string, media_type: string, byte_size: number}} */ ( + current.compilation.artifact + ); + const source = await downloadArtifact({ + apiUrl: state.api_url, + metadata: artifactMetadata, + fetchFunction, + createRequestSignal, + }); + const artifact = parseCompilationArtifact(source, { + projectId: state.project_id, + compilationId: current.compilation.id, + graphVersion: current.compilation.graph_version, + headSourceSha256, + analysisRunId: current.compilation.analysis_run_id, + compilerRelease: current.compilation.compiler_release, + target: current.compilation.target, + }); + const materialized = materializeCompilationArtifact(artifact, outputTarget); + + return { + project: current.project, + compilation: current.compilation, + output: materialized, + }; +} + +/** + * @param {object} options + * @param {URL} options.endpoint + * @param {string} options.projectId + * @param {string} options.etag + * @param {typeof globalThis.fetch} options.fetchFunction + * @param {(timeoutMs: number) => AbortSignal} options.createRequestSignal + */ +async function startCompilation({ + endpoint, + projectId, + etag, + fetchFunction, + createRequestSignal, +}) { + let response; + let body; + try { + response = await sendRequest(fetchFunction, endpoint, { + method: "POST", + headers: { + Accept: "application/json, application/problem+json", + "If-Match": etag, + }, + redirect: "error", + signal: createRequestSignal(REQUEST_TIMEOUT_MS), + }); + body = await readResponseBody(response); + } catch (error) { + if ( + error instanceof FirstDraftNetworkError || + error instanceof FirstDraftProtocolError + ) { + throw new CompilationRequestOutcomeUnknownError(error.status); + } + + throw error; + } + + if (response.status !== 202) { + const problem = safeProblem(response, body); + if (response.ok || problem === null) { + throw new CompilationRequestOutcomeUnknownError(response.status); + } + + throw new CompilationStartRejectedError(response.status, problem); + } + + const parsed = parseCompilationResponse(body, projectId); + if ( + responseMediaType(response) !== "application/json" || + parsed === null || + response.headers.get("location") !== parsed.compilation.status_path + ) { + throw new CompilationRequestOutcomeUnknownError(response.status); + } + + return parsed; +} + +/** + * @param {object} options + * @param {string} options.apiUrl + * @param {string} options.path + * @param {string} options.projectId + * @param {typeof globalThis.fetch} options.fetchFunction + * @param {(timeoutMs: number) => AbortSignal} options.createRequestSignal + * @param {number} options.requestTimeout + */ +async function readCompilationStatus({ + apiUrl, + path: statusPath, + projectId, + fetchFunction, + createRequestSignal, + requestTimeout, +}) { + const endpoint = new URL(statusPath, apiUrl); + let response; + let body; + try { + response = await sendRequest(fetchFunction, endpoint, { + method: "GET", + headers: { Accept: "application/json, application/problem+json" }, + redirect: "error", + signal: createRequestSignal(requestTimeout), + }); + body = await readResponseBody(response); + } catch (error) { + if (error instanceof FirstDraftNetworkError) { + throw new CompilationStatusUnavailableError(error.status); + } + if (error instanceof FirstDraftProtocolError) { + throw new CompilationStatusInvalidError(error.status); + } + + throw error; + } + + if (response.status !== 200) { + if (!response.ok) { + throw new CompilationStatusUnavailableError( + response.status, + safeProblem(response, body), + ); + } + throw new CompilationStatusInvalidError(response.status); + } + + const parsed = parseCompilationResponse(body, projectId); + if (responseMediaType(response) !== "application/json" || parsed === null) { + throw new CompilationStatusInvalidError(response.status); + } + + return parsed; +} + +/** + * @param {object} options + * @param {string} options.apiUrl + * @param {{path: string, sha256: string, media_type: string, byte_size: number}} options.metadata + * @param {typeof globalThis.fetch} options.fetchFunction + * @param {(timeoutMs: number) => AbortSignal} options.createRequestSignal + */ +async function downloadArtifact({ + apiUrl, + metadata, + fetchFunction, + createRequestSignal, +}) { + const endpoint = new URL(metadata.path, apiUrl); + let response; + let source; + try { + response = await sendRequest(fetchFunction, endpoint, { + method: "GET", + headers: { + Accept: `${ARTIFACT_MEDIA_TYPE}, application/problem+json`, + }, + redirect: "error", + signal: createRequestSignal(REQUEST_TIMEOUT_MS), + }); + source = await readResponseBytes(response, MAX_ARTIFACT_BYTES); + } catch (error) { + if (error instanceof FirstDraftNetworkError) { + throw new CompilationArtifactUnavailableError(error.status); + } + if (error instanceof FirstDraftProtocolError) { + throw new CompilationArtifactResponseInvalidError(error.status); + } + + throw error; + } + + if (response.status !== 200) { + let body = null; + try { + body = JSON.parse( + new TextDecoder("utf-8", { fatal: true }).decode(source), + ); + } catch (error) { + if (!(error instanceof SyntaxError || error instanceof TypeError)) { + throw error; + } + } + throw new CompilationArtifactUnavailableError( + response.status, + safeProblem(response, body), + ); + } + + const contentLength = response.headers.get("content-length"); + if ( + responseMediaType(response) !== ARTIFACT_MEDIA_TYPE || + contentLength === null || + contentLength !== String(metadata.byte_size) || + source.byteLength !== metadata.byte_size || + response.headers.get("etag") !== `"sha256:${metadata.sha256}"` || + sha256(source) !== metadata.sha256 + ) { + throw new CompilationArtifactResponseInvalidError(response.status); + } + + return source; +} + +/** + * @param {unknown} value + * @param {string} projectId + * @returns {CompilationResponse | null} + */ +function parseCompilationResponse(value, projectId) { + if ( + !hasExactKeySet(value, RESPONSE_KEYS) || + !hasExactKeySet(value.project, PROJECT_KEYS) || + !hasExactKeySet(value.compilation, COMPILATION_KEYS) + ) { + return null; + } + + const project = value.project; + const compilation = value.compilation; + if ( + project.id !== projectId || + !isGraphVersion(project.graph_version) || + !isUuidV7(compilation.id) || + !isUuidV7(compilation.analysis_run_id) || + !isGraphVersion(compilation.graph_version) || + compilation.graph_version !== project.graph_version || + typeof compilation.status !== "string" || + !ALL_STATUSES.has(compilation.status) || + !isRelease(compilation.compiler_release) || + !hasExactKeySet(compilation.target, TARGET_KEYS) || + !isRelease(compilation.target.id) || + !isRelease(compilation.target.profile) || + !isCompilationPath(compilation.status_path, projectId, compilation.id) || + !isCompilationPath(compilation.cancel_path, projectId, compilation.id) || + !isNullableArtifact(compilation.artifact, projectId, compilation.id) || + !isNullableFailure(compilation.failure) || + !isTimestamp(compilation.created_at) || + !isNullableTimestamp(compilation.started_at) || + !isNullableTimestamp(compilation.completed_at) || + !hasValidStatusFields(compilation) + ) { + return null; + } + + const target = /** @type {{id: string, profile: string}} */ ( + compilation.target + ); + const artifact = + /** @type {null | {path: string, sha256: string, media_type: string, byte_size: number}} */ ( + compilation.artifact + ); + const failure = + /** @type {null | {phase: string, code: string, message: string}} */ ( + compilation.failure + ); + return { + project: { id: project.id, graph_version: project.graph_version }, + compilation: { + id: compilation.id, + analysis_run_id: compilation.analysis_run_id, + graph_version: compilation.graph_version, + status: compilation.status, + compiler_release: /** @type {string} */ (compilation.compiler_release), + target: { + id: target.id, + profile: target.profile, + }, + status_path: /** @type {string} */ (compilation.status_path), + cancel_path: /** @type {string} */ (compilation.cancel_path), + artifact: + artifact === null + ? null + : { + path: artifact.path, + sha256: artifact.sha256, + media_type: artifact.media_type, + byte_size: artifact.byte_size, + }, + failure: + failure === null + ? null + : { + phase: failure.phase, + code: failure.code, + message: failure.message, + }, + created_at: /** @type {string} */ (compilation.created_at), + started_at: /** @type {string | null} */ (compilation.started_at), + completed_at: /** @type {string | null} */ (compilation.completed_at), + }, + }; +} + +/** @param {Record} compilation */ +function hasValidStatusFields(compilation) { + const status = compilation.status; + const terminal = TERMINAL_STATUSES.has(String(status)); + if ( + (status === "queued" && compilation.started_at !== null) || + (status === "running" && compilation.started_at === null) || + (terminal && compilation.completed_at === null) || + (!terminal && compilation.completed_at !== null) || + (status === "succeeded" && + (compilation.started_at === null || + compilation.artifact === null || + compilation.failure !== null)) || + (status === "failed" && + (compilation.started_at === null || + compilation.artifact !== null || + compilation.failure === null)) || + (status === "cancelled" && + (compilation.artifact !== null || compilation.failure !== null)) || + ((status === "queued" || status === "running") && + (compilation.artifact !== null || compilation.failure !== null)) + ) { + return false; + } + + const created = timestampMilliseconds(String(compilation.created_at)); + const started = + compilation.started_at === null + ? null + : timestampMilliseconds(String(compilation.started_at)); + const completed = + compilation.completed_at === null + ? null + : timestampMilliseconds(String(compilation.completed_at)); + return ( + (started === null || started >= created) && + (completed === null || completed >= created) && + (started === null || completed === null || completed >= started) + ); +} + +/** + * @param {unknown} value + * @param {string} projectId + * @param {string} compilationId + */ +function isNullableArtifact(value, projectId, compilationId) { + return ( + value === null || + (hasExactKeySet(value, ARTIFACT_KEYS) && + isCompilationPath(value.path, projectId, compilationId) && + typeof value.sha256 === "string" && + SHA256_PATTERN.test(value.sha256) && + value.media_type === ARTIFACT_MEDIA_TYPE && + Number.isSafeInteger(value.byte_size) && + Number(value.byte_size) >= 0 && + Number(value.byte_size) <= MAX_ARTIFACT_BYTES) + ); +} + +/** @param {unknown} value */ +function isNullableFailure(value) { + return ( + value === null || + (hasExactKeySet(value, FAILURE_KEYS) && + isRelease(value.phase) && + isRelease(value.code) && + typeof value.message === "string" && + Buffer.byteLength(value.message) > 0 && + Buffer.byteLength(value.message) <= MAX_FAILURE_MESSAGE_BYTES) + ); +} + +/** + * @param {unknown} value + * @param {string} projectId + * @param {string} compilationId + */ +function isCompilationPath(value, projectId, compilationId) { + if ( + typeof value !== "string" || + !value.startsWith("/") || + value.startsWith("//") || + value.includes("\\") + ) { + return false; + } + + const parsed = new URL(value, "https://firstdraft.invalid"); + const scope = `/v1/projects/${projectId}/compilations/${compilationId}`; + return ( + parsed.origin === "https://firstdraft.invalid" && + parsed.pathname === value && + parsed.search === "" && + parsed.hash === "" && + (value === scope || value.startsWith(`${scope}/`)) + ); +} + +/** @param {CompilationResponse} first @param {CompilationResponse} current */ +function sameCompilation(first, current) { + return ( + current.project.id === first.project.id && + current.project.graph_version === first.project.graph_version && + current.compilation.id === first.compilation.id && + current.compilation.analysis_run_id === first.compilation.analysis_run_id && + current.compilation.graph_version === first.compilation.graph_version && + current.compilation.compiler_release === + first.compilation.compiler_release && + current.compilation.target.id === first.compilation.target.id && + current.compilation.target.profile === first.compilation.target.profile && + current.compilation.status_path === first.compilation.status_path && + current.compilation.cancel_path === first.compilation.cancel_path && + current.compilation.created_at === first.compilation.created_at + ); +} + +/** @param {CompilationResponse} previous @param {CompilationResponse} current */ +function validTransition(previous, current) { + const from = previous.compilation.status; + const to = current.compilation.status; + if (from === to) { + return ( + current.compilation.started_at === previous.compilation.started_at && + current.compilation.completed_at === previous.compilation.completed_at && + JSON.stringify(current.compilation.artifact) === + JSON.stringify(previous.compilation.artifact) && + JSON.stringify(current.compilation.failure) === + JSON.stringify(previous.compilation.failure) + ); + } + + if (from === "queued") { + return ( + to === "running" || + to === "succeeded" || + to === "failed" || + to === "cancelled" + ); + } + return ( + from === "running" && + (to === "succeeded" || to === "failed" || to === "cancelled") && + current.compilation.started_at === previous.compilation.started_at + ); +} + +/** @param {Response} response @param {unknown} body */ +function safeProblem(response, body) { + if (!isRecord(body) || !isProblemBody(response, body)) return null; + + return { + ...(body.type === "about:blank" ? { type: body.type } : {}), + title: body.title, + status: body.status, + code: body.code, + detail: body.detail, + }; +} + +/** @param {number} delayMs */ +function sleepFor(delayMs) { + return new Promise((resolve) => { + setTimeout(resolve, delayMs); + }); +} + +/** @param {Buffer} value */ +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +/** @param {unknown} value @returns {value is string} */ +function isRelease(value) { + return ( + typeof value === "string" && + Buffer.byteLength(value) <= MAX_IDENTIFIER_BYTES && + RELEASE_PATTERN.test(value) + ); +} + +/** @param {unknown} value @returns {value is number} */ +function isGraphVersion(value) { + return Number.isSafeInteger(value) && Number(value) >= 1; +} + +/** @param {unknown} value @returns {value is string | null} */ +function isNullableTimestamp(value) { + return value === null || isTimestamp(value); +} + +/** @param {unknown} value @returns {value is string} */ +function isTimestamp(value) { + return timestampParts(value) !== null; +} + +/** @param {string} value */ +function timestampMilliseconds(value) { + const parsed = Date.parse(value); + if (Number.isFinite(parsed)) return parsed; + + return Date.parse(value.replace(/:60(?=[.,Zz+-])/, ":59")) + 1_000; +} + +/** @param {unknown} value */ +function timestampParts(value) { + if (typeof value !== "string") return null; + + const match = + /^(\d{4})-(\d{2})-(\d{2})[Tt](\d{2}):(\d{2}):(\d{2})(?:\.\d+)?(?:[Zz]|([+-])(\d{2}):(\d{2}))$/.exec( + value, + ); + if (!match) return null; + + const year = Number(match[1]); + const month = Number(match[2]); + const day = Number(match[3]); + const hour = Number(match[4]); + const minute = Number(match[5]); + const second = Number(match[6]); + const offsetHour = match[8] === undefined ? 0 : Number(match[8]); + const offsetMinute = match[9] === undefined ? 0 : Number(match[9]); + if ( + month < 1 || + month > 12 || + day < 1 || + day > daysInMonth(year, month) || + hour > 23 || + minute > 59 || + second > 60 || + (second === 60 && (hour !== 23 || minute !== 59)) || + offsetHour > 23 || + offsetMinute > 59 + ) { + return null; + } + + return true; +} + +/** @param {number} year @param {number} month */ +function daysInMonth(year, month) { + if (month === 2) { + return year % 4 === 0 && (year % 100 !== 0 || year % 400 === 0) ? 29 : 28; + } + + return [4, 6, 9, 11].includes(month) ? 30 : 31; +} + +/** + * @param {unknown} value + * @param {string[]} keys + * @returns {value is Record} + */ +function hasExactKeySet(value, keys) { + return ( + isRecord(value) && arraysEqual(Object.keys(value).sort(), [...keys].sort()) + ); +} + +/** @param {unknown} value @returns {value is Record} */ +function isRecord(value) { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** @param {unknown[]} left @param {unknown[]} right */ +function arraysEqual(left, right) { + return ( + left.length === right.length && + left.every((value, index) => value === right[index]) + ); +} + +export { + CompilationArtifactInvalidError, + CompilationMaterializationError, + CompilationOutputPathError, +}; diff --git a/src/compilation-artifact.js b/src/compilation-artifact.js new file mode 100644 index 0000000..9b279ae --- /dev/null +++ b/src/compilation-artifact.js @@ -0,0 +1,701 @@ +import { createHash } from "node:crypto"; +import { + chmodSync, + lstatSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + renameSync, + rmSync, + writeFileSync, +} from "node:fs"; +import path from "node:path"; + +import { isFileSystemError } from "./file-system.js"; +import { isUuidV7 } from "./plan-state.js"; + +export const ARTIFACT_MEDIA_TYPE = + "application/vnd.firstdraft.compilation-artifact+json"; +export const ARTIFACT_FORMAT = "firstdraft.compilation-artifact/1"; +export const FOUNDATION_PLAN_FORMAT = "firstdraft.foundation-plan.sketch/0.19"; +export const MAX_ARTIFACT_BYTES = 16 * 1024 * 1024; + +const ARTIFACT_KEYS = ["format", "provenance", "manifest_sha256", "files"]; +const PROVENANCE_KEYS = [ + "compilation_id", + "project_id", + "graph_version", + "head_source_sha256", + "foundation_plan", + "analysis", + "compiler_release", + "target", + "core", +]; +const FOUNDATION_PLAN_KEYS = ["format", "sha256"]; +const ANALYSIS_KEYS = ["id", "release"]; +const TARGET_KEYS = ["id", "profile"]; +const CORE_KEYS = ["repository", "revision", "sha256"]; +const FILE_KEYS = [ + "path", + "sha256", + "mode", + "owner", + "source_subject_uuids", + "contents_base64", +]; +const SHA1_PATTERN = /^[0-9a-f]{40}$/; +const SHA256_PATTERN = /^[0-9a-f]{64}$/; +const RELEASE_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._:/-]*$/; +const REPOSITORY_PATTERN = /^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/; +const OWNER_PATTERN = /^[A-Za-z][A-Za-z0-9_.:-]*$/; +const COMPONENT_PATTERN = /^[A-Za-z0-9._][A-Za-z0-9._-]*$/; +const WINDOWS_DEVICE_PATTERN = /^(?:CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9])$/i; +const STRICT_BASE64_PATTERN = + /^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/; +const MAX_PATH_BYTES = 1_024; +const MAX_COMPONENT_BYTES = 255; +const MAX_IDENTIFIER_BYTES = 256; +const DIRECTORY_MODE = 0o755; +const POSIX_MODE_BITS_SUPPORTED = process.platform !== "win32"; + +export class CompilationArtifactInvalidError extends Error {} +export class CompilationMaterializationError extends Error {} +export class CompilationOutputPathError extends Error {} + +/** + * @typedef {object} CompilationArtifactExpectations + * @property {string} projectId + * @property {string} compilationId + * @property {number} graphVersion + * @property {string} headSourceSha256 + * @property {string} analysisRunId + * @property {string} compilerRelease + * @property {{id: string, profile: string}} target + */ + +/** + * @typedef {object} ValidatedArtifactFile + * @property {string} path + * @property {string} sha256 + * @property {420 | 493} mode + * @property {string} owner + * @property {string[]} source_subject_uuids + * @property {Buffer} contents + */ + +/** + * @typedef {object} ValidatedCompilationArtifact + * @property {string} manifest_sha256 + * @property {Record} provenance + * @property {ValidatedArtifactFile[]} files + */ + +/** + * @param {Buffer} source + * @param {CompilationArtifactExpectations} expected + * @returns {ValidatedCompilationArtifact} + */ +export function parseCompilationArtifact(source, expected) { + let text; + let parsed; + + try { + text = new TextDecoder("utf-8", { fatal: true }).decode(source); + parsed = JSON.parse(text); + } catch (error) { + if (!(error instanceof SyntaxError || error instanceof TypeError)) { + throw error; + } + + throw new CompilationArtifactInvalidError( + "The compilation artifact is not valid UTF-8 JSON.", + { cause: error }, + ); + } + + if ( + !Buffer.from(JSON.stringify(parsed), "utf8").equals(source) || + !hasExactKeys(parsed, ARTIFACT_KEYS) || + parsed.format !== ARTIFACT_FORMAT || + typeof parsed.manifest_sha256 !== "string" || + !SHA256_PATTERN.test(parsed.manifest_sha256) || + !Array.isArray(parsed.files) + ) { + throw new CompilationArtifactInvalidError( + "The compilation artifact envelope is invalid.", + ); + } + + const manifestSha256 = /** @type {string} */ (parsed.manifest_sha256); + const provenance = parseProvenance(parsed.provenance, expected); + const files = parsed.files.map(parseArtifactFile); + validateManifest(files, manifestSha256); + + return { + manifest_sha256: manifestSha256, + provenance, + files, + }; +} + +/** + * @param {object} options + * @param {string} options.cwd + * @param {string} options.output + */ +export function resolveOutputTarget({ cwd, output }) { + if ( + typeof output !== "string" || + output.length === 0 || + output.includes("\0") + ) { + throw new CompilationOutputPathError( + "The compilation output path is invalid.", + ); + } + + const target = path.resolve(cwd, output); + const parent = path.dirname(target); + if ( + target === parent || + path.basename(target).length === 0 || + !isInitiallyAbsent(target) + ) { + throw new CompilationOutputPathError( + "The compilation output path must be absent.", + ); + } + + let parentStat; + try { + parentStat = lstatSync(parent); + } catch (error) { + if (!isFileSystemError(error)) throw error; + + throw new CompilationOutputPathError( + "The compilation output parent could not be read.", + { cause: error }, + ); + } + if (!parentStat.isDirectory() || parentStat.isSymbolicLink()) { + throw new CompilationOutputPathError( + "The compilation output parent must be a real directory.", + ); + } + + return target; +} + +/** + * @param {ValidatedCompilationArtifact} artifact + * @param {string} target + */ +export function materializeCompilationArtifact(artifact, target) { + const parent = path.dirname(target); + const prefix = path.join(parent, `.firstdraft-${path.basename(target)}-`); + let temporaryDirectory = null; + + try { + assertTargetStillAvailable(target, parent); + temporaryDirectory = mkdtempSync(prefix); + assertOwnedTemporaryDirectory(temporaryDirectory, parent, prefix); + + writeArtifactTree(artifact.files, temporaryDirectory); + applyMode(temporaryDirectory, DIRECTORY_MODE); + verifyArtifactTree(artifact.files, temporaryDirectory); + assertTargetStillAvailable(target, parent); + renameSync(temporaryDirectory, target); + temporaryDirectory = null; + } catch (error) { + if ( + error instanceof CompilationMaterializationError || + isFileSystemError(error) + ) { + throw new CompilationMaterializationError( + "The compilation artifact could not be materialized.", + { cause: error }, + ); + } + + throw error; + } finally { + if (temporaryDirectory !== null) { + removeOwnedTemporaryDirectory(temporaryDirectory, parent, prefix); + } + } + + return { + path: target, + file_count: artifact.files.length, + manifest_sha256: artifact.manifest_sha256, + }; +} + +/** + * @param {unknown} value + * @param {CompilationArtifactExpectations} expected + */ +function parseProvenance(value, expected) { + if ( + !hasExactKeys(value, PROVENANCE_KEYS) || + value.compilation_id !== expected.compilationId || + value.project_id !== expected.projectId || + value.graph_version !== expected.graphVersion || + value.head_source_sha256 !== expected.headSourceSha256 || + !hasExactKeys(value.foundation_plan, FOUNDATION_PLAN_KEYS) || + value.foundation_plan.format !== FOUNDATION_PLAN_FORMAT || + typeof value.foundation_plan.sha256 !== "string" || + !SHA256_PATTERN.test(value.foundation_plan.sha256) || + !hasExactKeys(value.analysis, ANALYSIS_KEYS) || + value.analysis.id !== expected.analysisRunId || + !isRelease(value.analysis.release) || + value.compiler_release !== expected.compilerRelease || + !hasExactKeys(value.target, TARGET_KEYS) || + value.target.id !== expected.target.id || + value.target.profile !== expected.target.profile || + !hasExactKeys(value.core, CORE_KEYS) || + !isRepository(value.core.repository) || + typeof value.core.revision !== "string" || + !SHA1_PATTERN.test(value.core.revision) || + typeof value.core.sha256 !== "string" || + !SHA256_PATTERN.test(value.core.sha256) + ) { + throw new CompilationArtifactInvalidError( + "The compilation artifact provenance is invalid.", + ); + } + + return { + compilation_id: /** @type {string} */ (value.compilation_id), + project_id: /** @type {string} */ (value.project_id), + graph_version: /** @type {number} */ (value.graph_version), + head_source_sha256: /** @type {string} */ (value.head_source_sha256), + foundation_plan: { + format: /** @type {string} */ (value.foundation_plan.format), + sha256: /** @type {string} */ (value.foundation_plan.sha256), + }, + analysis: { + id: /** @type {string} */ (value.analysis.id), + release: /** @type {string} */ (value.analysis.release), + }, + compiler_release: /** @type {string} */ (value.compiler_release), + target: { + id: /** @type {string} */ (value.target.id), + profile: /** @type {string} */ (value.target.profile), + }, + core: { + repository: /** @type {string} */ (value.core.repository), + revision: /** @type {string} */ (value.core.revision), + sha256: /** @type {string} */ (value.core.sha256), + }, + }; +} + +/** + * @param {unknown} value + * @param {number} index + * @returns {ValidatedArtifactFile} + */ +function parseArtifactFile(value, index) { + if ( + !hasExactKeys(value, FILE_KEYS) || + !isArtifactPath(value.path) || + typeof value.sha256 !== "string" || + !SHA256_PATTERN.test(value.sha256) || + (value.mode !== 0o644 && value.mode !== 0o755) || + !isOwner(value.owner) || + !isSourceSubjectUuids(value.source_subject_uuids) || + typeof value.contents_base64 !== "string" || + !STRICT_BASE64_PATTERN.test(value.contents_base64) + ) { + throw new CompilationArtifactInvalidError( + `Compilation artifact file ${index} is invalid.`, + ); + } + + const pathValue = /** @type {string} */ (value.path); + const digest = /** @type {string} */ (value.sha256); + const mode = /** @type {420 | 493} */ (value.mode); + const owner = /** @type {string} */ (value.owner); + const sourceSubjectUuids = /** @type {string[]} */ ( + value.source_subject_uuids + ); + const contentsBase64 = /** @type {string} */ (value.contents_base64); + const contents = Buffer.from(contentsBase64, "base64"); + if ( + contents.toString("base64") !== contentsBase64 || + sha256(contents) !== digest + ) { + throw new CompilationArtifactInvalidError( + `Compilation artifact file ${index} has invalid contents.`, + ); + } + + return { + path: pathValue, + sha256: digest, + mode, + owner, + source_subject_uuids: [...sourceSubjectUuids], + contents, + }; +} + +/** + * @param {ValidatedArtifactFile[]} files + * @param {string} expectedManifestSha256 + */ +function validateManifest(files, expectedManifestSha256) { + const claims = new Map(); + let previousPath = null; + + for (const file of files) { + if (previousPath !== null && file.path <= previousPath) { + throw new CompilationArtifactInvalidError( + "Compilation artifact files are not in manifest order.", + ); + } + previousPath = file.path; + + const components = file.path.split("/"); + for (let index = 0; index < components.length; index += 1) { + const spelling = components.slice(0, index + 1).join("/"); + const collisionKey = spelling.toLowerCase(); + const kind = index === components.length - 1 ? "file" : "directory"; + const existing = claims.get(collisionKey); + if ( + existing !== undefined && + (existing.kind === "file" || + kind === "file" || + existing.spelling !== spelling) + ) { + throw new CompilationArtifactInvalidError( + "Compilation artifact paths conflict.", + ); + } + claims.set(collisionKey, { spelling, kind }); + } + } + + const metadata = files.map((file) => ({ + path: file.path, + sha256: file.sha256, + mode: file.mode, + owner: file.owner, + source_subject_uuids: file.source_subject_uuids, + })); + const manifestSha256 = sha256( + Buffer.from(JSON.stringify({ files: metadata }), "utf8"), + ); + if (manifestSha256 !== expectedManifestSha256) { + throw new CompilationArtifactInvalidError( + "The compilation artifact manifest digest is invalid.", + ); + } +} + +/** @param {unknown} value @returns {value is string} */ +function isArtifactPath(value) { + if ( + typeof value !== "string" || + !isAscii(value) || + Buffer.byteLength(value) === 0 || + Buffer.byteLength(value) > MAX_PATH_BYTES + ) { + return false; + } + + const components = value.split("/"); + return components.every((component) => { + if ( + component.length === 0 || + component === "." || + component === ".." || + Buffer.byteLength(component) > MAX_COMPONENT_BYTES || + !COMPONENT_PATTERN.test(component) || + component.endsWith(".") || + component.toLowerCase() === ".git" + ) { + return false; + } + + return !WINDOWS_DEVICE_PATTERN.test(component.split(".", 1)[0] ?? ""); + }); +} + +/** @param {unknown} value */ +function isOwner(value) { + return ( + typeof value === "string" && + Buffer.byteLength(value) <= MAX_IDENTIFIER_BYTES && + OWNER_PATTERN.test(value) + ); +} + +/** @param {unknown} value */ +function isSourceSubjectUuids(value) { + if (!Array.isArray(value) || !value.every(isUuidV7)) return false; + + const canonical = [...new Set(value)].sort(); + return arraysEqual(value, canonical); +} + +/** @param {unknown} value */ +function isRelease(value) { + return ( + typeof value === "string" && + Buffer.byteLength(value) <= MAX_IDENTIFIER_BYTES && + RELEASE_PATTERN.test(value) + ); +} + +/** @param {unknown} value */ +function isRepository(value) { + return ( + typeof value === "string" && + Buffer.byteLength(value) <= MAX_IDENTIFIER_BYTES && + REPOSITORY_PATTERN.test(value) + ); +} + +/** @param {ValidatedArtifactFile[]} files @param {string} root */ +function writeArtifactTree(files, root) { + const createdDirectories = new Set([""]); + + for (const file of files) { + const components = file.path.split("/"); + for (let index = 1; index < components.length; index += 1) { + const relativeDirectory = components.slice(0, index).join("/"); + if (createdDirectories.has(relativeDirectory)) continue; + + mkdirSync(path.join(root, ...components.slice(0, index)), { + mode: DIRECTORY_MODE, + }); + applyMode(path.join(root, ...components.slice(0, index)), DIRECTORY_MODE); + createdDirectories.add(relativeDirectory); + } + + const destination = path.join(root, ...components); + writeFileSync(destination, file.contents, { + flag: "wx", + mode: file.mode, + }); + applyMode(destination, file.mode); + } +} + +/** @param {ValidatedArtifactFile[]} files @param {string} root */ +function verifyArtifactTree(files, root) { + const expectedFiles = new Map(files.map((file) => [file.path, file])); + const expectedDirectories = new Set(); + for (const file of files) { + const components = file.path.split("/"); + for (let index = 1; index < components.length; index += 1) { + expectedDirectories.add(components.slice(0, index).join("/")); + } + } + + const actualFiles = new Set(); + const actualDirectories = new Set(); + const rootStat = lstatSync(root); + if ( + !rootStat.isDirectory() || + rootStat.isSymbolicLink() || + !hasExpectedMode(rootStat.mode, DIRECTORY_MODE) + ) { + throw new CompilationMaterializationError( + "The materialized compilation root is invalid.", + ); + } + walkTree(root, "", actualFiles, actualDirectories); + if ( + !setsEqual(actualFiles, new Set(expectedFiles.keys())) || + !setsEqual(actualDirectories, expectedDirectories) + ) { + throw new CompilationMaterializationError( + "The materialized compilation tree is invalid.", + ); + } + + for (const relativePath of expectedDirectories) { + const stat = lstatSync(path.join(root, ...relativePath.split("/"))); + if ( + !stat.isDirectory() || + stat.isSymbolicLink() || + !hasExpectedMode(stat.mode, DIRECTORY_MODE) + ) { + throw new CompilationMaterializationError( + "A materialized compilation directory is invalid.", + ); + } + } + + for (const [relativePath, expected] of expectedFiles) { + const fullPath = path.join(root, ...relativePath.split("/")); + const stat = lstatSync(fullPath); + const contents = readFileSync(fullPath); + if ( + !stat.isFile() || + stat.isSymbolicLink() || + !hasExpectedMode(stat.mode, expected.mode) || + !contents.equals(expected.contents) || + sha256(contents) !== expected.sha256 + ) { + throw new CompilationMaterializationError( + "A materialized compilation file is invalid.", + ); + } + } +} + +/** @param {string} target @param {number} mode */ +function applyMode(target, mode) { + if (POSIX_MODE_BITS_SUPPORTED) chmodSync(target, mode); +} + +/** @param {number} actual @param {number} expected */ +function hasExpectedMode(actual, expected) { + return !POSIX_MODE_BITS_SUPPORTED || (actual & 0o777) === expected; +} + +/** + * @param {string} root + * @param {string} relative + * @param {Set} files + * @param {Set} directories + */ +function walkTree(root, relative, files, directories) { + const directory = relative ? path.join(root, ...relative.split("/")) : root; + const entries = readdirSync(directory, { withFileTypes: true }); + + for (const entry of entries) { + const entryRelative = relative ? `${relative}/${entry.name}` : entry.name; + if (entry.isDirectory()) { + directories.add(entryRelative); + walkTree(root, entryRelative, files, directories); + } else if (entry.isFile()) { + files.add(entryRelative); + } else { + throw new CompilationMaterializationError( + "The materialized compilation tree contains a special file.", + ); + } + } +} + +/** @param {string} target @param {string} parent */ +function assertTargetStillAvailable(target, parent) { + if (!isAbsent(target)) { + throw new CompilationMaterializationError( + "The compilation output path is no longer absent.", + ); + } + + const parentStat = lstatSync(parent); + if (!parentStat.isDirectory() || parentStat.isSymbolicLink()) { + throw new CompilationMaterializationError( + "The compilation output parent is no longer a real directory.", + ); + } +} + +/** @param {string} temporaryDirectory @param {string} parent @param {string} prefix */ +function assertOwnedTemporaryDirectory(temporaryDirectory, parent, prefix) { + const stat = lstatSync(temporaryDirectory); + if ( + path.dirname(temporaryDirectory) !== parent || + !temporaryDirectory.startsWith(prefix) || + temporaryDirectory === prefix || + !stat.isDirectory() || + stat.isSymbolicLink() + ) { + throw new CompilationMaterializationError( + "The compilation temporary directory is invalid.", + ); + } +} + +/** @param {string} temporaryDirectory @param {string} parent @param {string} prefix */ +function removeOwnedTemporaryDirectory(temporaryDirectory, parent, prefix) { + try { + assertOwnedTemporaryDirectory(temporaryDirectory, parent, prefix); + rmSync(temporaryDirectory, { recursive: true, force: true }); + } catch (error) { + if (!isFileSystemError(error)) throw error; + } +} + +/** @param {string} candidate */ +function isAbsent(candidate) { + try { + lstatSync(candidate); + return false; + } catch (error) { + if (!isFileSystemError(error)) throw error; + if (error.code === "ENOENT") return true; + + throw new CompilationMaterializationError( + "The compilation output path could not be inspected.", + { cause: error }, + ); + } +} + +/** @param {string} candidate */ +function isInitiallyAbsent(candidate) { + try { + lstatSync(candidate); + return false; + } catch (error) { + if (!isFileSystemError(error)) throw error; + if (error.code === "ENOENT") return true; + + throw new CompilationOutputPathError( + "The compilation output path could not be inspected.", + { cause: error }, + ); + } +} + +/** @param {Buffer} value */ +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +/** @param {string} value */ +function isAscii(value) { + return [...value].every( + (character) => (character.codePointAt(0) ?? 0) <= 0x7f, + ); +} + +/** + * @param {unknown} value + * @param {string[]} keys + * @returns {value is Record} + */ +function hasExactKeys(value, keys) { + return isRecord(value) && arraysEqual(Object.keys(value), keys); +} + +/** @param {unknown} value @returns {value is Record} */ +function isRecord(value) { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** @param {unknown[]} left @param {unknown[]} right */ +function arraysEqual(left, right) { + return ( + left.length === right.length && + left.every((value, index) => value === right[index]) + ); +} + +/** @param {Set} left @param {Set} right */ +function setsEqual(left, right) { + return ( + left.size === right.size && [...left].every((value) => right.has(value)) + ); +} diff --git a/test/compilation-artifact.test.js b/test/compilation-artifact.test.js new file mode 100644 index 0000000..1a399d2 --- /dev/null +++ b/test/compilation-artifact.test.js @@ -0,0 +1,415 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { + lstatSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + symlinkSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import test from "node:test"; + +import { + CompilationArtifactInvalidError, + CompilationOutputPathError, + FOUNDATION_PLAN_FORMAT, + MAX_ARTIFACT_BYTES, + materializeCompilationArtifact, + parseCompilationArtifact, + resolveOutputTarget, +} from "../src/compilation-artifact.js"; + +const PROJECT_ID = "01900000-0000-7000-8000-000000000801"; +const COMPILATION_ID = "01900000-0000-7000-8000-000000000802"; +const ANALYSIS_ID = "01900000-0000-7000-8000-000000000803"; +const SUBJECT_ID = "01900000-0000-7000-8000-000000000804"; +const HEAD_SHA256 = "1".repeat(64); +const COMPILER_RELEASE = "foundation-plan-rails/compiler-scalar-2026-07"; +const TARGET = { id: "rails", profile: "rails-sketch/2026-07" }; +const EXPECTED = { + projectId: PROJECT_ID, + compilationId: COMPILATION_ID, + graphVersion: 7, + headSourceSha256: HEAD_SHA256, + analysisRunId: ANALYSIS_ID, + compilerRelease: COMPILER_RELEASE, + target: TARGET, +}; + +test("parses canonical binary-safe artifact bytes and materializes an exact tree", (context) => { + assert.equal(MAX_ARTIFACT_BYTES, 16 * 1024 * 1024); + const fixture = artifactFixture(); + const artifact = parseCompilationArtifact(fixture.source, EXPECTED); + const parent = temporaryDirectory(context); + const target = resolveOutputTarget({ + cwd: parent, + output: "generated", + }); + const result = materializeCompilationArtifact(artifact, target); + + assert.deepEqual(result, { + path: target, + file_count: 2, + manifest_sha256: fixture.manifestSha256, + }); + assert.equal( + readFileSync(path.join(target, "app/models/movie.rb"), "utf8"), + "class Movie < ApplicationRecord\nend\n", + ); + assert.deepEqual( + readFileSync(path.join(target, "bin/setup")), + Buffer.from([0, 255]), + ); + assert.equal( + [...lstatDirectories(parent)].some((name) => + name.startsWith(".firstdraft-generated-"), + ), + false, + ); +}); + +test("materialization enforces supported permission bits independently of umask", (context) => { + const previousUmask = + process.platform === "win32" ? null : process.umask(0o077); + + try { + const fixture = artifactFixture(); + const artifact = parseCompilationArtifact(fixture.source, EXPECTED); + const parent = temporaryDirectory(context); + const target = resolveOutputTarget({ + cwd: parent, + output: "generated", + }); + materializeCompilationArtifact(artifact, target); + + const entries = [ + [target, 0o755, "directory"], + [path.join(target, "app"), 0o755, "directory"], + [path.join(target, "app/models"), 0o755, "directory"], + [path.join(target, "app/models/movie.rb"), 0o644, "file"], + [path.join(target, "bin/setup"), 0o755, "file"], + ]; + + for (const [entryPath, expectedMode, type] of entries) { + const stat = lstatSync(String(entryPath)); + assert.equal( + type === "directory" ? stat.isDirectory() : stat.isFile(), + true, + ); + if (process.platform !== "win32") { + assert.equal(stat.mode & 0o777, expectedMode); + } + } + } finally { + if (previousUmask !== null) process.umask(previousUmask); + } +}); + +test("rejects noncanonical, duplicate-key, additive, and non-UTF-8 envelopes", () => { + const fixture = artifactFixture(); + const additive = { ...fixture.body, canary: "secret" }; + const duplicate = Buffer.from( + fixture.source + .toString("utf8") + .replace( + '{"format":"firstdraft.compilation-artifact/1"', + '{"format":"firstdraft.compilation-artifact/1","format":"firstdraft.compilation-artifact/1"', + ), + ); + + for (const source of [ + Buffer.from(`${fixture.source.toString("utf8")}\n`), + Buffer.from(JSON.stringify(additive)), + duplicate, + Buffer.from([0xff]), + ]) { + assert.throws( + () => parseCompilationArtifact(source, EXPECTED), + CompilationArtifactInvalidError, + ); + } +}); + +test("pins every available provenance identity", () => { + /** @type {[string, string | number][]} */ + const cases = [ + ["compilation_id", "01900000-0000-7000-8000-000000000899"], + ["project_id", "01900000-0000-7000-8000-000000000899"], + ["graph_version", 8], + ["head_source_sha256", "9".repeat(64)], + ["compiler_release", "other/release"], + ]; + + for (const [key, value] of cases) { + const fixture = artifactFixture({ + provenance: { [key]: value }, + }); + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + ); + } + + for (const fixture of [ + artifactFixture({ + provenance: { + foundation_plan: { + format: "firstdraft.foundation-plan.sketch/0.18", + sha256: "2".repeat(64), + }, + }, + }), + artifactFixture({ + provenance: { + analysis: { + id: "01900000-0000-7000-8000-000000000899", + release: "foundation-plan-rails/scalar-2026-07", + }, + }, + }), + artifactFixture({ + provenance: { target: { id: "other", profile: TARGET.profile } }, + }), + artifactFixture({ + provenance: { + core: { + repository: "not-a-repository", + revision: "3".repeat(40), + sha256: "4".repeat(64), + }, + }, + }), + ]) { + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + ); + } +}); + +test("requires strict Base64, exact file digests, and the metadata-only manifest digest", () => { + const valid = artifactFixture(); + const malformedBase64 = structuredClone(valid.body); + assert(malformedBase64.files[0]); + malformedBase64.files[0].contents_base64 = "YQ"; + const wrongFileDigest = structuredClone(valid.body); + assert(wrongFileDigest.files[0]); + wrongFileDigest.files[0].sha256 = "0".repeat(64); + const wrongManifestDigest = structuredClone(valid.body); + wrongManifestDigest.manifest_sha256 = "0".repeat(64); + + for (const body of [malformedBase64, wrongFileDigest, wrongManifestDigest]) { + assert.throws( + () => + parseCompilationArtifact(Buffer.from(JSON.stringify(body)), EXPECTED), + CompilationArtifactInvalidError, + ); + } +}); + +test("rejects nonportable paths, unsupported metadata, and noncanonical UUID lists", () => { + const invalidPaths = [ + "", + "/absolute", + "back\\slash", + "empty//component", + "./dot", + "parent/../escape", + "trailing.", + ".git/config", + "NUL.txt", + "white space", + "é.rb", + ]; + for (const filePath of invalidPaths) { + const fixture = artifactFixture({ + files: [fileFixture(filePath, "contents")], + }); + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + filePath, + ); + } + + for (const file of [ + fileFixture("valid", "contents", { mode: 0o600 }), + fileFixture("valid", "contents", { owner: "invalid owner" }), + fileFixture("valid", "contents", { + source_subject_uuids: [SUBJECT_ID, SUBJECT_ID], + }), + fileFixture("valid", "contents", { + source_subject_uuids: [ + "01900000-0000-7000-8000-000000000899", + SUBJECT_ID, + ], + }), + ]) { + const fixture = artifactFixture({ files: [file] }); + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + ); + } +}); + +test("rejects duplicate, case-folded, and file-directory prefix conflicts", () => { + const cases = [ + [fileFixture("app/model.rb", "one"), fileFixture("app/model.rb", "two")], + [fileFixture("app/model.rb", "one"), fileFixture("APP/MODEL.RB", "two")], + [fileFixture("app", "one"), fileFixture("app/model.rb", "two")], + [fileFixture("App/one.rb", "one"), fileFixture("app/two.rb", "two")], + ]; + + for (const files of cases) { + const fixture = artifactFixture({ files, preserveOrder: true }); + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + ); + } +}); + +test("requires strict manifest byte order", () => { + const fixture = artifactFixture({ + files: [fileFixture("z-last", "last"), fileFixture("a-first", "first")], + preserveOrder: true, + }); + + assert.throws( + () => parseCompilationArtifact(fixture.source, EXPECTED), + CompilationArtifactInvalidError, + ); +}); + +test("preflight never follows an output-parent symlink or accepts an existing target", (context) => { + const root = temporaryDirectory(context); + const realParent = path.join(root, "real"); + const linkedParent = path.join(root, "linked"); + mkdirSync(realParent); + symlinkSync(realParent, linkedParent); + + assert.throws( + () => + resolveOutputTarget({ + cwd: root, + output: "linked/generated", + }), + CompilationOutputPathError, + ); + + mkdirSync(path.join(realParent, "existing")); + assert.throws( + () => + resolveOutputTarget({ + cwd: root, + output: "real/existing", + }), + CompilationOutputPathError, + ); + assert.deepEqual([...lstatDirectories(realParent)], ["existing"]); +}); + +/** + * @param {{provenance?: Record, files?: Record[], preserveOrder?: boolean}} [changes] + */ +function artifactFixture(changes = {}) { + const defaultFiles = [ + fileFixture( + "app/models/movie.rb", + "class Movie < ApplicationRecord\nend\n", + { owner: "renderer:model", source_subject_uuids: [SUBJECT_ID] }, + ), + fileFixture("bin/setup", Buffer.from([0, 255]), { + mode: 0o755, + owner: "core:foundation-rails-core", + }), + ]; + const files = [...(changes.files ?? defaultFiles)]; + if (!changes.preserveOrder) { + files.sort((left, right) => + String(left.path).localeCompare(String(right.path), "en", { + sensitivity: "variant", + }), + ); + } + const metadata = files.map( + ({ path: filePath, sha256, mode, owner, source_subject_uuids }) => ({ + path: filePath, + sha256, + mode, + owner, + source_subject_uuids, + }), + ); + const manifestSha256 = sha256( + Buffer.from(JSON.stringify({ files: metadata })), + ); + const baseProvenance = { + compilation_id: COMPILATION_ID, + project_id: PROJECT_ID, + graph_version: 7, + head_source_sha256: HEAD_SHA256, + foundation_plan: { + format: FOUNDATION_PLAN_FORMAT, + sha256: "2".repeat(64), + }, + analysis: { + id: ANALYSIS_ID, + release: "foundation-plan-rails/scalar-2026-07", + }, + compiler_release: COMPILER_RELEASE, + target: TARGET, + core: { + repository: "firstdraft/foundation-rails-core", + revision: "3".repeat(40), + sha256: "4".repeat(64), + }, + }; + const body = { + format: "firstdraft.compilation-artifact/1", + provenance: { ...baseProvenance, ...changes.provenance }, + manifest_sha256: manifestSha256, + files, + }; + const source = Buffer.from(JSON.stringify(body)); + return { body, source, manifestSha256 }; +} + +/** + * @param {string} filePath + * @param {string | Buffer} contents + * @param {{mode?: number, owner?: string, source_subject_uuids?: string[]}} [attributes] + */ +function fileFixture(filePath, contents, attributes = {}) { + const bytes = Buffer.isBuffer(contents) ? contents : Buffer.from(contents); + return { + path: filePath, + sha256: sha256(bytes), + mode: attributes.mode ?? 0o644, + owner: attributes.owner ?? "renderer:test", + source_subject_uuids: attributes.source_subject_uuids ?? [], + contents_base64: bytes.toString("base64"), + }; +} + +/** @param {Buffer} value */ +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +/** @param {import("node:test").TestContext} context */ +function temporaryDirectory(context) { + const directory = mkdtempSync(path.join(tmpdir(), "firstdraft-artifact-")); + context.after(() => rmSync(directory, { recursive: true, force: true })); + return directory; +} + +/** @param {string} directory */ +function lstatDirectories(directory) { + return new Set(readdirSync(directory).sort()); +} diff --git a/test/plan-compile.test.js b/test/plan-compile.test.js new file mode 100644 index 0000000..2b0234a --- /dev/null +++ b/test/plan-compile.test.js @@ -0,0 +1,1065 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { + lstatSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + rmSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { createServer } from "node:http"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import test from "node:test"; + +import { + ARTIFACT_MEDIA_TYPE, + FOUNDATION_PLAN_FORMAT, +} from "../src/compilation-artifact.js"; +import { run } from "../src/cli.js"; + +const PROJECT_ID = "01900000-0000-7000-8000-000000000701"; +const COMPILATION_ID = "01900000-0000-7000-8000-000000000703"; +const OTHER_COMPILATION_ID = "01900000-0000-7000-8000-000000000704"; +const ANALYSIS_ID = "01900000-0000-7000-8000-000000000705"; +const SUBJECT_ID = "01900000-0000-7000-8000-000000000706"; +const HEAD_SHA256 = "1".repeat(64); +const ETAG = `"sha256:${HEAD_SHA256}"`; +const CREATED_AT = "2026-07-30T12:00:00.000Z"; +const STARTED_AT = "2026-07-30T12:00:01.000Z"; +const COMPLETED_AT = "2026-07-30T12:00:02.000Z"; +const STATUS_PATH = `/v1/projects/${PROJECT_ID}/compilations/${COMPILATION_ID}`; +const CANCEL_PATH = `${STATUS_PATH}/cancel`; +const ARTIFACT_PATH = `${STATUS_PATH}/artifact`; +const COMPILER_RELEASE = "foundation-plan-rails/compiler-scalar-2026-07"; +const TARGET = { id: "rails", profile: "rails-sketch/2026-07" }; +const PLAN_COMPILE_HELP = `First Draft CLI + +Usage: + firstdraft plan compile --output + +Options: + --output Materialize the generated application here + -h, --help Show help + +The command starts one compilation of the exact Plan ETag pinned by the +last successful push, waits up to ten minutes, validates the complete +artifact, and atomically renames it into an absent output path. +`; + +test("plan compile uses one pinned POST, sequential polling, and one artifact GET", async (context) => { + /** @type {{method: string | undefined, url: string | undefined, headers: import("node:http").IncomingHttpHeaders, body: Buffer}[]} */ + const requests = []; + const artifact = artifactFixture(); + let statusReads = 0; + const server = createServer(async (request, response) => { + const body = await readRequestBody(request); + requests.push({ + method: request.method, + url: request.url, + headers: request.headers, + body, + }); + + if (request.method === "POST" && request.url === compilationCollection()) { + respondJson(response, 202, compilationBody("queued"), { + Location: STATUS_PATH, + }); + return; + } + if (request.method === "GET" && request.url === STATUS_PATH) { + statusReads += 1; + respondJson( + response, + 200, + statusReads === 1 + ? compilationBody("running") + : compilationBody("succeeded", { artifact }), + ); + return; + } + if (request.method === "GET" && request.url === ARTIFACT_PATH) { + response.writeHead(200, { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": artifact.source.byteLength, + ETag: `"sha256:${artifact.sha256}"`, + }); + response.end(artifact.source); + return; + } + + response.writeHead(404).end(); + }); + const apiUrl = await listen(context, server); + const cwd = remoteDirectory(context, apiUrl); + const output = path.join(cwd, "generated-app"); + /** @type {string[]} */ + const events = []; + /** @type {number[]} */ + const timeouts = []; + const result = await invoke(["plan", "compile", "--output", output], { + cwd, + apiUrl: "https://canary-secret.example", + planCompileSleep: async (/** @type {number} */ delayMs) => { + events.push(`sleep:${delayMs}`); + }, + createRequestSignal: (/** @type {number} */ timeoutMs) => { + timeouts.push(timeoutMs); + return new AbortController().signal; + }, + }); + + assert.equal(result.status, 0); + assert.equal(result.stderr, ""); + const printed = JSON.parse(result.stdout); + assert.equal(printed.project.id, PROJECT_ID); + assert.equal(printed.compilation.id, COMPILATION_ID); + assert.equal(printed.compilation.status, "succeeded"); + assert.deepEqual(printed.output, { + path: output, + file_count: 2, + manifest_sha256: artifact.manifestSha256, + }); + assert.deepEqual(events, ["sleep:1000", "sleep:1000"]); + assert.deepEqual(timeouts, [30_000, 30_000, 30_000, 30_000]); + assert.deepEqual( + requests.map(({ method, url }) => [method, url]), + [ + ["POST", compilationCollection()], + ["GET", STATUS_PATH], + ["GET", STATUS_PATH], + ["GET", ARTIFACT_PATH], + ], + ); + + const [start, firstStatus, secondStatus, artifactRequest] = requests; + assert(start); + assert.equal(start.body.byteLength, 0); + assert.equal(start.headers["if-match"], ETAG); + assert.equal( + start.headers.accept, + "application/json, application/problem+json", + ); + assert.equal(start.headers["content-type"], undefined); + for (const statusRequest of [firstStatus, secondStatus]) { + assert(statusRequest); + assert.equal( + statusRequest.headers.accept, + "application/json, application/problem+json", + ); + } + assert(artifactRequest); + assert.equal( + artifactRequest.headers.accept, + `${ARTIFACT_MEDIA_TYPE}, application/problem+json`, + ); + assert.equal( + readFileSync(path.join(output, "app/models/movie.rb"), "utf8"), + "class Movie < ApplicationRecord\nend\n", + ); + assert.deepEqual( + readFileSync(path.join(output, "bin/setup")), + Buffer.from([0, 255]), + ); + if (process.platform !== "win32") { + assert.equal( + lstatSync(path.join(output, "app/models/movie.rb")).mode & 0o777, + 0o644, + ); + assert.equal(lstatSync(path.join(output, "bin/setup")).mode & 0o777, 0o755); + } + assert.doesNotMatch(result.stdout, /canary-secret|sha256:1111/); +}); + +test("plan compile help and invalid arguments do not touch local or network dependencies", async () => { + const inaccessible = () => { + throw new Error("the dependency must remain inaccessible"); + }; + + for (const argv of [ + ["plan", "compile", "--help"], + ["plan", "compile", "-h"], + ["plan", "compile", "--output", "canary-secret", "--help"], + ]) { + assert.deepEqual( + await invoke(argv, { + cwd: process.cwd(), + getCwd: inaccessible, + fetchFunction: inaccessible, + planPushFileSystem: inaccessibleFileSystem(), + }), + { status: 0, stdout: PLAN_COMPILE_HELP, stderr: "" }, + ); + } + + for (const argv of [ + ["plan", "compile"], + ["plan", "compile", "--output"], + ["plan", "compile", "--output", "one", "--output", "two"], + ["plan", "compile", "canary-secret"], + ["plan", "compile", "--canary-secret"], + ]) { + const result = await invoke(argv, { + getCwd: () => process.cwd(), + fetchFunction: inaccessible, + planPushFileSystem: inaccessibleFileSystem(), + }); + + assert.equal(result.status, 2); + assert.equal(result.stdout, ""); + assert.deepEqual(JSON.parse(result.stderr), { + error: "invalid_arguments", + detail: + "Invalid arguments. Run 'firstdraft plan compile --help' for usage.", + }); + assert.doesNotMatch(result.stderr, /canary-secret/); + } +}); + +test("local prerequisites fail before a compilation can start", async (context) => { + const inaccessible = inaccessibleFetch(); + const unpushed = localDirectory(context, { + format: "firstdraft.cli-state/1", + project_id: PROJECT_ID, + }); + assertHandledFailure( + await invoke(["plan", "compile", "--output", "generated"], { + cwd: unpushed, + fetchFunction: inaccessible, + }), + "project_not_pushed", + ); + + const opaque = localDirectory(context, { + format: "firstdraft.cli-state/1", + project_id: PROJECT_ID, + api_url: "https://api.example.test", + foundation_plan_etag: '"opaque"', + }); + assertHandledFailure( + await invoke(["plan", "compile", "--output", "generated"], { + cwd: opaque, + fetchFunction: inaccessible, + }), + "invalid_configuration", + 2, + ); + + const existing = remoteDirectory(context, "https://api.example.test"); + mkdirSync(path.join(existing, "generated")); + assertHandledFailure( + await invoke(["plan", "compile", "--output", "generated"], { + cwd: existing, + fetchFunction: inaccessible, + }), + "invalid_output_path", + 2, + ); + + const symlinkedParent = path.join(existing, "linked-parent"); + symlinkSync(existing, symlinkedParent); + assertHandledFailure( + await invoke(["plan", "compile", "--output", "linked-parent/generated"], { + cwd: existing, + fetchFunction: inaccessible, + }), + "invalid_output_path", + 2, + ); +}); + +test("a sent start request is never retried when its outcome is ambiguous", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + let requests = 0; + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: async () => { + requests += 1; + throw new TypeError("canary-secret-network"); + }, + }); + + assert.equal(requests, 1); + assertHandledFailure(result, "request_outcome_unknown"); + assert.doesNotMatch(result.stderr, /canary-secret|sha256:1111/); +}); + +test("a validated start rejection is safe and does not poll", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: async () => + problemResponse(409, "project_not_valid", "Compile is unavailable."), + }); + + assert.deepEqual(JSON.parse(result.stderr), { + error: "compilation_start_rejected", + detail: "First Draft rejected the compilation start request.", + status: 409, + response: { + type: "about:blank", + title: "Conflict", + status: 409, + code: "project_not_valid", + detail: "Compile is unavailable.", + }, + }); + assert.equal(result.status, 1); +}); + +test("an unvalidated non-success start response remains ambiguous", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + /** @type {unknown[]} */ + const calls = []; + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch( + [ + new Response("bad gateway", { + status: 502, + headers: { "Content-Type": "text/html" }, + }), + ], + calls, + ), + }); + + assert.equal(calls.length, 1); + assertHandledFailure(result, "request_outcome_unknown"); + assert.equal(JSON.parse(result.stderr).status, 502); + assert.doesNotMatch(result.stderr, /html|gateway/); +}); + +test("invalid accepted start responses remain ambiguous", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + for (const response of [ + jsonResponse(compilationBody("queued"), 200), + jsonResponse(compilationBody("queued"), 202), + jsonResponse(compilationBody("queued"), 202, { + Location: "/wrong", + }), + new Response("{}", { + status: 202, + headers: { "Content-Type": "text/plain", Location: STATUS_PATH }, + }), + ]) { + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch([response]), + }); + + assertHandledFailure(result, "request_outcome_unknown"); + } +}); + +test("server-returned request paths stay inside the pinned compilation", async (context) => { + const invalidStatusPaths = [ + `${STATUS_PATH}/../../../evil`, + `${STATUS_PATH}/%2e%2e/evil`, + "//evil.example/x", + `${STATUS_PATH}?x=1`, + `/v1/projects/${PROJECT_ID}/compilations/${OTHER_COMPILATION_ID}`, + ]; + + for (const [index, statusPath] of invalidStatusPaths.entries()) { + const cwd = remoteDirectory(context, `https://api-${index}.example.test`); + const result = await invoke( + ["plan", "compile", "--output", `generated-status-${index}`], + { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse( + compilationBody("queued", { + compilation: { status_path: statusPath }, + }), + 202, + { Location: statusPath }, + ), + ]), + }, + ); + + assertHandledFailure(result, "request_outcome_unknown"); + } + + for (const [index, cancelPath] of [ + `${CANCEL_PATH}?x=1`, + `/v1/projects/${PROJECT_ID}/compilations/${OTHER_COMPILATION_ID}/cancel`, + ].entries()) { + const cwd = remoteDirectory( + context, + `https://api-cancel-${index}.example.test`, + ); + const result = await invoke( + ["plan", "compile", "--output", `generated-cancel-${index}`], + { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse( + compilationBody("queued", { + compilation: { cancel_path: cancelPath }, + }), + 202, + { Location: STATUS_PATH }, + ), + ]), + }, + ); + + assertHandledFailure(result, "request_outcome_unknown"); + } + + const artifact = artifactFixture(); + const cwd = remoteDirectory(context, "https://api-artifact.example.test"); + /** @type {unknown[]} */ + const calls = []; + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch( + [ + jsonResponse( + compilationBody("succeeded", { + artifact, + compilation: { + artifact: { + path: `/v1/projects/${PROJECT_ID}/compilations/${OTHER_COMPILATION_ID}/artifact`, + sha256: artifact.sha256, + media_type: ARTIFACT_MEDIA_TYPE, + byte_size: artifact.source.byteLength, + }, + }, + }), + 202, + { Location: STATUS_PATH }, + ), + ], + calls, + ), + }); + + assert.equal(calls.length, 1); + assertHandledFailure(result, "request_outcome_unknown"); +}); + +test("returned same-origin paths may use scoped action suffixes", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + const artifact = artifactFixture(); + const statusPath = `${STATUS_PATH}/status`; + const artifactPath = `${STATUS_PATH}/downloads/artifact`; + const body = compilationBody("succeeded", { + artifact, + compilation: { + status_path: statusPath, + cancel_path: `${STATUS_PATH}/actions/cancel`, + artifact: { + path: artifactPath, + sha256: artifact.sha256, + media_type: ARTIFACT_MEDIA_TYPE, + byte_size: artifact.source.byteLength, + }, + }, + }); + const output = path.join(cwd, "generated"); + const result = await invoke(["plan", "compile", "--output", output], { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(body, 202, { Location: statusPath }), + new Response(artifact.source, { + status: 200, + headers: { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": String(artifact.source.byteLength), + ETag: `"sha256:${artifact.sha256}"`, + }, + }), + ]), + }); + + assert.equal(result.status, 0); + assert.equal(JSON.parse(result.stdout).compilation.status_path, statusPath); +}); + +test("polling pins compilation identity and stops on its first failed read", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + const changed = compilationBody("running", { + compilation: { + id: OTHER_COMPILATION_ID, + status_path: `/v1/projects/${PROJECT_ID}/compilations/${OTHER_COMPILATION_ID}`, + cancel_path: `/v1/projects/${PROJECT_ID}/compilations/${OTHER_COMPILATION_ID}/cancel`, + }, + }); + /** @type {unknown[]} */ + const calls = []; + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch( + [ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + jsonResponse(changed), + ], + calls, + ), + planCompileSleep: async () => undefined, + }); + + assert.equal(calls.length, 2); + assertHandledFailure(result, "compilation_changed"); + assert.equal( + JSON.parse(result.stderr).current.compilation.id, + OTHER_COMPILATION_ID, + ); + + const unavailable = await invoke( + ["plan", "compile", "--output", "another-generated"], + { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + async () => { + throw new TypeError("canary-secret-network"); + }, + ]), + planCompileSleep: async () => undefined, + }, + ); + assertHandledFailure(unavailable, "compilation_status_unavailable"); + assert.doesNotMatch(unavailable.stderr, /canary-secret/); +}); + +test("polling distinguishes a validated unavailable status from a protocol mismatch", async (context) => { + const unavailableDirectory = remoteDirectory( + context, + "https://api.example.test", + ); + const unavailable = await invoke( + ["plan", "compile", "--output", "generated-unavailable"], + { + cwd: unavailableDirectory, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + problemResponse(503, "temporarily_unavailable", "Try later."), + ]), + planCompileSleep: async () => undefined, + }, + ); + assertHandledFailure(unavailable, "compilation_status_unavailable"); + assert.equal(JSON.parse(unavailable.stderr).status, 503); + + const invalidDirectory = remoteDirectory(context, "https://api.example.test"); + const invalid = await invoke( + ["plan", "compile", "--output", "generated-invalid"], + { + cwd: invalidDirectory, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + new Response('{"canary":"secret"}', { + status: 200, + headers: { "Content-Type": "application/json" }, + }), + ]), + planCompileSleep: async () => undefined, + }, + ); + assertHandledFailure(invalid, "invalid_compilation_status"); + assert.doesNotMatch(invalid.stderr, /canary|secret/); +}); + +test("polling enforces the bounded ten-minute deadline", async (context) => { + const cwd = remoteDirectory(context, "https://api.example.test"); + let currentTime = 0; + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + ]), + planCompileNow: () => currentTime, + planCompileSleep: async (/** @type {number} */ delayMs) => { + assert.equal(delayMs, 1_000); + currentTime = 600_000; + }, + }); + + assertHandledFailure(result, "compilation_wait_timed_out"); + assert.equal(JSON.parse(result.stderr).current.compilation.status, "queued"); +}); + +test("failed and cancelled compilations are domain failures without artifact reads", async (context) => { + for (const status of ["failed", "cancelled"]) { + const cwd = remoteDirectory(context, "https://api.example.test"); + /** @type {unknown[]} */ + const calls = []; + const body = + status === "failed" + ? compilationBody(status, { + compilation: { + failure: { + phase: "render", + code: "render_failed", + message: "The renderer failed safely.", + }, + }, + }) + : compilationBody(status); + const result = await invoke( + ["plan", "compile", "--output", `generated-${status}`], + { + cwd, + fetchFunction: sequenceFetch( + [jsonResponse(body, 202, { Location: STATUS_PATH })], + calls, + ), + }, + ); + + assert.equal(calls.length, 1); + assertHandledFailure(result, `compilation_${status}`); + assert.equal(JSON.parse(result.stderr).current.compilation.status, status); + } +}); + +test("artifact transport metadata and exact bytes are verified before parsing", async (context) => { + const validArtifact = artifactFixture(); + const cases = [ + new Response(validArtifact.source, { + status: 200, + headers: { + "Content-Type": "application/json", + "Content-Length": String(validArtifact.source.byteLength), + ETag: `"sha256:${validArtifact.sha256}"`, + }, + }), + new Response(validArtifact.source, { + status: 200, + headers: { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": String(validArtifact.source.byteLength + 1), + ETag: `"sha256:${validArtifact.sha256}"`, + }, + }), + new Response(validArtifact.source, { + status: 200, + headers: { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": `0${validArtifact.source.byteLength}`, + ETag: `"sha256:${validArtifact.sha256}"`, + }, + }), + new Response(validArtifact.source, { + status: 200, + headers: { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": String(validArtifact.source.byteLength), + ETag: `"sha256:${"0".repeat(64)}"`, + }, + }), + ]; + + for (const [index, artifactResponse] of cases.entries()) { + const cwd = remoteDirectory(context, "https://api.example.test"); + const result = await invoke( + ["plan", "compile", "--output", `generated-${index}`], + { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse( + compilationBody("succeeded", { artifact: validArtifact }), + 202, + { Location: STATUS_PATH }, + ), + artifactResponse, + ]), + }, + ); + + assertHandledFailure(result, "invalid_artifact"); + } +}); + +test("an unavailable artifact is distinct from an invalid artifact", async (context) => { + const artifact = artifactFixture(); + const cwd = remoteDirectory(context, "https://api.example.test"); + const result = await invoke(["plan", "compile", "--output", "generated"], { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("succeeded", { artifact }), 202, { + Location: STATUS_PATH, + }), + problemResponse(503, "artifact_not_ready", "Try later."), + ]), + }); + + assertHandledFailure(result, "artifact_unavailable"); + assert.equal(JSON.parse(result.stderr).status, 503); +}); + +test("a target created while compiling is preserved and never replaced", async (context) => { + const artifact = artifactFixture(); + const cwd = remoteDirectory(context, "https://api.example.test"); + const output = path.join(cwd, "generated"); + const marker = path.join(output, "belongs-to-user"); + const result = await invoke(["plan", "compile", "--output", output], { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(compilationBody("queued"), 202, { + Location: STATUS_PATH, + }), + jsonResponse(compilationBody("succeeded", { artifact })), + new Response(artifact.source, { + status: 200, + headers: { + "Content-Type": ARTIFACT_MEDIA_TYPE, + "Content-Length": String(artifact.source.byteLength), + ETag: `"sha256:${artifact.sha256}"`, + }, + }), + ]), + planCompileSleep: async () => { + mkdirSync(output); + writeFileSync(marker, "preserve me"); + }, + }); + + assertHandledFailure(result, "materialization_failed"); + assert.equal(readFileSync(marker, "utf8"), "preserve me"); + assert.deepEqual( + readdirSync(cwd) + .filter((name) => name.startsWith(".firstdraft-generated-")) + .sort(), + [], + ); +}); + +test("exact status shapes and lifecycle timestamps are required", async (context) => { + const invalidBodies = [ + { + ...compilationBody("queued"), + canary: "canary-secret", + }, + compilationBody("queued", { + compilation: { started_at: STARTED_AT }, + }), + compilationBody("running", { + compilation: { completed_at: COMPLETED_AT }, + }), + compilationBody("succeeded", { + compilation: { artifact: null }, + }), + compilationBody("failed", { + compilation: { failure: null }, + }), + ]; + + for (const [index, body] of invalidBodies.entries()) { + const cwd = remoteDirectory(context, "https://api.example.test"); + const result = await invoke( + ["plan", "compile", "--output", `generated-${index}`], + { + cwd, + fetchFunction: sequenceFetch([ + jsonResponse(body, 202, { Location: STATUS_PATH }), + ]), + }, + ); + + assertHandledFailure(result, "request_outcome_unknown"); + assert.doesNotMatch(result.stderr, /canary-secret/); + } +}); + +/** @param {string} status @param {{artifact?: ReturnType, project?: Record, compilation?: Record}} [changes] */ +function compilationBody(status, changes = {}) { + const artifact = changes.artifact; + const terminal = ["succeeded", "failed", "cancelled"].includes(status); + const started = status === "queued" ? null : STARTED_AT; + return { + project: { + id: PROJECT_ID, + graph_version: 7, + ...changes.project, + }, + compilation: { + id: COMPILATION_ID, + analysis_run_id: ANALYSIS_ID, + graph_version: 7, + status, + compiler_release: COMPILER_RELEASE, + target: TARGET, + status_path: STATUS_PATH, + cancel_path: CANCEL_PATH, + artifact: + status === "succeeded" && artifact + ? { + path: ARTIFACT_PATH, + sha256: artifact.sha256, + media_type: ARTIFACT_MEDIA_TYPE, + byte_size: artifact.source.byteLength, + } + : null, + failure: + status === "failed" + ? { + phase: "compile", + code: "compilation_failed", + message: "The compilation failed.", + } + : null, + created_at: CREATED_AT, + started_at: started, + completed_at: terminal ? COMPLETED_AT : null, + ...changes.compilation, + }, + }; +} + +function artifactFixture() { + const files = [ + artifactFile( + "app/models/movie.rb", + "class Movie < ApplicationRecord\nend\n", + 0o644, + "renderer:model", + [SUBJECT_ID], + ), + artifactFile( + "bin/setup", + Buffer.from([0, 255]), + 0o755, + "core:foundation-rails-core", + [], + ), + ]; + const metadata = files.map( + ({ path: filePath, sha256, mode, owner, source_subject_uuids }) => ({ + path: filePath, + sha256, + mode, + owner, + source_subject_uuids, + }), + ); + const manifestSha256 = sha256( + Buffer.from(JSON.stringify({ files: metadata })), + ); + const body = { + format: "firstdraft.compilation-artifact/1", + provenance: { + compilation_id: COMPILATION_ID, + project_id: PROJECT_ID, + graph_version: 7, + head_source_sha256: HEAD_SHA256, + foundation_plan: { + format: FOUNDATION_PLAN_FORMAT, + sha256: "2".repeat(64), + }, + analysis: { + id: ANALYSIS_ID, + release: "foundation-plan-rails/scalar-2026-07", + }, + compiler_release: COMPILER_RELEASE, + target: TARGET, + core: { + repository: "firstdraft/foundation-rails-core", + revision: "3".repeat(40), + sha256: "4".repeat(64), + }, + }, + manifest_sha256: manifestSha256, + files, + }; + const source = Buffer.from(JSON.stringify(body)); + return { + body, + files, + source, + sha256: sha256(source), + manifestSha256, + }; +} + +/** @param {string} filePath @param {string | Buffer} contents @param {number} mode @param {string} owner @param {string[]} subjectIds */ +function artifactFile(filePath, contents, mode, owner, subjectIds) { + const bytes = Buffer.isBuffer(contents) ? contents : Buffer.from(contents); + return { + path: filePath, + sha256: sha256(bytes), + mode, + owner, + source_subject_uuids: subjectIds, + contents_base64: bytes.toString("base64"), + }; +} + +/** @param {Buffer} value */ +function sha256(value) { + return createHash("sha256").update(value).digest("hex"); +} + +function compilationCollection() { + return `/v1/projects/${PROJECT_ID}/compilations`; +} + +/** @param {import("node:http").ServerResponse} response @param {number} status @param {unknown} body @param {Record} [headers] */ +function respondJson(response, status, body, headers = {}) { + const source = Buffer.from(JSON.stringify(body)); + response.writeHead(status, { + "Content-Type": "application/json", + "Content-Length": source.byteLength, + ...headers, + }); + response.end(source); +} + +/** @param {unknown} body @param {number} [status] @param {Record} [headers] */ +function jsonResponse(body, status = 200, headers = {}) { + return new Response(JSON.stringify(body), { + status, + headers: { "Content-Type": "application/json", ...headers }, + }); +} + +/** @param {number} status @param {string} code @param {string} detail */ +function problemResponse(status, code, detail) { + return new Response( + JSON.stringify({ + type: "about:blank", + title: status === 409 ? "Conflict" : "Service Unavailable", + status, + code, + detail, + }), + { + status, + headers: { "Content-Type": "application/problem+json" }, + }, + ); +} + +/** + * @param {(Response | (() => Promise))[]} responses + * @param {unknown[]} [calls] + * @returns {typeof globalThis.fetch} + */ +function sequenceFetch(responses, calls = []) { + return async (input, init) => { + calls.push({ input, init }); + const response = responses.shift(); + assert(response, "unexpected request"); + return typeof response === "function" ? response() : response; + }; +} + +/** @param {import("node:http").IncomingMessage} request */ +async function readRequestBody(request) { + const chunks = []; + for await (const chunk of request) chunks.push(Buffer.from(chunk)); + return Buffer.concat(chunks); +} + +/** @param {import("node:test").TestContext} context @param {import("node:http").Server} server */ +async function listen(context, server) { + await new Promise((/** @type {(value?: void) => void} */ resolve) => { + server.listen(0, "127.0.0.1", () => { + resolve(); + }); + }); + context.after( + () => + new Promise( + ( + /** @type {(value?: void) => void} */ resolve, + /** @type {(error: Error) => void} */ reject, + ) => { + server.close((error) => { + if (error) reject(error); + else resolve(); + }); + }, + ), + ); + const address = server.address(); + assert(address && typeof address === "object"); + return `http://127.0.0.1:${address.port}`; +} + +/** @param {import("node:test").TestContext} context @param {string} apiUrl */ +function remoteDirectory(context, apiUrl) { + return localDirectory(context, { + format: "firstdraft.cli-state/1", + project_id: PROJECT_ID, + api_url: apiUrl, + foundation_plan_etag: ETAG, + }); +} + +/** @param {import("node:test").TestContext} context @param {Record} state */ +function localDirectory(context, state) { + const directory = mkdtempSync(path.join(tmpdir(), "firstdraft-compile-")); + context.after(() => rmSync(directory, { recursive: true, force: true })); + mkdirSync(path.join(directory, ".firstdraft")); + writeFileSync( + path.join(directory, ".firstdraft", "state.json"), + `${JSON.stringify(state, null, 2)}\n`, + { mode: 0o600 }, + ); + return directory; +} + +/** @param {readonly string[]} argv @param {Record} [options] */ +async function invoke(argv, options = {}) { + let stdout = ""; + let stderr = ""; + const status = await run({ + argv, + stdout: { write: (text) => (stdout += text) }, + stderr: { write: (text) => (stderr += text) }, + ...options, + }); + return { status, stdout, stderr }; +} + +/** + * @param {{status: number, stdout: string, stderr: string}} result + * @param {string} error + * @param {number} [status] + */ +function assertHandledFailure(result, error, status = 1) { + assert.equal(result.status, status); + assert.equal(result.stdout, ""); + assert.equal(JSON.parse(result.stderr).error, error); +} + +function inaccessibleFetch() { + return async () => { + throw new Error("network must remain inaccessible"); + }; +} + +function inaccessibleFileSystem() { + return { + lstatSync: () => { + throw new Error("filesystem must remain inaccessible"); + }, + readFileSync: () => { + throw new Error("filesystem must remain inaccessible"); + }, + renameSync: () => { + throw new Error("filesystem must remain inaccessible"); + }, + writeFileSync: () => { + throw new Error("filesystem must remain inaccessible"); + }, + }; +} diff --git a/test/plan-init.test.js b/test/plan-init.test.js index 91a6572..ddc76ef 100644 --- a/test/plan-init.test.js +++ b/test/plan-init.test.js @@ -30,6 +30,7 @@ Commands: subject-id Generate a UUIDv7 for a new Plan subject push Send the local Foundation Plan to First Draft status Read the current whole-graph analysis status + compile Compile the accepted Plan into a new local directory Options: -h, --help Show help