Re-enable integration tests

.github/copilot-instructions.md

@@ -0,0 +1,28 @@
+# Copilot Working Contract For Integration Analysis
+
+This repository uses an artifact-first workflow for integration command triage.
+
+## Non-negotiable Rules
+- Prefer deterministic, repository-auditable tooling over ad-hoc shell snippets.
+- Use TypeScript tools under `src/test/integration/tools/` for analysis workflows.
+- Treat integration NDJSON logs as source of truth for command identity and outcomes.
+- Do not rediscover command lists by scanning source files when NDJSON already contains `commandId` and `sourceFile`.
+- Do not use regex heuristics to infer categories if `failureCategory` is present in machine logs.
+- Fail fast on missing required artifact fields; do not silently degrade.
+- Keep data flow one-way: producer test -> machine log -> analyzer -> reports.
+
+## Integration Triage Pipeline
+1. Run integration matrix and emit NDJSON events.
+2. Run analyzer tool(s) that consume NDJSON and map commands to API descriptors/OpenAPI operations.
+3. Generate machine-readable JSON and human-readable markdown outputs.
+4. Triage failures by category using analyzer outputs.
+
+## Tooling Boundaries
+- Avoid importing runtime discovery modules in standalone analyzers if they pull config from dist-relative paths.
+- Keep analyzer dependencies explicit and minimal.
+- Record provenance in outputs (input log, openapi path, generation timestamp).
+
+## Behavior Expectations
+- Ask one clarifying question if requirements are ambiguous and would change output contract.
+- Prefer small, reversible diffs that preserve existing architecture constraints.
+- When constraints conflict with quick fixes, prioritize architecture constraints.

.gitignore

@@ -65,3 +65,9 @@ atlassian-ide-plugin.xml
 
 # Editor-based Rest Client
 .idea/httpRequests
+
+# integration testing artifacts
+openapi.json
+run-all-commands.ndjson
+command-endpoint-map.json
+command-endpoint-map.md

docs/integration-tests-reactivation-plan.md

@@ -0,0 +1,273 @@
+# Integration Tests Reactivation Plan (No HTTP Mocking)
+
+## Goal
+Run an integration command matrix for all CLI commands, not just selected skipped tests.
+Use static code analysis to discover commands and synthesize runnable argument sets.
+Reduce non-actionable failures so remaining failures indicate real command or contract issues.
+
+## Scope Update (2026-07-29)
+- Previous scope (reactivate only two tests) is replaced.
+- Primary scope is now run-all-commands integration coverage.
+- Command discovery and invocation quality are now the main workstream.
+
+## Current Baseline (Latest Full Run)
+- Total commands executed: 185
+- Successful: 100
+- Failed: 85
+
+Top failure signals from testrun_manual.log:
+- Missing required arg: 20
+- Missing required flag: 14
+- Exactly one of required options: 6
+- Interactive input required: 3
+- Nonexistent flag: 1
+- Resource precondition failures (not found/does not exist): 26
+- Contract-shape/runtime issues (`Invalid Version`, iterable/shape/type errors): remaining bucket
+
+Conclusion:
+- Largest improvement potential is discovery and invocation synthesis.
+- Next milestone is lowering argument-misuse failures to isolate real product issues.
+
+## Progress Update (2026-07-30)
+- ARG_MISUSE failures are now at 0.
+- Discovery hardening for argument and required-flag synthesis appears complete.
+- Integration runs already surfaced one minor real command bug.
+- Task 0 has been completed in parallel: a fully automated Mockoon environment is now running, and a separate repository was created to conserve and share this setup.
+
+## Constraints
+- No nock and no per-test HTTP stubbing.
+- HTTP must flow through `MITTWALD_API_BASE_URL`.
+- Deterministic and CI-compatible runs only.
+
+## Mock Target Position
+- Programmatic Mockoon runtime orchestration is active and automated.
+- The environment is already integrated into full automated runs.
+- A separate repository exists to preserve and share the Mockoon environment.
+
+## Immediate Next Task: Discovery Hardening
+Owner: Code Agent  
+Status: In Progress (Argument Hardening Largely Complete)
+
+Actions:
+1. Replace the current discovery flow in src/test/integration/command-discovery.ts with a new static schema-driven discovery flow that satisfies required args and required flags with type-aware defaults. (Done)
+2. Add deterministic handling for mutually exclusive requirements (exactly-one groups). (Done)
+3. Add interactive-command policy:
+   - provide non-interactive flags when supported, otherwise classify as INTERACTIVE_REQUIRED.
+4. Add stale example detection and fallback when example flags are invalid.
+5. Add command-specific invocation profiles for highest-failure domains.
+6. Replace runner input wiring so run-all consumes only the new synthesized invocation payload from discovery (single contract, no dual path).
+7. Emit failure taxonomy in run summary:
+   - ARG_MISUSE
+   - INTERACTIVE_REQUIRED
+   - RESOURCE_PRECONDITION
+   - CONTRACT_SHAPE
+   - COMMAND_BUG
+
+Acceptance Criteria:
+- Significant reduction in ARG_MISUSE failures versus 2026-07-29 baseline.
+- Interactive-only commands are deterministically handled or explicitly classified.
+- Stale flag/example failures drop to zero.
+- Residual failures are clearer and mostly actionable as real command or contract issues.
+
+Technical Design Blueprint (implementation-ready):
+
+Greenfield rule for this task:
+- No backward compatibility layer is required for discovery internals.
+- Discovery and the run-all integration test must share one explicit new contract in this workstream.
+
+Hard constraints:
+- No dual-path execution (old and new discovery paths in parallel) in final implementation.
+- No compatibility adapters for legacy discovery output.
+- Remove or retire superseded discovery wiring once the new contract is in place.
+
+1. Discovery Output Model (new normalized schema)
+- Expand discovery output from plain runnable args to a normalized per-command model:
+   - commandId
+   - sourceFile
+   - commandTokens
+   - parsedArgs: positional args with required, default, and inferred placeholder kind
+   - parsedFlags: flags with required, type, options, multiple, default, exactlyOne, exclusive, dependsOn
+   - interactiveSignals: static interactive hints from source scan
+   - invocationProfilesApplied: profile IDs applied to this command
+- Define a single synthesized invocation payload consumed by the integration runner.
+
+2. Static Metadata Extraction Strategy (no help command execution)
+- Resolve command metadata directly from source and composition patterns:
+   - inline static args/static flags
+   - spread imports and shared flag sets (for example project/app/resource flag sets)
+   - flag factory invocations (for example flag definitions with required and constraint options)
+- Stop relying on help output parsing completely.
+- Add extraction diagnostics to report unresolved spreads/factories explicitly (instead of silently degrading).
+
+3. Deterministic Arg/Flag Synthesis Engine
+- Build runnable invocation in fixed passes:
+   1) seed command tokens
+   2) satisfy required positional args
+   3) satisfy required flags
+   4) resolve exactly-one and exclusive groups
+   5) close dependency edges (dependsOn)
+   6) apply profile overrides
+- Type-aware placeholder mapping (deterministic defaults):
+   - uuid/id -> 00000000-0000-4000-8000-000000000000
+   - email -> integration@example.com
+   - url/uri -> https://example.com
+   - duration/ttl/interval -> 1h (or command-specific profile value)
+   - enum options -> first safe option unless profile overrides
+   - file/directory -> deterministic local temp-safe path values
+
+4. Constraint Resolution Rules (precedence)
+- Precedence order when constraints collide:
+   1) command-specific profile rule
+   2) explicit example-derived value (if validated)
+   3) deterministic global heuristic
+   4) fail classification as ARG_MISUSE (never randomize)
+- Exactly-one policy:
+   - choose the option with the smallest dependency closure
+   - prefer non-interactive branch when one branch requires interaction
+   - prefer context-compatible branch (project scoped over org scoped when project context exists)
+- Exclusive policy:
+   - if both appear after merge, keep higher-precedence value and drop the other deterministically.
+
+5. Interactive Command Policy
+- Static detection signals include:
+   - addInput, addSelect, addConfirmation usage
+   - editor fallback behavior
+   - stdin-dependent branches
+- Decision contract per command:
+   - NON_INTERACTIVE_RESOLVED: inject supported non-interactive flags/values
+   - INTERACTIVE_REQUIRED: classify and skip execution attempt
+- Baseline targeted substitutions:
+   - destructive confirmation flows -> --force where supported
+   - consent prompts -> --consent where supported
+   - password prompts -> explicit password flags where supported
+   - passphrase prompts -> no-passphrase where supported
+   - project-type selections -> explicit override-type where supported
+
+6. Example Validation and Stale Example Fallback
+- Validate parsed examples against extracted schema before selecting them:
+   - unknown flags
+   - missing required args/flags
+   - exactly-one/exclusive/dependsOn violations
+- If invalid, mark source as stale-example and synthesize from schema/profile.
+- Add stale example counters to run summary.
+
+7. Command-Specific Invocation Profile Schema
+- Add a small declarative profile registry keyed by commandId or prefix.
+- Profile fields:
+   - match: exact commandId or prefix
+   - requiredFlagDefaults: map of flag -> value
+   - requiredArgDefaults: map of arg -> value
+   - exactlyOneChoice: map of constraint group -> preferred member
+   - interactivePolicy: resolve | classify
+   - disableExampleSource: boolean
+   - notes: short rationale
+- Initial high-failure domains to profile first:
+   - backup create / backup schedule create
+   - cronjob create
+   - extension install / extension list-installed
+   - sftp-user create / ssh-user create
+   - database mysql create / mysql user create
+   - ddev init
+   - login token
+
+8. Failure Taxonomy Contract for Run Summary
+- Emit machine-countable categories per command failure:
+   - ARG_MISUSE
+   - INTERACTIVE_REQUIRED
+   - RESOURCE_PRECONDITION
+   - CONTRACT_SHAPE
+   - COMMAND_BUG
+- Add deterministic signature rules to classify stderr/stdout patterns.
+- Print summary table with counts, plus sample command IDs per category.
+
+9. Delivery Phases and Validation Gates
+- Phase A: schema extractor and fixture tests for representative commands.
+- Phase B: synthesis engine with constraint solver.
+- Phase C: interactive policy and initial profile registry.
+- Phase D: example validation and stale fallback.
+- Phase E: runner wiring switch to the new payload as the only path; remove superseded discovery wiring.
+- Phase F: taxonomy emission and baseline comparison report.
+
+10. Definition of Success for this task
+- ARG_MISUSE category reduced significantly from baseline.
+- INTERACTIVE_REQUIRED is explicit and deterministic (no prompt-time crashes).
+- Nonexistent/stale example flag failures are zero.
+- Remaining failures are primarily RESOURCE_PRECONDITION, CONTRACT_SHAPE, or COMMAND_BUG and actionable.
+
+## Backlog Tasks (Reordered)
+### Task 0: Mockoon Runtime Orchestration (Completed)
+- Programmatic startup/teardown is in place and actively used.
+- Automated Mockoon-backed full runs are already operational.
+- The setup is conserved in a separate shareable repository.
+
+### Task 4: Dedicated Integration Scripts
+- Add `test:integration:commands` and optional Mockoon-backed variant.
+- Keep unit and existing test flows unchanged.
+
+### Task 5: CI Hardening
+- Stabilize retries, resource isolation, and cleanup.
+- Promote run-all matrix to optional or gated CI step once discovery quality improves.
+
+### Task 7: Contract-Shape Guardrails
+- Add preflight checks and endpoint-shape diagnostics so contract mismatches are isolated from discovery quality metrics.
+
+### Task 8: Create waiver mechanics, move config to JSON (Completed)
+- Status: Implemented (2026-07-30)
+- Config now lives in visible JSON artifacts:
+   - `src/test/integration/config/invocation-profiles.json`
+   - `src/test/integration/config/command-waivers.json`
+- Runner now enforces strict waiver mechanics:
+   - a command is skipped only when an explicit waiver exists
+   - every skipped command must have a non-empty reason and category
+   - interactive-required commands without waiver are treated as failures
+   - stale waivers (waiver for undiscovered command) fail the run
+
+### Task 9: Agent Behavior Overhaul (Artifact-First)
+- Status: Planned (2026-07-31)
+- Purpose: prevent heuristic drift and force deterministic, auditable follow-up analysis.
+
+Hard rules for analysis tooling:
+- The integration runner is the only producer of command discovery facts.
+- Follow-up analyzers must consume machine artifacts; they must not rediscover commands from source tree scans.
+- No regex reconstruction of command identity when machine events already provide commandId and sourceFile.
+- Standalone analyzer tools must not import test discovery modules that load runtime config from dist-relative paths.
+- Generated outputs must always include provenance fields (log path, schema inputs, timestamp, tool version).
+
+Canonical data flow:
+1. `run-all-commands.test.ts` emits NDJSON events.
+2. `command-start` events provide discovery metadata and sourceFile.
+3. `command-result` events provide status and failureCategory.
+4. analyzer joins command metadata + result category + descriptor mapping + OpenAPI operation data.
+5. analyzer writes deterministic artifacts (json + markdown).
+
+Required NDJSON contract for analyzer inputs:
+- `command-start`:
+   - commandId
+   - sourceFile
+   - parsedArgs
+   - parsedFlags
+   - interactiveSignals
+   - invocationProfilesApplied
+   - extractionDiagnostics
+- `command-result`:
+   - commandId
+   - status
+   - failureCategory
+
+Acceptance criteria:
+- Analyzer fails fast when required machine-log fields are missing.
+- Analyzer behavior is stable across repeated runs with identical inputs.
+- Endpoint mapping is descriptor-first and OpenAPI-backed, without command rediscovery fallbacks.
+- Category slices (for example RESOURCE_PRECONDITION) are computed from `command-result` events only.
+
+Operator workflow (human or agent):
+1. Run integration matrix and produce NDJSON log.
+2. Run command-endpoint analyzer against that NDJSON log.
+3. Filter by failure category and review endpoint coverage/deprecation metadata.
+4. Triage into: product bug, contract-shape issue, precondition gap, or invocation profile gap.
+5. Update profiles/waivers/tooling, then rerun end-to-end.
+
+## Definition Of Done
+- Integration command matrix runs against configurable HTTP target without in-test mocking.
+- Discovery quality is improved enough that failures are predominantly real issues, not invocation noise.
+- Failure taxonomy is reported and used for follow-up prioritization.

package.json

@@ -38,7 +38,9 @@
     "test:format": "yarn lint && yarn format:prettier --check",
     "test:licenses": "yarn license-check --summary --unknown --failOn 'UNLICENSED;UNKNOWN'",
     "test:readme": "yarn generate:readme && git diff --exit-code README.md docs/*.md",
-    "test:unit": "NODE_NO_WARNINGS=1 yarn node --experimental-vm-modules $(yarn bin jest) ./src"
+    "test:unit": "NODE_NO_WARNINGS=1 yarn node --experimental-vm-modules $(yarn bin jest) ./src",
+    "tool:integration:generate-command-endpoint-map": "yarn compile && node dist/test/integration/tools/generate-command-endpoint-map.js",
+    "tool:integration:generate-resource-precondition-map": "yarn compile && node dist/test/integration/tools/generate-command-endpoint-map.js --category RESOURCE_PRECONDITION"
   },
   "files": [
     ".deps",

src/commands/app/database/link.tsx

@@ -71,7 +71,7 @@
     });
 
     await process.runStep("linking database", async () => {
-      const response = await this.apiClient.app.linkDatabase({
+      const response = await this.apiClient.app.linkDatabase({  // XXX: deprecated?! Should use UPDATE on app installation instead!
         appInstallationId,
         data: {
           databaseId,

src/commands/backup/download.tsx

@@ -135,7 +135,7 @@ export class Download extends ExecRenderBaseCommand<typeof Download, Result> {
           }
 
           return null;
-        }, Duration.fromString("1h"));
+        }, Duration.fromString("1h")); // XXX: may i have a word here, too?!
       },
     );