Add hidden App Doctor commands

[jplhomer]

Problem

Shopify app developers need a Shopify-specific security review workflow that combines deterministic checks with structured prompts for deeper review by their coding agent. The coding-agent procedure should come from the same Shopify CLI build that owns the review artifacts rather than requiring developers to install AI Toolkit or copy a separately versioned skill.

The deterministic scanner must also treat the scanned repository and submitted findings as untrusted input. Unsupported or incomplete analysis must never be presented as a clean security grade.

Solution

Add hidden shopify app doctor commands directly to Shopify CLI.

shopify app doctor

• discovers the app root from an explicit path or nested working directory;
• runs the current 14-check deterministic catalog across supported Shopify React Router apps, theme app extensions, and structured Shopify configuration;
• uses structured TOML parsing, narrow JavaScript/TypeScript analysis, Theme Check Liquid AST analysis, and isolated package-manager audits;
• writes app-doctor-review.json with versioned semantic checks for coding-agent follow-up;
• writes an unsigned App Doctor trace using schema version 2;
• accepts --findings <file> to validate agent results and compile them into the trace;
• supports human and JSON output plus high | medium | low | none CI blocking;
• records per-check status, implementation provenance, inspected files, skipped inputs, and explicit coverage gaps;
• withholds the grade when required coverage is unsupported, unresolved, unreadable, or rejected;
• redacts matched secrets, including complete and malformed private-key material, from findings, evidence, output, and traces.

Security boundaries

• repository reads are bounded, regular-file-only, no-follow, root-contained, and revalidated against path races;
• generated artifacts use atomic, symlink-safe writes and are excluded from subsequent scan inputs;
• nested apps, dependency trees, generated output, tests, and fixtures are excluded recursively;
• Git probes resolve a trusted executable outside the scanned repository, use a sanitized environment, disable execution-capable configuration, and avoid worktree status/filter execution;
• dependency audits run in a private temporary sandbox containing only sanitized manifest data and the selected lockfile, with scripts/plugins/local dependencies removed, a trusted registry, a sanitized environment, and an explicitly selected Yarn Berry version;
• submitted agent and external findings receive strict runtime shape, provenance, path, and scanned-input validation;
• standalone instructions never infer completion from a persisted review pack, and agent fallback never instructs the agent to execute repository-controlled package-manager code.

shopify app doctor instructions

• prints instructions to stdout by default;
• supports --copy and --write <path>;
• always starts from a fresh shopify app doctor invocation;
• embeds adjacent Markdown as the source of truth, so the published CLI does not depend on runtime instruction assets.

Both commands remain callable but hidden from normal command listings while the workflow is in development. The engine lives inside @shopify/app; shopify app doctor submit remains reserved for future authenticated upload.

Risk

This adds a substantial local scanner and a new trace schema. Static analysis cannot prove an app secure, so the implementation explicitly records incomplete coverage and withholds a grade rather than reporting a false clean result. The command does not reuse the unrelated DoctorSuite release-test harness.

Top-hatting

pnpm install --frozen-lockfile
pnpm --filter @shopify/app type-check
pnpm --filter @shopify/app lint
pnpm --filter @shopify/app build
pnpm --filter @shopify/cli bundle

Run a scan:

node packages/cli/bin/run.js app doctor /path/to/app --skip-instructions
CI=1 node packages/cli/bin/run.js app doctor /path/to/app --json --skip-instructions

Print, copy, or write coding-agent instructions:

node packages/cli/bin/run.js app doctor instructions /path/to/app
node packages/cli/bin/run.js app doctor instructions /path/to/app --copy
node packages/cli/bin/run.js app doctor instructions /path/to/app --write /tmp/app-doctor.md

After completing the generated review pack:

node packages/cli/bin/run.js app doctor /path/to/app \
  --findings /path/to/app/findings.json \
  --skip-instructions

Verification

• Focused App Doctor engine, command, and service suite: 15 files, 161 tests passing.
• CLI registration suite: 3 tests passing.
• App type-check and lint pass.
• Knip passes.
• App build and CLI bundle pass.
• Prompt embedding and OCLIF manifest regeneration are clean.
• Bundled help confirms shopify app doctor and shopify app doctor instructions.
• git diff --check passes.
• All actionable findings from iterative Binks reviews were addressed.

[jplhomer]

/snapit

[github-actions]

🫰✨ Thanks @jplhomer! Your snapshot has been published to npm.

Test the snapshot by installing your package globally:

pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20260828224943

Caution

After installing, validate the version by running shopify version in your terminal.
If the versions don't match, you might have multiple global instances installed.
Use which shopify to find out which one you are running and uninstall it.

[jplhomer]

/snapit

[github-actions]

🫰✨ Thanks @jplhomer! Your snapshot has been published to npm.

Test the snapshot by installing your package globally:

pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20260831163348

Caution

After installing, validate the version by running shopify version in your terminal.
If the versions don't match, you might have multiple global instances installed.
Use which shopify to find out which one you are running and uninstall it.

[jplhomer]

/snapit

[github-actions]

🫰✨ Thanks @jplhomer! Your snapshot has been published to npm.

Test the snapshot by installing your package globally:

pnpm i -g --@shopify:registry=https://registry.npmjs.org @shopify/cli@0.0.0-snapshot-20260831165144

Caution

After installing, validate the version by running shopify version in your terminal.
If the versions don't match, you might have multiple global instances installed.
Use which shopify to find out which one you are running and uninstall it.

[nickwesselman]

Can we make the scan command just shopify app doctor for simplicity? Commands can have subcommands so this doesn't preclude shopify app doctor submit