feat(cli): add an experimental diff command by vadyvas · Pull Request #2940 · Redocly/redocly-cli

vadyvas · 2026-07-07T15:08:12Z

What/Why/How?

Adds an experimental redocly diff <base> <revision> command that compares two API descriptions and reports what was added, removed, and changed.

Structural diff for every supported specification (OpenAPI 2/3.0/3.1/3.2, AsyncAPI 2/3, Arazzo, Overlay, OpenRPC) by reusing the openapi-core type trees — no hand-written traversal per type.
Breaking-change verdicts for OpenAPI 3.x and AsyncAPI 3, from a lint-style rule registry keyed by node type. A rule is a small { id, description, visit(change, ctx) } object; every rule runs, every verdict is kept, and the worst one decides the change. The scale is binary — breaking / non-breaking — because a verdict either breaks a consumer or it does not. What cannot be judged (a $ref that now points elsewhere) is reported as breaking.
Pipeline: bundle each side → walk it with walkDocument into a flat map keyed by a stable pointer → compare the two maps in two passes → classify. Array indexes become identity keys (a parameter by in + name, a server by url), so reordering is not a change. A $ref is recorded as an attribute and never followed, so a shared component is compared once, at its own path.
Direction decides the verdict. Narrowing what a request accepts breaks the clients that send the old value; widening what a response returns breaks the clients that read it. The direction comes from the node types along the ancestor chain, never from pointer text. A component takes it from the sites that reference it, callbacks and webhooks invert it, and AsyncAPI 3 declares it outright with action: send|receive — which is why an AsyncAPI payload is judged by the same schema rules as an OpenAPI one.
Output formats: stylish, json, markdown, html, plus the lint formats (codeframe, checkstyle, codeclimate, summary, github-actions, junit) for the breaking changes, so existing CI integrations accept the report as it is.
CI gate: --fail-on breaking|none (default breaking) sets the exit code.

redocly diff v1/openapi.yaml v2/openapi.yaml
redocly diff https://example.com/openapi.yaml ./openapi.yaml --format=json
redocly diff main-openapi.yaml pr-openapi.yaml --format=github-actions
redocly diff v1.yaml v2.yaml --format=html -o diff-report.html

Nothing is added to openapi-core. The diff of this branch against main touches packages/cli, tests/e2e/diff, docs, and the changeset only. The command reuses what core already exports — bundle, walkDocument, normalizeVisitors, getTypes, detectSpec, getLineColLocation, formatProblems, dequal — and keeps everything of its own under packages/cli/src/commands/diff.

Comparison by URL or path is supported; comparison of two Git revisions is not wired up yet.

Reference

Documented in docs/@v2/commands/diff.md: the options, how the direction is decided, the rule catalog for both specifications, and the limitations (component renames, positional matching inside allOf/oneOf/anyOf, and the AsyncAPI features that have no rules yet).

The rule catalog in the docs is checked against the registries by a unit test, so a rule cannot ship without a row, and a row cannot outlive its rule.

Testing

Unit — packages/cli/src/commands/diff/__tests__. The engine invariants that the terminal output cannot show: stable pointers and identity keys, subtree collapse in the comparison, path-rename matching and its ambiguous fallback, the direction for both specification families (including a payload that references itself), verdict aggregation, and the rendered reports as snapshots.

e2e — tests/e2e/diff. One fixture per rule, named oas3-* or async3-*, each a base.yaml/revision.yaml pair differing only in what it exercises. Each test reads the verdict from the machine-readable report and snapshots the terminal one. A rule is registered per node type, so only a real run catches a rule wired to a type the walker never reports — two such gaps in the parameter rules were found this way. Mirror fixtures cover the edits that must stay non-breaking, so a rule that stops reading the direction fails there instead of passing everywhere.

Screenshots (optional)

Check yourself

This PR follows the contributing guide
All new/updated code is covered by tests
Core code changed? - Tested with other Redocly products (internal contributions only)
New package installed? - Tested in different environments (browser/node)
Documentation update has been considered

Security

The security impact of the change has been considered
Code follows company security practices and guidelines

changeset-bot · 2026-07-07T15:08:20Z

🦋 Changeset detected

Latest commit: 8e52378

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
@redocly/cli	Minor
@redocly/openapi-core	Minor
@redocly/respect-core	Minor
@redocly/client-generator	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vadyvas · 2026-07-09T09:49:22Z

@RomanHotsiy
short overview of how the diff command works and what it looks like. interested in your feedback on the design

flowchart LR
    A["base + revision"] --> C["collect ×2<br/>(walkDocument → flat maps)"] --> D["compare<br/>(set diff)"] --> E["classify<br/>(polarity + rules)"] --> F["report<br/>(stylish/json/md/html)"]
    C -. "$ref edges" .-> U[UsageIndex] -. polarity .-> E

Inputs for now are files or urls - resolved through the existing BaseResolver; comparing against git revisions isn't supported

Collect — each side (base and revision) is bundled and walked with the existing walkDocument + type trees into a flat Map<stablePointer, NodeEntry>. Pointers are stable: list items are keyed by identity (e.g. params by in+name), not array index, so reordering isn't a change; $ref are kept as attributes, so a shared schema is diffed once at its component path.
Compare — dumb two-pass set-diff over the union of keys: only in base → removed, only in revision → added, in both → shallow property diff. A removed/added subtree collapses into one change.
Classify — verdicts are binary: breaking / non-breaking. The key concept is polarity: the same change flips meaning by direction (a property becoming required breaks requests but is safe in responses). Polarity is derived from the pointer; for shared components it comes from the UsageIndex, where the component is referenced (request/response), transitively.
Rules are tiny lint-style {id, description, visit()} objects in a registry (granular, since the rule set will likely grow)
Report — stylish, json, markdown, html.

redocly diff base.yaml revision.yaml 
redocly diff base.yaml revision.yaml --fail-on=breaking --format=json

GET /pets
  ✖ breaking      changed  parameters/{query:limit} · required
      Parameter became required. (parameter-became-required)

3 breaking, 1 non-breaking.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Compare two API descriptions and report added, removed, and changed parts. Structural diff works for every supported spec type via the existing openapi-core type trees; breaking-change classification (breaking / warning / non-breaking) applies to OpenAPI 3.x. The diff engine lives entirely in the CLI package and consumes only the public @redocly/openapi-core API (walkDocument, type trees, bundle) — packages/core is untouched. Pipeline: collect each side into a flat stable-pointer map, two-pass compare into a change list, then classify with a polarity-aware lint-style rule registry (worst verdict wins). Supports stylish, json, markdown, and html output and a --fail-on CI gate. Marked [experimental]; 14 starter rules documented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

… verdicts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ons, and path-param matching Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…e case Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Adapt to the new collectSpecData signature (it now takes the document, not its parsed value) and regenerate the e2e snapshots, which still held the original flat output: they predate the two-level compat model, the per-operation grouping, the location and verdict lines, and the fail-on summary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The lint formatters in core already cover the formats CI tools expect, so map breaking changes onto lint problems and hand them to formatProblems instead of writing six more serializers. This adds codeframe, checkstyle, codeclimate, summary, github-actions, and junit to the diff command; with github-actions, every breaking change becomes an inline pull request annotation. A lint problem always carries a severity, so these formats describe breaking changes only — the full change list stays in the json format. They print to stdout, and --output now reports that clearly instead of writing nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

One test per rule, each with its own minimal base/revision pair, asserting the rule id the command should attribute the change to. Thirteen of them fail today and describe the intended contract: request body required and removed, string and numeric constraint tightening, additionalProperties, oneOf narrowing, format, the three security cases, response headers, and parameter serialization. Two of the failures are false positives rather than gaps: widening a request type is reported as breaking, and 3.0 `nullable: true` compared against 3.1 `type: [.., 'null']` reports a change although both describe the same schema. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Polarity was inferred from the pointer text, which mistook a schema property named `responses` for the context of the same name and had to give up entirely under `callbacks` and `webhooks`. It now walks the ancestors the walker recorded and reads their node types, so the direction below a callback or a webhook is flipped rather than skipped, and a property can no longer pose as a context. That also uncovered a real defect: a usage edge named the `$ref` path, which is not a node and so could never be looked up; it now names the node holding the reference. The new rules cover request bodies becoming required or disappearing, numeric and string constraints, `format`, `additionalProperties`, `oneOf`/`allOf` membership, response headers, parameter serialization, and security schemes and requirements. They share one vocabulary: a constraint moves `tighter` or `looser`, and the engine's polarity decides which of the two breaks. Two false positives are gone with them. A type is now compared as the set of values it accepts, so widening a request type is no longer breaking, and 3.0's `nullable: true` folds into 3.1's `type: [..., 'null']` so the two spellings compare as equal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The per-rule fixtures now sit beside breaking-changes and follow the same shape — base.yaml, revision.yaml and a stylish snapshot — so each rule's report is reviewable as the output a reader actually sees. Every test still names the rule id it exercises, so a regenerated snapshot cannot quietly stop covering it. Reviewing the snapshots turned up a stray label: a change on the document root rendered with an empty name before the property. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

github-actions · 2026-08-07T16:50:13Z

Performance Benchmark (Lower is Faster)

CLI Version	Bundle	Lint	Check Config
cli-latest	▓ 1.00x ± 0.01	▓ 1.00x ± 0.01	▓ 1.00x (Fastest)
cli-next	▓ 1.00x (Fastest)	▓ 1.00x (Fastest)	▓ 1.00x ± 0.01

github-actions · 2026-08-07T16:50:14Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	77.32% (🎯 77%)	12093 / 15639
🔵	Statements	77.34% (🎯 77%)	12985 / 16789
🔵	Functions	81.4% (🎯 81%)	2465 / 3028
🔵	Branches	70.66% (🎯 70%)	8914 / 12614

File Coverage

File	Stmts	Branches	Functions	Lines	Uncovered Lines
Changed Files
packages/cli/src/commands/diff/fail-on.ts	0%	0%	0%	0%	8-14
packages/cli/src/commands/diff/index.ts	0%	0%	0%	0%	49-111
packages/cli/src/commands/diff/engine/align-paths.ts	100%	95.83%	100%	100%
packages/cli/src/commands/diff/engine/collect.ts	97.61%	91.17%	100%	100%	39
packages/cli/src/commands/diff/engine/compare.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/index.ts	95.45%	66.66%	80%	95.23%	73
packages/cli/src/commands/diff/engine/locate.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/node-identity.ts	80%	66.66%	83.33%	88.88%	19, 24
packages/cli/src/commands/diff/engine/predicates.ts	86.53%	78.78%	100%	88.88%	17, 52, 58, 98, 110-114
packages/cli/src/commands/diff/engine/types.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/classify/async3.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/classify/chain.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/classify/index.ts	100%	90%	100%	100%
packages/cli/src/commands/diff/engine/classify/oas3.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/classify/polarity.ts	92.15%	86.66%	100%	92.5%	10-11, 64
packages/cli/src/commands/diff/engine/classify/usage.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/engine/classify/rules/channel.ts	25%	0%	0%	33.33%	12-17, 25-28
packages/cli/src/commands/diff/engine/classify/rules/message.ts	25%	0%	0%	33.33%	9-14, 22-25
packages/cli/src/commands/diff/engine/classify/rules/operation.ts	50%	33.33%	33.33%	55.55%	7-8, 27-30
packages/cli/src/commands/diff/engine/classify/rules/parameter.ts	55.55%	40%	80%	55%	14-18, 28-33, 45, 58-61
packages/cli/src/commands/diff/engine/classify/rules/ref.ts	87.5%	75%	100%	83.33%	14-16
packages/cli/src/commands/diff/engine/classify/rules/request-body.ts	20%	0%	0%	25%	8-12, 21-22
packages/cli/src/commands/diff/engine/classify/rules/response.ts	41.66%	10%	33.33%	44.44%	8, 16-17, 27-32
packages/cli/src/commands/diff/engine/classify/rules/schema.ts	52.43%	36.48%	81.81%	52.23%	22-24, 43, 46, 61, 74, 83-87, 97-101, 111-120, 135-141, 181-195
packages/cli/src/commands/diff/engine/classify/rules/security.ts	17.85%	0%	0%	23.8%	13-15, 25-27, 44-54, 62-63, 72-75
packages/cli/src/commands/diff/engine/classify/rules/server.ts	25%	0%	0%	33.33%	9-12
packages/cli/src/commands/diff/serializers/change-side.ts	100%	66.66%	100%	100%
packages/cli/src/commands/diff/serializers/html.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/serializers/json.ts	0%	100%	0%	0%	4
packages/cli/src/commands/diff/serializers/markdown.ts	100%	100%	100%	100%
packages/cli/src/commands/diff/serializers/problems.ts	100%	76.92%	100%	100%
packages/cli/src/commands/diff/serializers/stylish.ts	93.75%	73.68%	87.5%	97.67%	56, 62, 81

Generated in workflow #11270 for commit 8e52378 by the Vitest Coverage Report Action

Follows up the code review on the diff command. Rules and engine: - Revert the `html` addition to core's `OutputFormat`. `formatProblems` has no `case 'html'` and no `default`, so it typechecked and silently did nothing. The diff command spells out its own formats instead. - Report `security-requirement-added` when an explicitly empty `security: []` list gets its first entry, and add `security-scopes-added`. - Register `parameter-removed` and `parameter-added-required` for `ParameterList` as well: the last parameter of an operation leaves with the whole `parameters` list, and the first one arrives with it. - Invert the direction once per nesting level under `callbacks`, so a callback inside a callback points the right way. - Add six AsyncAPI 3 rules (`channel-removed`, `channel-address-changed`, `message-removed`, `message-content-type-changed`, `operation-action-changed`, `server-removed`) and reuse the schema rules for payloads. AsyncAPI declares the direction with `action`, so it gets its own resolver; a channel takes its direction from the operations that reference it. - Drop `isScalarArray([])`, which reported an empty list twice. - Use core's `dequal` and `isAbsoluteUrl` instead of local equivalents. - Fix double escaping in the markdown report, which broke the rule-id code span. - Reject `--output` for stdout-only formats before bundling both documents. Tests: - One `__tests__` folder, as every other command in the package has. - Snapshot the collected map, the emitted changes, the verdicts and the reports, so the whole output stays visible. - Drop tests that asserted a dependency's behaviour or a one-line helper. - Move per-rule coverage to e2e: one fixture per rule, prefixed `oas3-` or `async3-`, asserted on the machine-readable report. Docs: rewrite the command page in simpler English and stop leaking the internal word "polarity" into user-facing text.

+  return value
+    .replace(/\|/g, '\\|')
+    .replace(/`/g, '\\`')


+  return value
+    .replace(/\|/g, '\\|')


github-advanced-security AI found potential problems Jul 7, 2026

View reviewed changes

Comment thread packages/cli/src/commands/diff/serializers/markdown.ts Fixed

vadyvas self-assigned this Aug 7, 2026

vadyvas and others added 19 commits August 7, 2026 18:30

docs: add diff command design spec

649c795

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: add diff command implementation plan

b35c879

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: isolate diff engine in CLI package, mark command experimental

bae9999

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs: add diff command improvements plan

df0d63d

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

refactor(cli): simplify diff compat model to breaking/non-breaking

5a03534

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(cli): keep every rule verdict on diff changes

9945f46

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(cli): report diff --fail-on failure visibly and print execution time

e875ff4

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

refactor(cli): derive diff output format from core OutputFormat

429fd2a

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(cli): attach file/line/col locations to diff change sides

25c8896

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(cli): group diff stylish output per operation with locations and…

07006f1

… verdicts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

feat(cli): match renamed path parameters at the diff compare stage

c7b429c

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs: update diff command docs for two-level compat, verdicts, locati…

fac2bd3

…ons, and path-param matching Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(cli): polish diff stylish labels and document callback rename edg…

75834bc

…e case Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

vadyvas force-pushed the feat/diff-command branch from fe59519 to 7d66075 Compare August 7, 2026 16:47

vadyvas changed the title ~~Feat/diff command~~ feat(cli): add an experimental diff command Aug 10, 2026

github-advanced-security AI found potential problems Aug 10, 2026

View reviewed changes

Comment thread packages/cli/src/commands/diff/serializers/markdown.ts

Comment on lines +12 to +14

return value

.replace(/\|/g, '\\|')

.replace(/`/g, '\\`')

Comment thread packages/cli/src/commands/diff/serializers/markdown.ts

Comment on lines +12 to +13

return value

.replace(/\|/g, '\\|')

Navigation Menu

Uh oh!

feat(cli): add an experimental diff command - #2940

feat(cli): add an experimental diff command#2940
vadyvas wants to merge 20 commits into
mainfrom
feat/diff-command

vadyvas commented Jul 7, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

vadyvas commented Jul 9, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 7, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

vadyvas commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What/Why/How?

Reference

Testing

Screenshots (optional)

Check yourself

Security

Uh oh!

changeset-bot Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Uh oh!

vadyvas commented Jul 9, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Benchmark (Lower is Faster)

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vadyvas commented Jul 7, 2026 •

edited

Loading

changeset-bot Bot commented Jul 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading