Repository navigation
Report the share of SDK code that comes from Snipsync - #5426
Conversation
Add bin/report-snipsync-coverage.js (yarn report:snipsync-coverage), which counts non-blank lines in SDK-language code blocks under docs/ and reports how many come from Snipsync, by section, language, and source repository. It reads the working tree or any commit (without a checkout), compares two commits, and rebuilds the trend from git history as CSV. The Snipsync Coverage workflow runs the comparison on pull requests and comments only when SDK line counts change. On main it writes the numbers to the job summary. The code block parser moves to bin/code-blocks.js so the sample checkers can share it.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
📖 Docs PR preview linksThis PR does not change any pages in /docs. If you make updates, links to the modified pages will appear here. |
There was a problem hiding this comment.
🔵 Needs a closer look
Repository review policy excludes pull requests with no changes under docs/, so these tooling and workflow changes require human review.
0 open findings
What changed in this PR
Adds tooling and automation to report how much SDK sample code is managed by Snipsync.
Changes:
- Adds code-block parsing, coverage reporting, history, comparison, and tests.
- Adds a non-blocking GitHub Actions workflow and failure notification.
- Documents the report and exposes it as a package script.
| File | Description |
|---|---|
AGENTS.md |
Documents Snipsync coverage guidance. |
readme/UTILITIES.md |
Documents report behavior and usage. |
readme/AUTOMATIONS.md |
Lists the new workflow. |
package.json |
Adds the report script. |
bin/code-blocks.js |
Extracts fenced blocks and Snipsync metadata. |
bin/report-snipsync-coverage.js |
Implements coverage reports and history. |
bin/report-snipsync-coverage.test.js |
Tests parsing, counting, and formatting. |
.github/workflows/snipsync-coverage.yml |
Runs reporting in CI. |
.github/workflows/notify-automation-failures.yml |
Monitors workflow failures. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Adds a metric for how much of the SDK code in
docs/comes from Snipsync, so we can track the move of hand-written samples intodocumentation-sdk-code-examplesand other CI-built sample repos. This lands before any of that migration, so the starting point is on record.What it measures
Non-blank lines inside fenced code blocks tagged with an SDK language (Python, TypeScript/JavaScript, Go, Java, C#, Ruby, PHP, Rust). A block counts as synced when it sits between
SNIPSTARTandSNIPENDmarkers, in either comment form. The Snipsync source-link line isn't counted as code, but its URL attributes synced lines to a source repository. A second line reports every fenced block except mermaid.Changes
bin/report-snipsync-coverage.js(yarn report:snipsync-coverage): breaks the numbers down by section (develop/split by SDK), language, and source repository. It reads the working tree or any commit without a checkout, compares two commits (--base), and rebuilds the trend as CSV (--history). Output is plain text,--json, or--markdown.bin/code-blocks.js: the code block parser from Add advisory checker for hand-written Python samples #5413, plus origin capture. Once both are merged, Add advisory checker for hand-written Python samples #5413'sbin/code-samples.jscan import it instead of keeping its own copy..github/workflows/snipsync-coverage.yml(Snipsync Coverage): on pull requests that touchdocs/, compares the merge commit with the base it merges into and writes the result to the job summary. It keeps one PR comment up to date and only posts when SDK line counts change. Onmainit writes the job summary. It never fails. Nothing is stored, because--historyrebuilds the trend from git.notify-automation-failures.ymlfor the push-to-mainruns, plus docs inreadme/UTILITIES.md,readme/AUTOMATIONS.md, andAGENTS.md.Baseline (main at
1c2c9d2)15.0% of SDK code lines come from Snipsync: 5,611 of 37,346.
Monthly trend (
yarn report:snipsync-coverage --history --since 2026-01-01 --every month):The July drop is /design-patterns and /guides arriving with about 13,000 inline lines. Synced lines also fell by 180 in September. Several PRs replaced synced blocks with hand-written ones: the Nexus feature guides (#5203, −304 lines across four SDKs), .NET and Java Lambda Workers (#5229 and others, −162), and OpenAI Agents 1.0 (#5352, −70). The PR comment will now flag that kind of change.
Testing
node --test bin/report-snipsync-coverage.test.js: 28 tests. They cover fence edge cases (tabs, indentation,~~~, nested fences, unclosed comments), both wrapper forms, origin parsing, language and section bucketing,git cat-file --batchparsing with multi-byte text, period sampling, and comparison formatting.Local runs: working tree in 0.3s,
--ref HEAD~50,--base, and--historyfrom January in about 1s.Throwaway PR THROWAWAY: test Snipsync Coverage workflow (do not merge) #5418 (closed) exercised the workflow end to end:
The run surfaced one display bug: the change was calculated before rounding, so the comment read "15.0% → 15.1% (±0.0 pts)". It's fixed in the second commit.
This PR changes no docs, so its own run should post no comment.