Skip to content

release: SkillSpector 2.12.0 - #550

Open
mohgupta-ship-it wants to merge 61 commits into
mainfrom
codex/release-2.11.3
Open

mohgupta-ship-it wants to merge 61 commits into
mainfrom
codex/release-2.11.3

Conversation

@mohgupta-ship-it

@mohgupta-ship-it mohgupta-ship-it commented Sep 15, 2026

Copy link
Copy Markdown
Member

Prepare SkillSpector 2.12.0 from v2.11.2, with the package version, lockfile, changelog, and release notes covering 65 merged PRs through 224ba292d91b9e5fc59398fc38ab8971f9ab01ec.

The release remains 2.12.0 because it adds CLI controls, integrations, LLM provenance, dependency-source analysis, GitHub subdirectory inputs, and other capabilities alongside security and correctness fixes. The latest 18 merges are #576, #556, #551, #580, #383, #584, #600, #560, #561, #542, #546, #541, #544, #539, #588, #591, #607, and #597. Migration requirements and changes to findings, completeness, and installation decisions are detailed below.

Included changes

Capabilities and providers

Security and completeness

Accuracy, output, and performance

Compatibility and migration

  • Report consumers should tolerate additive provenance fields and new ledger reasons analyzer_load_error and transitive_child_scan_failed, preserve aggregate failure/omission evidence, and distinguish JSON zero-based columns from SARIF one-based Unicode-codepoint columns.
  • LLM seeds must fit signed 64-bit integers. Source revision can be supplied using SKILLSPECTOR_BUILD_REVISION; unavailable revision data stays unknown, and provenance is not a reproducibility guarantee.
  • SC10 may flag intentional private registries as HIGH; review and use the existing baseline workflow for accepted cases. There is no registry-reputation or organization-allowlist check. GHSA normalization and added detection can increase scores/findings.
  • Description activation analysis falls back only when explicit legacy triggers are absent/empty. Compact prompt numbering is opt-in. GitHub tree inputs require an advertised branch/tag rather than an arbitrary commit SHA.
  • The passive-PNG AE1 exception removes a false evasion finding, not the underlying incomplete coverage or strict installation block. Resolved occurrence-column and recursive Markdown-output limitations are no longer listed as open.
  • OpenCode users must install authenticated 1.18.31; the earlier candidate's 1.18.30 pin is no longer accepted. The native OpenCode tool is copied from .opencode/ in a checkout and is not installed by the wheel.
  • Pi extension users must install SkillSpector in the extension's .venv or set SKILLSPECTOR_BIN to an existing absolute executable path. Ambient PATH lookup is removed, and reports must be written within the current workspace. These restrictions do not change the standalone CLI or OpenCode tool's output-path contract.
  • Local registry files are capped at 16 MiB, 64 nesting levels, and 10,000 combined server/package/remote records. Baselines are capped at 2 MiB, 64 nesting levels, and 10,000 rules/fingerprints, with additional YAML node/scalar/alias bounds.
  • Consumers should recognize reference_missing separately from reference_unresolved. MCP safe_to_install can be true for a missing-reference-only caveat even though the report remains incomplete/CAUTION; partially inspected files, ambiguity, and requested semantic failures still block.
  • TP4 fences, reflective Python access, and environment bytecode can add findings or semantic work. Paragraph-boundary and letter-spacing corrections can remove false findings. Unmodeled hook payloads remain incomplete/non-SAFE.
  • Transient provider failures can incur bounded additional requests and backoff within the workflow deadline; live-provider cost and latency depend on the configured service.
  • Interactive progress uses stderr; JSON and SARIF stay parseable on stdout. Forge /blob/ scans now analyze raw file bytes.
  • --fail-on-findings is opt-in. Valid model-registry overrides now affect CLI-provider token budgets. The static allowance defaults to 300 seconds per artifact within the existing workflow deadline; set SKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACT=30 to retain the earlier allowance.
  • Third-party dependency versions are unchanged from v2.11.2. Use --no-llm for intentional static-only scans; unavailable or incomplete requested semantic work remains incomplete.

Validation

  • Main baseline: 224ba292d91b9e5fc59398fc38ab8971f9ab01ec (fix(analyzer): distinguish format-only opaque references in AE1 #597); all 65 merged PRs since v2.11.2 are cataloged.
  • Release commit: 41fca42e980d955d355b717a93a30831c26b284c.
  • Python 3.12 suite with coverage, excluding integration and live-provider tests: 6,843 passed, 14 skipped, 134 deselected, 4 expected failures; 90% coverage (767.13 seconds).
  • OpenCode and Pi extension tests: 65 passed.
  • Selected static integration tests for opaque-reference reporting and dependency-source behavior: 95 passed, 8 deselected, without live providers.
  • Locked dependency installation, Ruff lint/format, CLI version, release-helper dry run, wheel/source builds, Twine validation, and whitespace checks: passed.
  • Hosted validation: all six checks passed in CI run 35820158876: Python tests, lint, extension tests, Docker smoke, changed-file checks, and DCO. Previous run 35624899510 applies only to the earlier candidate and is not certification for these 18 additional merges.
  • CI uses GitHub test-merge commit 340f81bc9ff36cde656893de3941edd5d2436334, whose parents are the recorded main baseline and release head; its tree fc50629557ef61b9821b941576202796d3857a24 is identical to the release head's tree.

Validation applies to the commit recorded here. Previous candidate test counts and workflow runs do not certify subsequent main merges.

Known limitations and release status

  • Instrumented AST fallback can raise TypeError when an optional end column is absent, although normal parsed files pass.
  • Letter-spacing reconstruction remains incomplete for newlines and some alternating-width short runs. Companion-context PE3/RA1 variants can still be incorrectly downgraded or missed.
  • Recursive root exclusions lack public scope-exclusion metadata.
  • Consumer summary grouping can merge distinct fingerprints or same-line occurrences. Raw and normalized views can duplicate a P9 finding, and consumer archive-path handling can replace original finding detail with an execution failure; existing gates still block those cases.
  • OpenCode model availability, authentication, and rate limits remain external; token-usage accounting is unavailable. Multilingual batch gap-fill qualification with live providers remains pending.
  • Unmerged runtime-selected-command and Markdown-reference fixes in fix: keep runtime-selected printf reconstruction incomplete #514 and fix: preserve complete Markdown reference destinations #553 are not included.
  • Deployment/provider, Windows, published-artifact, downstream adoption, and independent release sign-off remain separate release gates. Existing requested changes still require review.

No 2.12.0 release has been published.

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it mohgupta-ship-it added the release:publish Marks an OSS release pull request for publication label Sep 15, 2026
Comment thread docs/release/skillspector-2.12.0.md
@rng1995
rng1995 marked this pull request as ready for review September 16, 2026 16:21
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@rng1995 rng1995 changed the title release: SkillSpector 2.11.3 release: SkillSpector 2.12.0 Sep 16, 2026
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
@mohgupta-ship-it

Copy link
Copy Markdown
Member Author

Codex on behalf of Mohit Gupta — independent qualification update, 16 September 2026.

The freshly built 2.12.0 wheel from 65ca9bdd289e70ae6e413c205c7530c86e6b39aa has SHA256 bcb03b0aae2266aff57bac6e3fd0a5c51e83b64f866c79193c7862912aed877d. All 98 installed Python modules and 111 package files match that immutable source. The later 5134f62458fa235b487e823d1c0472be54678295 revision changes only the changelog/release notes: all package files, dependencies and build inputs match, so the runtime evidence applies by source equivalence. Artifact hashes remain tied to the actual built bytes.

Fresh results after the AS3 update:

Check Result
Focused scanner/release-helper tests 283 passed
Exact installed CLI/MCP and AS3 controls 9 passed
Wheel/sdist build and Twine Passed
Frozen 54-case edge corpus 39 passed / 15 failed
Frozen 32-case golden corpus 27 passed / 5 failed
Expanded downstream consumer 1,156 passed / 1 failed: existing PE3 read-only Docker identity-mount expectation
Real renderer 29 passed
Network-disabled downstream CI sanity 29/29 commands passed; 281 Python passes, 3 existing runtime-dependent skips

Both frozen corpora retain exactly the previous failure sets. Their behavioral expectations and fixture bytes were unchanged; only candidate identity/version provenance was refreshed. The consumer policy/limit assertions were retained. An initial provider-enabled consumer harness attempt was terminated and excluded; the reported counts come from the subsequent run with the established sanitized environment.

Draft #563 separately fixes unsupported-primary and pure/mixed newline completeness gaps. At c6aa326, it passes 5,292 required tests, 90% coverage, lint/format, packaging and local Docker smoke; the installed edge corpus improves to 43/54, with no new failures, and 18/18 CLI/MCP coexistence controls pass. Scratch integration with #514 at 63658bc and #553 at 18c897a passed 1,257 focused tests after resolving one small adjacency conflict. None of these open fixes is included in this release PR.

Current-head hosted CI/review and the remaining production/provider, Windows, published-artifact and adoption gates are still required. These results do not constitute release sign-off; no release or production pin was changed.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Reviewed current head e6bcf1044b781d9a28ae537cd8a56147edc9afa4, including the complete version/changelog/release-note diff, candidate history, existing review thread, version/lock consistency, component source identity, exact-head checks, and the stated release gates. The package and lock versions agree and all five hosted checks pass.

Changes are requested because two security guarantees in the release notes are not true of this exact candidate. Its letter-spacing implementation is byte-identical to the currently reviewed #470 implementation and still permits alternating-width spacing to evade P3/P4 and AE6. Its companion-classification analyzer blobs are identical to the unresolved #547 implementation, where four contextual fail-open paths remain. Either correct and requalify those implementations before publication or describe the precise limitations without claiming the risky cases remain covered. Deployment/provider, Windows, published-artifact, downstream-adoption, and independent release sign-off also remain merge gates.

Comment thread docs/release/skillspector-2.12.0.md Outdated
Comment thread docs/release/skillspector-2.12.0.md Outdated

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head ff10467cd93832bc5a9a0ee8c07a2bcd3b91e01f after the automatic synchronization with main. This merge has the previously reviewed head e6bcf1044b781d9a28ae537cd8a56147edc9afa4 and current main e19ec01cc85b9891b2976a1157fce492504cfc13 as its exact parents. The only intervening tree changes are #558's three batch-scan files; none overlaps the four release files, whose reviewed contents are unchanged.

The two existing non-outdated release-note findings therefore remain current and unresolved. The candidate still claims irregular letter spacing fails closed although alternating-width short runs can evade P3/P4 and AE6, and it still makes an unconditional companion-context guarantee despite #547's unresolved PE3 and RA1 paths. Correct and requalify those implementations or accurately document the limitations. I have not duplicated the inline findings.

Exact-head CI run 35136927518 concluded action_required without starting jobs, leaving no check results. The security-release corrections, unresolved threads, active change request, missing exact-head checks, GitHub BLOCKED state, and previously identified release qualification/sign-off gates prevent merge.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed current head 09c3613e06eb2bfd6cdd6e4ce27a183330ee6b6d after the final automatic synchronization with settled main d162d9b343e559be13df8ebba093df3bc9d58c90. I verified the merge parentage, complete four-file release diff, prior findings/threads, version/lock consistency, and the post-baseline main history.

The earlier letter-spacing and companion-classifier claims remain correctly qualified. The release-integrity finding remains open and has widened: this exact candidate now incorporates the post-validation merges for #562, #526, #421, #491, #517, #571, and #575 (plus documentation cleanup), while the changelog/release notes still claim validation only through 548e5e0 and omit these included changes. Update the candidate inventory and security/behavior notes, record the true exact baseline, and rerun/record release validation for this head.

GitHub currently reports no checks for this exact head and mergeStateStatus=BLOCKED. The stale release inventory/validation, missing exact-head CI, and remaining release qualification gates prevent merge.

Priority: P0 — release traceability and validation must describe the code actually being published.

@rng1995

rng1995 commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

P0 release traceability and validation — ready for re-review

The P0 review referred to 09c3613e06eb2bfd6cdd6e4ce27a183330ee6b6d. The inventory and validation corrections are now present in release commit 3646bb3, which is still the current PR head. I rechecked the following against Git history, source, and the completed CI logs:

The diff against the recorded baseline remains limited to the changelog, release notes, and the matching 2.12.0 package/lockfile versions; third-party dependency versions are unchanged. No additional source commit or CI rerun is needed for this response because the validated head has not changed.

Please re-review the stale-inventory/missing-current-CI P0 finding using this evidence. This does not waive the separate deployment/live-provider, Windows, published-artifact, downstream-adoption, or independent release-sign-off gates, nor carry forward older qualification results to this candidate. Those remain explicitly pending. The requested-changes review has not been dismissed, and nothing has been merged or published.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed the complete four-file release diff, version/lockfile consistency, prior review, and commit ancestry through the current head. The formerly missing #562/#526/#421/#491/#517/#571/#575 entries are now documented.

The same release-baseline requirement is not yet satisfied for this newer candidate. docs/release/skillspector-2.12.0.md still says the catalog covers 47 PRs through a345778, while the actual head includes #576, #556, #551, #580, #383, #584, #600, #560, #561, #542, #546, #541, and now #544 beyond that baseline. In particular, Known Limitations still says SARIF lacks occurrence columns although #584 is included. Update CHANGELOG.md, the release catalog/migration notes, and stale limitations to reflect the actual shipping commit; record validation against the refreshed candidate instead of referring to older counts/runs as current. Existing candidate/pending language should remain until owner release gates are satisfied. No duplicate inline thread added for this continuing baseline issue.

The version bump itself is consistent between pyproject.toml and uv.lock. Exact-head checks are currently unavailable; that separately blocks merging, not the reason for the documentation change request. Contributor code/tests were not executed locally.

Reviewed head: 9319c497cbed529cd0c12bd0e24c3c2a177c305a.
Priority: P0 — Release owners need an accurate shipping inventory and candidate-specific validation.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Incremental re-review after the latest automatic main synchronization. The delta only imports the already-reviewed #539/#588/#591/#607/#597; none of this PR's four release files changed. Therefore the outstanding release-inventory/validation request in review #550 (review) remains open; the newly included PRs also belong in the final shipping inventory. I have not repeated the finding inline. No exact-head checks are attached. The candidate remains unmerged.

Static review only; no contributor code/tests executed.
Reviewed head: 4a282fff4b3d9d3feaabbf9e65c742fd9dbd54a7.
Priority: P0 — Release sign-off requires an accurate candidate inventory and validation record.

Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release-candidate sign-off — SkillSpector 2.12.0

Approved at commit 41fca42e980d955d355b717a93a30831c26b284c against main baseline 224ba292d91b9e5fc59398fc38ab8971f9ab01ec.

The P0 inventory/validation request in review 5286455426, continuing review 5286353393, is addressed:

  • All 65 merged PRs since v2.11.2 are cataloged, including the latest 18 merges. The changelog, release notes, migration guidance, and PR description match the current candidate. Fixed occurrence-column and recursive Markdown-output limitations are no longer described as open.
  • The complete diff against main contains only the changelog, versioned release notes, and matching 2.12.0 package/lockfile versions. Third-party dependency versions are unchanged. The minor version remains appropriate for the added capabilities.
  • CI run 35820158876 completed successfully for this head: all six checks passed. Its test-merge 340f81bc9ff36cde656893de3941edd5d2436334 has the documented main/head parents and the identical release source tree fc50629557ef61b9821b941576202796d3857a24.
  • Hosted and local Python validation each recorded 6,843 passed, 14 skipped, 134 deselected, 4 expected failures, and 90% coverage. The integration/live-provider exclusions remain explicit. The recorded candidate validation also includes 95 selected static integration tests, 65 extension tests, locked installation, lint/format, version/release-helper checks, wheel/sdist builds, Twine validation, and whitespace checks.
  • Prior inline threads are resolved. Letter-spacing, companion-classifier, and other documented product limitations remain accurately qualified; unmerged #514/#553 are not included.

This is sign-off and approval of the current release-preparation PR, based on the candidate-specific evidence above. It is not a claim that documented scanner limitations have been fixed or that live-provider/deployment, Windows, published-artifact, or downstream-adoption qualification has passed. Those separate owner release gates remain recorded and are not waived by this review. Approval applies only to this commit; later merges require refreshed inventory and validation. No merge, tag, publication, or deployment is performed by this approval.

@mohgupta-ship-it

Copy link
Copy Markdown
Member Author

514 and 563 are pending for merge. 553 is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:publish Marks an OSS release pull request for publication

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants