release: SkillSpector 2.12.0 - #550
mohgupta-ship-it wants to merge 61 commits into
Conversation
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
|
Codex on behalf of Mohit Gupta — independent qualification update, 16 September 2026. The freshly built 2.12.0 wheel from Fresh results after the AS3 update:
Both frozen corpora retain exactly the previous failure sets. Their behavioral expectations and fixture bytes were unchanged; only candidate identity/version provenance was refreshed. The consumer policy/limit assertions were retained. An initial provider-enabled consumer harness attempt was terminated and excluded; the reported counts come from the subsequent run with the established sanitized environment. Draft #563 separately fixes unsupported-primary and pure/mixed newline completeness gaps. At Current-head hosted CI/review and the remaining production/provider, Windows, published-artifact and adoption gates are still required. These results do not constitute release sign-off; no release or production pin was changed. |
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed current head e6bcf1044b781d9a28ae537cd8a56147edc9afa4, including the complete version/changelog/release-note diff, candidate history, existing review thread, version/lock consistency, component source identity, exact-head checks, and the stated release gates. The package and lock versions agree and all five hosted checks pass.
Changes are requested because two security guarantees in the release notes are not true of this exact candidate. Its letter-spacing implementation is byte-identical to the currently reviewed #470 implementation and still permits alternating-width spacing to evade P3/P4 and AE6. Its companion-classification analyzer blobs are identical to the unresolved #547 implementation, where four contextual fail-open paths remain. Either correct and requalify those implementations before publication or describe the precise limitations without claiming the risky cases remain covered. Deployment/provider, Windows, published-artifact, downstream-adoption, and independent release sign-off also remain merge gates.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head ff10467cd93832bc5a9a0ee8c07a2bcd3b91e01f after the automatic synchronization with main. This merge has the previously reviewed head e6bcf1044b781d9a28ae537cd8a56147edc9afa4 and current main e19ec01cc85b9891b2976a1157fce492504cfc13 as its exact parents. The only intervening tree changes are #558's three batch-scan files; none overlaps the four release files, whose reviewed contents are unchanged.
The two existing non-outdated release-note findings therefore remain current and unresolved. The candidate still claims irregular letter spacing fails closed although alternating-width short runs can evade P3/P4 and AE6, and it still makes an unconditional companion-context guarantee despite #547's unresolved PE3 and RA1 paths. Correct and requalify those implementations or accurately document the limitations. I have not duplicated the inline findings.
Exact-head CI run 35136927518 concluded action_required without starting jobs, leaving no check results. The security-release corrections, unresolved threads, active change request, missing exact-head checks, GitHub BLOCKED state, and previously identified release qualification/sign-off gates prevent merge.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 09c3613e06eb2bfd6cdd6e4ce27a183330ee6b6d after the final automatic synchronization with settled main d162d9b343e559be13df8ebba093df3bc9d58c90. I verified the merge parentage, complete four-file release diff, prior findings/threads, version/lock consistency, and the post-baseline main history.
The earlier letter-spacing and companion-classifier claims remain correctly qualified. The release-integrity finding remains open and has widened: this exact candidate now incorporates the post-validation merges for #562, #526, #421, #491, #517, #571, and #575 (plus documentation cleanup), while the changelog/release notes still claim validation only through 548e5e0 and omit these included changes. Update the candidate inventory and security/behavior notes, record the true exact baseline, and rerun/record release validation for this head.
GitHub currently reports no checks for this exact head and mergeStateStatus=BLOCKED. The stale release inventory/validation, missing exact-head CI, and remaining release qualification gates prevent merge.
Priority: P0 — release traceability and validation must describe the code actually being published.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
P0 release traceability and validation — ready for re-reviewThe P0 review referred to
The diff against the recorded baseline remains limited to the changelog, release notes, and the matching 2.12.0 package/lockfile versions; third-party dependency versions are unchanged. No additional source commit or CI rerun is needed for this response because the validated head has not changed. Please re-review the stale-inventory/missing-current-CI P0 finding using this evidence. This does not waive the separate deployment/live-provider, Windows, published-artifact, downstream-adoption, or independent release-sign-off gates, nor carry forward older qualification results to this candidate. Those remain explicitly pending. The requested-changes review has not been dismissed, and nothing has been merged or published. |
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed the complete four-file release diff, version/lockfile consistency, prior review, and commit ancestry through the current head. The formerly missing #562/#526/#421/#491/#517/#571/#575 entries are now documented.
The same release-baseline requirement is not yet satisfied for this newer candidate. docs/release/skillspector-2.12.0.md still says the catalog covers 47 PRs through a345778, while the actual head includes #576, #556, #551, #580, #383, #584, #600, #560, #561, #542, #546, #541, and now #544 beyond that baseline. In particular, Known Limitations still says SARIF lacks occurrence columns although #584 is included. Update CHANGELOG.md, the release catalog/migration notes, and stale limitations to reflect the actual shipping commit; record validation against the refreshed candidate instead of referring to older counts/runs as current. Existing candidate/pending language should remain until owner release gates are satisfied. No duplicate inline thread added for this continuing baseline issue.
The version bump itself is consistent between pyproject.toml and uv.lock. Exact-head checks are currently unavailable; that separately blocks merging, not the reason for the documentation change request. Contributor code/tests were not executed locally.
Reviewed head: 9319c497cbed529cd0c12bd0e24c3c2a177c305a.
Priority: P0 — Release owners need an accurate shipping inventory and candidate-specific validation.
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Incremental re-review after the latest automatic main synchronization. The delta only imports the already-reviewed #539/#588/#591/#607/#597; none of this PR's four release files changed. Therefore the outstanding release-inventory/validation request in review #550 (review) remains open; the newly included PRs also belong in the final shipping inventory. I have not repeated the finding inline. No exact-head checks are attached. The candidate remains unmerged.
Static review only; no contributor code/tests executed.
Reviewed head: 4a282fff4b3d9d3feaabbf9e65c742fd9dbd54a7.
Priority: P0 — Release sign-off requires an accurate candidate inventory and validation record.
Signed-off-by: Narendran Raghavan <nraghavan@nvidia.com>
rng1995
left a comment
There was a problem hiding this comment.
Release-candidate sign-off — SkillSpector 2.12.0
Approved at commit 41fca42e980d955d355b717a93a30831c26b284c against main baseline 224ba292d91b9e5fc59398fc38ab8971f9ab01ec.
The P0 inventory/validation request in review 5286455426, continuing review 5286353393, is addressed:
- All 65 merged PRs since v2.11.2 are cataloged, including the latest 18 merges. The changelog, release notes, migration guidance, and PR description match the current candidate. Fixed occurrence-column and recursive Markdown-output limitations are no longer described as open.
- The complete diff against main contains only the changelog, versioned release notes, and matching 2.12.0 package/lockfile versions. Third-party dependency versions are unchanged. The minor version remains appropriate for the added capabilities.
- CI run 35820158876 completed successfully for this head: all six checks passed. Its test-merge
340f81bc9ff36cde656893de3941edd5d2436334has the documented main/head parents and the identical release source treefc50629557ef61b9821b941576202796d3857a24. - Hosted and local Python validation each recorded 6,843 passed, 14 skipped, 134 deselected, 4 expected failures, and 90% coverage. The integration/live-provider exclusions remain explicit. The recorded candidate validation also includes 95 selected static integration tests, 65 extension tests, locked installation, lint/format, version/release-helper checks, wheel/sdist builds, Twine validation, and whitespace checks.
- Prior inline threads are resolved. Letter-spacing, companion-classifier, and other documented product limitations remain accurately qualified; unmerged #514/#553 are not included.
This is sign-off and approval of the current release-preparation PR, based on the candidate-specific evidence above. It is not a claim that documented scanner limitations have been fixed or that live-provider/deployment, Windows, published-artifact, or downstream-adoption qualification has passed. Those separate owner release gates remain recorded and are not waived by this review. Approval applies only to this commit; later merges require refreshed inventory and validation. No merge, tag, publication, or deployment is performed by this approval.
|
514 and 563 are pending for merge. 553 is merged. |
Prepare SkillSpector 2.12.0 from v2.11.2, with the package version, lockfile, changelog, and release notes covering 65 merged PRs through
224ba292d91b9e5fc59398fc38ab8971f9ab01ec.The release remains 2.12.0 because it adds CLI controls, integrations, LLM provenance, dependency-source analysis, GitHub subdirectory inputs, and other capabilities alongside security and correctness fixes. The latest 18 merges are #576, #556, #551, #580, #383, #584, #600, #560, #561, #542, #546, #541, #544, #539, #588, #591, #607, and #597. Migration requirements and changes to findings, completeness, and installation decisions are detailed below.
Included changes
Capabilities and providers
metadata.llm_provenanceschema 1 with configured/resolved/effective provider identity and requested-versus-observed sampling/reasoning controls; this does not guarantee deterministic provider behavior (feat(report): add LLM analysis provenance #556).SKILL.md/README.mdshell fences (Detect dependency source redirection #383)./tree/directory selection with longest advertised branch/tag resolution, including slash-containing refs, bounded ingestion, and path-escape rejection (feat(input): support GitHub tree subdirectories #561).SKILLSPECTOR_COMPACT_PROMPTS=1|true|yes; defaults remain unchanged (feat(llm): add compact prompt line numbering #542).--fail-on-findingsacross single-skill, recursive, and MCP registry scans (feat(cli): add fail-on-findings option #469), and a configurable per-artifact static-analysis allowance (feat: make static analysis runtime configurable #522).opencode_clisemantic provider, now requiring exactly OpenCode 1.18.31, with authentication and deny-all policy checks (feat(providers): add opencode_cli semantic-scan provider #536, feat(providers): verify OpenCode 1.18.31 for the deny-all policy #575)./skillspectorcommand andskillspector_scantool with static analysis by default and semantic analysis opt-in (feat(analyzer): add OpenCode-native SkillSpector invocation skill and tool #537).gemini-3.5-flashmetadata through the existing OpenAI-compatible provider (Feature/gemini support and cli ux #7).Retry-Afterhandling within the workflow deadline; retain sanitized incomplete-analysis diagnostics when recovery fails (fix(llm): retry transient provider failures #555).Security and completeness
analyzer_load_errorcoverage gaps rather than silently complete scans; execution success remains distinct from completeness (fix(analyzers): surface analyzer modules dropped at registry load time #591).reference_missing) from ambiguous references (reference_unresolved). MCPsafe_to_installcan allow missing-reference-only caveats if every discovered file was inspected and all other gates pass; ambiguous references and other coverage failures still block (fix(mcp): stop an unresolved reference from blocking safe_to_install #526).__dict__/vars(module)subscripts andget/setdefault/pop, emitting AST7 for dynamic keys and AST9 for dangerous literal names ( fix: flag module __dict__ subscript as reflective attribute access #517)..venv,venv, and.toxunder existing traversal limits (fix(sc8): inspect bytecode in virtual environments #571).opaque_contentcoverage for unmodeled active-hook payloads (fix(hooks): fail closed on unmodeled payload analysis #573).Accuracy, output, and performance
--output(fix(cli): make recursive scans fail closed #576).MODERATEseverity toMEDIUM(fix(supply-chain): normalize GHSA "MODERATE" severity to MEDIUM #588), and avoid Rustdrop(&mut self)false positives while preserving YARAROP(elf)controls (fix(yara): require word boundary on exploit_framework $rop_chain #607)./blob/URLs (fix(input): download the raw file for GitHub and GitLab /blob/ URLs #566).Compatibility and migration
analyzer_load_errorandtransitive_child_scan_failed, preserve aggregate failure/omission evidence, and distinguish JSON zero-based columns from SARIF one-based Unicode-codepoint columns.SKILLSPECTOR_BUILD_REVISION; unavailable revision data stays unknown, and provenance is not a reproducibility guarantee..opencode/in a checkout and is not installed by the wheel..venvor setSKILLSPECTOR_BINto an existing absolute executable path. AmbientPATHlookup is removed, and reports must be written within the current workspace. These restrictions do not change the standalone CLI or OpenCode tool's output-path contract.reference_missingseparately fromreference_unresolved. MCPsafe_to_installcan be true for a missing-reference-only caveat even though the report remains incomplete/CAUTION; partially inspected files, ambiguity, and requested semantic failures still block.SAFE./blob/scans now analyze raw file bytes.--fail-on-findingsis opt-in. Valid model-registry overrides now affect CLI-provider token budgets. The static allowance defaults to 300 seconds per artifact within the existing workflow deadline; setSKILLSPECTOR_MAX_STATIC_ANALYSIS_SECONDS_PER_ARTIFACT=30to retain the earlier allowance.--no-llmfor intentional static-only scans; unavailable or incomplete requested semantic work remains incomplete.Validation
224ba292d91b9e5fc59398fc38ab8971f9ab01ec(fix(analyzer): distinguish format-only opaque references in AE1 #597); all 65 merged PRs since v2.11.2 are cataloged.41fca42e980d955d355b717a93a30831c26b284c.340f81bc9ff36cde656893de3941edd5d2436334, whose parents are the recorded main baseline and release head; its treefc50629557ef61b9821b941576202796d3857a24is identical to the release head's tree.Validation applies to the commit recorded here. Previous candidate test counts and workflow runs do not certify subsequent main merges.
Known limitations and release status
TypeErrorwhen an optional end column is absent, although normal parsed files pass.No 2.12.0 release has been published.