Skip to content

fix: make JSON quote candidate scanning linear - #521

Merged
mohgupta-ship-it merged 2 commits into
codex/fix-documentation-analysis-limitsfrom
codex/linear-json-quote-scan
Sep 10, 2026
Merged

mohgupta-ship-it merged 2 commits into
codex/fix-documentation-analysis-limitsfrom
codex/linear-json-quote-scan

Conversation

@mohgupta-ship-it

Copy link
Copy Markdown
Member

Prepared by Codex on behalf of Mohit Gupta.

Long runs of escaped quotes could make JSON candidate discovery repeatedly rescan the remaining text and overrun the artifact deadline before producing a match. Replace that enumeration with cached linear scanning and deadline checks throughout the scan. Preserve the former candidate spans and the caller's existing structural ownership rules. The cache uses memory proportional to the number of quotes.

This is a separate performance fix stacked on #516. The PR changes one production module and two test files. Timeout configuration and the parent correctness fixes retain their separate scope. Review this separately, then integrate it with #516 before the final merge to main; that combined change needs current hosted checks. The quadratic runtime finding remains open in #516 until this fix is included.

Validation:

  • 197 focused documentation, runtime, and paired graph/MCP tests passed; 17 offline integration tests passed on the combined PR514/PR516/performance tree.
  • Independent differential comparison preserved exact spans for 2,045,539 inputs, plus 2,197 additional Unicode/control cases. Deterministic deadline and linear-work tests detect the former missed checks and repeated suffix work.
  • Five inert scenarios ran through the CLI in both modes: 10 successful scans and 19/19 successful actual codex_cli LLM calls. The clean placeholder stays SAFE; findings and incomplete-analysis barriers remain intact. Repeatable CI tests use mocked structured model transports and verify that enabled calls actually occur and disabled mode never invokes the model.
  • The 16,023-character quote-discovery probe improved from about 0.978 seconds to 0.00217 seconds locally. Ruff lint, formatting, DCO signoff, and diff checks passed.
  • Final combined full coverage: 4,426 passed, 14 skipped, 38 deselected, 4 expected failures; 89% coverage in 563.20 seconds. This is tree f560b471 with PR514 d95663d, PR516 040498c, and this PR edd66f3. The earlier timing-sensitive failures both passed in this serialized run.

The CI workflow currently runs pull requests targeting main, so this stacked draft does not imply that the complete hosted workflow has run. The final head also passed a local Linux/aarch64 Docker build and both local-directory and GitHub-repository smoke scans.

Preserve candidate spans while caching quote ends and checking the artifact deadline throughout scanning. Add deterministic deadline and reconstruction compatibility regressions. Prepared by Codex on behalf of Mohit Gupta.

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Verify bounded work for shared quote suffixes and exercise real graph/MCP findings and completeness with semantic analysis enabled and disabled. Structured model transports are mocked for repeatable CI.

Prepared by Codex on behalf of Mohit Gupta.

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it
mohgupta-ship-it marked this pull request as ready for review September 10, 2026 11:29
@mohgupta-ship-it
mohgupta-ship-it merged commit d3dd543 into codex/fix-documentation-analysis-limits Sep 10, 2026
@mohgupta-ship-it
mohgupta-ship-it deleted the codex/linear-json-quote-scan branch September 10, 2026 11:52
SanHsien added a commit to SanHsien/SkillSpector that referenced this pull request Sep 11, 2026
…VIDIA#524

Raise reviewed_pr_through to 527 and reviewed_issue_through to 524 in
tools/upstream_baseline.json (commit axis unchanged at 69dcdfb). Every
item gets a verdict in docs/DECISIONS.md: NVIDIA#493/NVIDIA#507/NVIDIA#508/NVIDIA#511 verified
via git merge-base --is-ancestor as already included through the
2.11.1/2.11.2 sync (including NVIDIA#521, which merged only into the still-
open NVIDIA#516 stack, not main); the remaining 27 items stay "wait for
upstream merge", none adopted now.

Two items get dedicated comparison notes per docs/DIVERGENCE.md's
static_runner.py and scripts/compare_scan_accuracy.py rows: NVIDIA#522 uses a
different env var name and different default/semantics than this
fork's SKILLSPECTOR_MAX_STATIC_SECONDS, so merging it cannot simply
delete the divergence row and needs a downstream env var migration
first; NVIDIA#490 extends this fork's own upstream PR NVIDIA#486 with a Python
3.14/POSIX edge case the fork's Windows environment does not hit, so
NVIDIA#486 is left untouched pending upstream's own resolution. NVIDIA#501-NVIDIA#505 and
NVIDIA#518 are also flagged as near-verbatim matches to this fork's existing
Windows test divergence rows, worth revisiting for row deletion once
merged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: SanHsien <34234698+SanHsien@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant