tokener: fix -Infinity parsing

[prownd]

The transition from the number state to the Infinity state used case_len (bytes appended during the current call) and never reset st_pos:

• L1: after another token set st_pos (e.g. "[true,-Infinity]"), the stale st_pos made the Infinity comparison length wrong, so a leading-minus Infinity after any other token failed to parse.
• L2: when "-" and "Infinity" arrived in separate incremental calls, case_len was 0 on the second call and the -Infinity branch was never taken, yielding "number expected".

Use tok->pb->bpos == 1 (the buffer holds exactly "-") to detect the leading minus regardless of call boundaries, and reset tok->st_pos.

---

Summary by cubic

Fixes -Infinity parsing in the tokener so leading-minus Infinity is recognized in all cases. Previously, -Infinity failed to parse when it followed another token (stale st_pos) or when - and Infinity arrived in separate incremental calls. Now the buffer position (bpos) is used to detect the leading minus and st_pos is reset, so -Infinity parses correctly regardless of call boundaries.

^{Written for commit 8638833. Summary will update on new commits.}

[cubic-dev-ai]

1 issue found across 1 file

Confidence score: 5/5

• json_tokener.c addresses stale st_pos and split -/Infinity incremental parsing regressions, but without targeted regression tests these fixes could regress unnoticed — add coverage in tests/test_parse.c for both scenarios.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="json_tokener.c">

<violation number="1" location="json_tokener.c:770">
P3: This change fixes two distinct regressions (stale `st_pos` after a previous token, and `-`/`Infinity` split across incremental calls), but no regression tests were added for either. `tests/test_parse.c` only exercises the standalone single-call cases. Since both breakages survive the existing test suite, add coverage for `[true,-Infinity]`-style input and for calling `fjson_tokener_parse_ex` twice (first with `-`, then with `Infinity`) on the same tokener, asserting the result is `-Infinity`.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P3: This change fixes two distinct regressions (stale st_pos after a previous token, and -/Infinity split across incremental calls), but no regression tests were added for either. tests/test_parse.c only exercises the standalone single-call cases. Since both breakages survive the existing test suite, add coverage for [true,-Infinity]-style input and for calling fjson_tokener_parse_ex twice (first with -, then with Infinity) on the same tokener, asserting the result is -Infinity.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At json_tokener.c, line 770:

<comment>This change fixes two distinct regressions (stale `st_pos` after a previous token, and `-`/`Infinity` split across incremental calls), but no regression tests were added for either. `tests/test_parse.c` only exercises the standalone single-call cases. Since both breakages survive the existing test suite, add coverage for `[true,-Infinity]`-style input and for calling `fjson_tokener_parse_ex` twice (first with `-`, then with `Infinity`) on the same tokener, asserting the result is `-Infinity`.</comment>

<file context>
@@ -762,7 +762,14 @@ struct fjson_object *fjson_tokener_parse_ex(struct fjson_tokener *tok, const cha
+				// that "-" and "Infinity" split across incremental calls
+				// are still recognized. Reset st_pos, which may carry a
+				// stale value from a previously parsed token.
+				if (tok->pb->bpos == 1 && tok->pb->buf[0] == '-' &&
+				    (c == 'i' || c == 'I')) {
+					tok->st_pos = 0;
</file context>