Skip to content

tokener: fix -Infinity parsing #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
prownd wants to merge 1 commit into
base: main
Choose a base branch
from
Open

tokener: fix -Infinity parsing #183

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion json_tokener.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -762,7 +762,14 @@ struct fjson_object *fjson_tokener_parse_ex(struct fjson_tokener *tok, const cha
printbuf_memappend_fast(tok->pb, case_start, case_len);

// Check for -Infinity
if (tok->pb->buf[0] == '-' && case_len == 1 && (c == 'i' || c == 'I')) {
// Use bpos (the total number of bytes buffered so far)
// rather than case_len (only what this call appended), so
// that "-" and "Infinity" split across incremental calls
// are still recognized. Reset st_pos, which may carry a
// stale value from a previously parsed token.
if (tok->pb->bpos == 1 && tok->pb->buf[0] == '-' &&

@cubic-dev-ai cubic-dev-ai Bot Sep 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This change fixes two distinct regressions (stale st_pos after a previous token, and -/Infinity split across incremental calls), but no regression tests were added for either. tests/test_parse.c only exercises the standalone single-call cases. Since both breakages survive the existing test suite, add coverage for [true,-Infinity]-style input and for calling fjson_tokener_parse_ex twice (first with -, then with Infinity) on the same tokener, asserting the result is -Infinity.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At json_tokener.c, line 770:

<comment>This change fixes two distinct regressions (stale `st_pos` after a previous token, and `-`/`Infinity` split across incremental calls), but no regression tests were added for either. `tests/test_parse.c` only exercises the standalone single-call cases. Since both breakages survive the existing test suite, add coverage for `[true,-Infinity]`-style input and for calling `fjson_tokener_parse_ex` twice (first with `-`, then with `Infinity`) on the same tokener, asserting the result is `-Infinity`.</comment>

<file context>
@@ -762,7 +762,14 @@ struct fjson_object *fjson_tokener_parse_ex(struct fjson_tokener *tok, const cha
+				// that "-" and "Infinity" split across incremental calls
+				// are still recognized. Reset st_pos, which may carry a
+				// stale value from a previously parsed token.
+				if (tok->pb->bpos == 1 && tok->pb->buf[0] == '-' &&
+				    (c == 'i' || c == 'I')) {
+					tok->st_pos = 0;
</file context>

(c == 'i' || c == 'I')) {
tok->st_pos = 0;
state = fjson_tokener_state_inf;
goto redo_char;
}
Expand Down