Repository navigation
DRU-1092 -- Strip OpenAI citation markers from Codex output - #1155
Merged
czpython merged 1 commit intoNov 10, 2026
Merged
Conversation
p-r-a-v-i-n
marked this pull request as draft
November 10, 2026 05:20
p-r-a-v-i-n
marked this pull request as ready for review
November 10, 2026 05:40
p-r-a-v-i-n
marked this pull request as draft
November 10, 2026 05:41
p-r-a-v-i-n
marked this pull request as ready for review
November 10, 2026 05:50
czpython
requested changes
Nov 10, 2026
czpython
left a comment
Owner
There was a problem hiding this comment.
The round trip in parse is correct. Checked with quotes, backslashes, newlines, nested lists, and a marker written as a \ue200 escape. Each one comes back clean and valid.
The pattern needs two changes. See the inline comments. The test needs the same escapes.
p-r-a-v-i-n
force-pushed
the
dru-1092-strip-codex-citations
branch
from
November 10, 2026 13:37
dc0bb87 to
95b542e
Compare
czpython
approved these changes
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CodexHarness.parsenow removes OpenAI citation markers from a Codex result. Before, a Codex run that searched the web could leave markers in its result:U+E200cite, aU+E202reference ID, andU+E201. They showed as stray text in saved results and in the prompts of later runs that read those results.parseserializes the payload withensure_ascii=False, removes each marker and the spaces before it with_CITATION_MARKER, and loads the JSON again. One pass covers every nested string.U+E201is left in place, and the JSON stays valid.output.jsonis written from the cleaned payload.