Truncate example output on rune boundaries in verifyexamples - #1154
PratikDhanave (PratikDhanave) wants to merge 1 commit into
Conversation
truncate sliced by byte index, which can split a multi-byte UTF-8 rune (examples print degree signs, emoji, etc.) and emit invalid UTF-8 into logs, CSV, and the AI-verifier prompt. Back off to a rune boundary before cutting.
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The core logic change is correct and is covered by a focused regression test; only minor comment wording nits were identified.
Review effort: Lite
Findings: 2
Open (2)
What changed in this PR
Fixes cmd/verifyexamples output truncation to avoid splitting multi-byte UTF-8 runes (e.g., °C, emoji), which could previously produce invalid UTF-8 in logs, CSV reports, and the AI-verifier prompt.
Changes:
- Update
truncateto back off to a UTF-8 rune boundary before slicing. - Add a unit test ensuring truncation does not produce invalid UTF-8 when cutting mid-rune.
| File | Description |
|---|---|
| cmd/verifyexamples/verifier.go | Adjust truncation logic to avoid cutting in the middle of multi-byte UTF-8 runes. |
| cmd/verifyexamples/verifier_test.go | Add regression test validating truncation preserves valid UTF-8 and doesn’t introduce replacement characters. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| // Back off to a rune boundary so truncation never splits a multi-byte rune | ||
| // (examples print °C, emoji, etc.) and emits invalid UTF-8 into logs/CSV. |
| // "°" is two bytes (0xC2 0xB0); truncating at byte 1 must not split it. | ||
| got := truncate("a°cdef", 2) |
|
Scope: internal-only Changed Go contract: None — Upstream evidence reviewed: No equivalent found. Result: out of scope — this PR only touches an internal Go tooling helper (
|

truncateincmd/verifyexamplessliced by byte index (text[:maxLength]), which can split a multi-byte UTF-8 rune — examples print°C, emoji, etc. — and emit invalid UTF-8 into logs, the CSV report, and the AI-verifier prompt.Change
utf8.RuneStart) before cutting.Test
TestTruncateDoesNotSplitRunes: truncating"a°cdef"mid-rune now yields valid UTF-8. Fails before ("a\xc2..."), passes after.