Repository navigation
Auto-translate: accept Chinese/Japanese batch replies that drop the "." separators (#16386) - #16824
Merged
niksedk merged 1 commit intoNov 11, 2026
Conversation
…." separators (SubtitleEdit#16386) Sources without sentence endings are sent as rows joined by lone "." lines, and models often drop some or all of those separators and answer with one line per row. The line-count fallback meant to accept such a reply never did, for two reasons: - ProcessNoSentenceEndingRow appended the next row (behind a separator) to the item it was closing, so every item but the last carried two rows. The fallback then dealt two reply lines to the first item, ran out of lines for the last ones, and the empty-line check rejected the reply. Each item now holds its own row only; the request text still carries the separators. - TrySplitByLineCount only opened when the reply had no "." line at all. It now drops the separator lines from the reply and compares content lines with the request's, so a reply that kept some separators is mapped too. A reply with the wrong number of content lines (merged or split rows, or a row with an internal line break answered on one line) is still rejected. The one-separator-missing reply the previous tests asserted as rejected is exactly the shape this issue asks to accept, so that case moves to the new accepted theory. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #16386. Sources without sentence endings (Chinese, Japanese) are sent as rows joined by lone
.separator lines, and models often drop some or all of those separators and answer with one line per row. The line-count fallback added in 5cac6fb was meant to accept such a reply but never did, so every affected batch was retried and then translated row by row.Cause
ProcessNoSentenceEndingRowappended the next row (behind a separator) to the item it was closing, so every item but the last carried two rows.TrySplitByLineCountthen dealt two reply lines to the first item, ran out of lines for the last ones, andHasSameEmptyLinesrejected the reply. The same inflated item text skewedHasPlausibleProportionsfor these sources..line at all, so a reply that kept some separators was rejected before any mapping was tried.Changes
TrySplitByLineCountdrops the.lines from the reply and compares content lines with the request's, for replies that kept none or only some separators. A reply with the wrong number of content lines, merged or split rows, or a row with an internal line break answered on one line, is still rejected.MergeAndTranslateIfPossible_InvalidPeriodMarkerSplitDoesNotUpdateRowsasserted as rejected is exactly the shape this issue asks to accept, so that case moved to the new accepted theory; the extra-row and doubled-separator cases stay rejected.Verification
MergeAndSplitHelperTests(8 new cases, 2 moved); SeConvTests unchanged and passing.llamacppengine with Hy-MT2 7B and its official prompt on two Chinese WebVTT scripts (38 and 146 cues), requests counted through a logging proxy, same model, prompt and files before and after:Every cue kept its timing in all four outputs. Before the fix, each batch reply with a dropped separator was rejected and its rows translated one request each; after it, those replies are mapped by line count and the row-by-row retries disappear.
🤖 Generated with Claude Code