Skip to content

Text that grows during parsing can overflow Parse_text #7870

Open

Description

@dustinspace217

Problem

read_raw_file_text sizes Parse_text and Parse_text_raw from the file's length (allocate_parse_text(file_len + 1) in code/parse/parselo.cpp), but the text written into them can be longer than the file:

  • In the default (non-Unicode) text mode, process_raw_file_text passes each line through maybe_convert_foreign_characters, which writes every ß (0xDF) as the two letters ss. That copy into Parse_text has no size check.
  • A file using the older 7-bit encryption is decrypted into Parse_text_raw, and that format unpacks every 7 stored bytes into 8 characters.

So a table made mostly of ß characters, or an old-style encrypted table, can write past the end of the heap buffer when it is the largest file parsed so far.

Reproduction

Found by reading the code while reviewing a new text reader for #7865; I haven't triggered it in game, since the buffers only grow and a larger file read earlier hides it. Running the same line-splitting, comment-stripping and maybe_convert_foreign_characters code over a 1 MB file of 0xDF bytes (a newline every 100 bytes) wrote 2,086,667 bytes, about twice the allocation allocate_parse_text would make for that file.

Expected vs actual

Expected: the converted text fits its buffer.
Actual: the conversion can write past the end of Parse_text, and decryption past Parse_text_raw.

Fix alternatives

  • Size both buffers for the worst case: twice the raw text in non-Unicode mode, 8/7 of the file for 7-bit encrypted files.
  • Or bound the copy in process_raw_file_text and grow the buffer when it would overflow.

The .eff reader has the same problem with fixed stack buffers: #7869.

Activity

  1. added
    bugAn issue from unintended consequences
    sanitizer/stabilityIssues caught by Address Sanitizer or other third-party tools.
    on Oct 17, 2026
  2. added this to the Release 27.0 milestone on Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugAn issue from unintended consequencessanitizer/stabilityIssues caught by Address Sanitizer or other third-party tools.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions