You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Text that grows during parsing can overflow Parse_text #7870
read_raw_file_text sizes Parse_text and Parse_text_raw from the file's length (allocate_parse_text(file_len + 1) in code/parse/parselo.cpp), but the text written into them can be longer than the file:
In the default (non-Unicode) text mode, process_raw_file_text passes each line through maybe_convert_foreign_characters, which writes every ß (0xDF) as the two letters ss. That copy into Parse_text has no size check.
A file using the older 7-bit encryption is decrypted into Parse_text_raw, and that format unpacks every 7 stored bytes into 8 characters.
So a table made mostly of ß characters, or an old-style encrypted table, can write past the end of the heap buffer when it is the largest file parsed so far.
Reproduction
Found by reading the code while reviewing a new text reader for #7865; I haven't triggered it in game, since the buffers only grow and a larger file read earlier hides it. Running the same line-splitting, comment-stripping and maybe_convert_foreign_characters code over a 1 MB file of 0xDF bytes (a newline every 100 bytes) wrote 2,086,667 bytes, about twice the allocation allocate_parse_text would make for that file.
Expected vs actual
Expected: the converted text fits its buffer.
Actual: the conversion can write past the end of Parse_text, and decryption past Parse_text_raw.
Fix alternatives
Size both buffers for the worst case: twice the raw text in non-Unicode mode, 8/7 of the file for 7-bit encrypted files.
Or bound the copy in process_raw_file_text and grow the buffer when it would overflow.
The .eff reader has the same problem with fixed stack buffers: #7869.
Activity