Skip to content

perf: serialize a directory entry to and from one 128 byte buffer - #93

Merged
mdsteele merged 1 commit into
mdsteele:masterfrom
francisdb:perf/direntry-serialization
Sep 14, 2026
Merged

mdsteele merged 1 commit into
mdsteele:masterfrom
francisdb:perf/direntry-serialization

Conversation

@francisdb

@francisdb francisdb commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

A directory entry was written to the file field by field, in about 45 small writes (one per UTF-16 unit of the name, then one per field), and read the same way. On an unbuffered File each of those is a syscall. Every stream creation writes at least two entries, and the balanced insert from #83 touches a few more, so this is what the create-heavy cases on disk spend their time on; opening a file reads every entry the same way.

Both directions now go through a 128 byte buffer in memory and touch the file once. Measured against master with #83 merged:

case (criterion) master after
write to disk, 10000 x 0 B 510 ms 24 ms
write to disk, 1000 x 64 B 62 ms 3.4 ms
write to disk, 100 x 4 KiB-1 (mini stream) 50 ms 6.4 ms
write to disk, 100 x 4 KiB 5.6 ms 0.5 ms
write to memory, 10000 x 0 B 20 ms 8.2 ms
write to memory, 100 x 4 KiB 220 µs 94 µs

Reading from disk, with the read_streams_disk group from #94 (open the file, then read every stream):

case (criterion) master after
read from disk, 10000 x 0 B 48 ms 5.9 ms
read from disk, 1000 x 64 B 6.1 ms 1.1 ms
read from disk, 100 x 4 KiB-1 (mini stream) 3.2 ms 2.5 ms
read from disk, 100 x 4 KiB 841 µs 208 µs

The large-stream cases are unchanged either way, as expected. For reference, 0.14.0 sat at 625 ms, 40 ms, 67 ms and 2.9 ms on the four disk write cases, so the two that #83 made slower on disk end up well ahead of the release.

A directory entry was written to the file field by field, in some forty
small writes (one per UTF-16 unit of the name), and read the same way.
Both now go through a 128 byte buffer in memory and touch the file once.
Every stream creation and every stream length update writes an entry,
so this shows up when a file has many streams.
@francisdb
francisdb force-pushed the perf/direntry-serialization branch from 0badeae to 66e2877 Compare September 10, 2026 06:14
@mdsteele
mdsteele merged commit c48acd4 into mdsteele:master Sep 14, 2026
4 checks passed
@francisdb
francisdb deleted the perf/direntry-serialization branch September 15, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants