Skip to content

Open and sort large folders faster - #26

Open
thisisgm wants to merge 18 commits into
omacom:mainfrom
thisisgm:large-folders
Open

thisisgm wants to merge 18 commits into
omacom:mainfrom
thisisgm:large-folders

Conversation

@thisisgm

@thisisgm thisisgm commented Nov 11, 2026 •

Copy link
Copy Markdown

Nine changes to how a folder is listed, sorted and kept, one commit each with its own tests, aimed at folders of tens of thousands of files. A folder of 100,000 files is ready in about two fifths of the time and Folio holds a fifth less memory with it open. Nothing changes in what a folder shows or in what order. The suite goes from 196 to 236 tests. Eight small commits on top answer the review: a folder whose date moved is counted again after a re-read, the watch on Hyprland is taken up again after a hang-up, a text file's page is asked for once, when its date is read, a folder no view has asked for yet is counted again when it changed, Hyprland's events are read 64 KiB a turn, the watch on Hyprland is taken up again after a pause when it hangs up while Folio stays in use, that try is made again after each pause until Hyprland answers, and a row a view asked for that a re-read could not read can be asked for again and is read at the folder's next re-read. A ninth keeps a row's type once it is read from its name, so that going back to a folder costs what it did on main. No library, module or depends entry is added.

What you'd notice

  • A large folder opens and settles sooner. Opening one asked the disk about every file twice, typed every file from its name before the first row showed, and read every size and date up front. A regular file is now typed from the name the listing already read, its type is worked out when a view asks for it, and a listing sorted by name or kind reads sizes and dates for the first 320 rows only. Later rows are read when a view shows them, a batch a turn, off the UI thread. A row not read yet shows no size and no date, never 0 B.
  • Sorting a large folder is faster, in the same order. Sorting 100,000 names collated both strings on every compare. A name or kind sort now makes one sort key an entry first, and a listing that arrives in a few ordered runs is merged. The comparator is the same one, so every order and every tie stays where it was, and the C locale compares as before.
  • A large folder is held once. A selection, a folder count or a second view of the same folder each made the model copy its whole listing. They now read it in place. Each entry also kept its own copy of its type, kind and extension, and the listing kept spare room for entries that never came: those strings are shared and the spare room is given back.
  • The preview's folder summary no longer sorts. It sorted every name to count them and to pick two pictures. It now keeps the first two pictures in one pass.
  • Folio is quieter when nothing happens. While active it ran hyprctl once a second to read the corner rounding; it now reads it at start, on a palette change, on return to the window and when Hyprland's event socket reports a reloaded config. Every refresh of Places, the 15 second poll included, told all four sidebar lists they had changed; each list now has its own signal and is announced only when its contents differ.

Numbers

Measured on a mini PC, cold start into a folder of 100,000 files, six runs a view, the main this branch is based on (059e688) against its head. Settled is the time until the window is up and the process has gone quiet. Both are the CMake build.

View main this branch
Grid First window 925 to 984 ms 924 to 1155 ms
Settled 3988 to 4159 ms 1554 to 1819 ms
Memory (PSS) 242.4 to 242.7 MiB 208.5 to 210.1 MiB
CPU time 3.30 to 3.36 s 1.37 to 1.45 s
List First window 923 to 1036 ms 917 to 995 ms
Settled 3871 to 3989 ms 1454 to 1509 ms
Memory (PSS) 201.2 to 218.8 MiB 156.3 to 157.3 MiB
CPU time 3.15 to 3.24 s 1.20 to 1.24 s
Columns First window 920 to 1028 ms 965 to 1031 ms
Settled 3908 to 4033 ms 1488 to 1586 ms
Memory (PSS) 211.8 to 233.4 MiB 168.2 to 182.6 MiB
CPU time 3.96 to 4.16 s 2.00 to 2.07 s

The time to the first window is unchanged (one grid run of the six took 1155 ms, the other five 924 to 960 ms): it is spent before the listing starts, and this branch leaves it alone.

Stepping through folders inside one open window, on the same mini PC, four runs a view, from the step to the frame that shows the rows, today's main against this branch:

Step View main this branch
Into /usr/bin (4,229 files), first time List 260 to 266 ms 66 to 68 ms
Grid 308 to 314 ms 116 to 123 ms
Columns 276 to 284 ms 82 to 87 ms
Into the folder of 100,000 files, first time List 1415 to 1455 ms 182 to 185 ms
Grid 1476 to 1504 ms 244 to 258 ms
Columns 1472 to 1542 ms 214 to 228 ms
Back into /usr/bin List 29 to 32 ms 30 ms
Grid 72 to 78 ms 74 to 80 ms
Columns 50 to 55 ms 54 to 65 ms

The way back is the one step this branch had made slower, until its last commit: a view asks for a row's kind, type and extension each time it makes the row, and each ask read them from the name again (48 microseconds a row in /usr/bin). It cost 35 to 37 ms in the list, 87 to 90 ms in the grid and 69 to 70 ms in the columns. The model now keeps up to 4,096 names' types until a folder is found changed.

A battery of 28 captures (every sort in every view, an empty, a locked and a looping folder, Recents, Trash, a search, a split, a second window, a file that vanishes) shows no difference between the two, and tiles 5,900 files down the large folder show their size and date.

How it stays the same

  • The sort is held to the old order: for every sort mode, under an English locale and under C, entries that arrive ordered, reversed, shuffled or in a few runs have to end in the order one stable sort with the old collating compare gives, ties included.
  • A sort by size or date still reads every entry before it sorts, and a sort by kind still types every entry, both in the worker.
  • A link, a pipe or a device keeps the full lookup; only a regular file is typed from its name.
  • A row whose size or date is not read yet shows nothing for it in a row, a tile and the details line, rather than a zero.
  • A read that lands for a folder already left is dropped, and a file that grew after its row was read shows its new size.

Tests

Forty new cases, each failing without its commit. Among them: a regular file is typed from what its listing read, and a link, a pipe and a device are still looked up; a selection, a count and a remembered listing do not copy the listing; entries share their type strings; only what a view asks for is typed; every sort gives the plain sort's order in both locales; rows past the first 320 are unread until a view asks, and a row shows no size or date until then; a snapshot taken while a batch runs is read by the next; what was read for a folder already left is dropped; a sidebar list is announced only when it changed, by a signal of its own; the rounding is asked when something changes and not on a timer, and is kept when hyprctl fails; a folder that gained a file is counted again when its parent is re-read; the event socket is reopened on return to the window after Hyprland hung up; a plugin's item keeps the kind and subtitle it was listed with; a text file's page is asked for once; a folder past the first rows, counted before any view asked for it, is counted again once it changed; a burst of Hyprland's events is read over several turns of the event loop; the rest of an event line too long to keep is dropped up to its end; a folder's count keeps the date it was made with, whichever of the count and a view's read of that date lands first; the watch on Hyprland is taken up again after a hang-up while Folio stays in use, and not while it is away; a row asked for and taken over by a re-read that could not read it can be asked for again, a row asked for and not read is read at the folder's next re-read, and a view is told of a row another view of the folder read meanwhile; a row's type is read from its name once, a folder, a pipe and a plugin's item of that name keep their own, and a folder found changed has its types read again. Every commit builds and passes the suite on its own, also with a temp folder whose name has a space.

@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Speeds up large folders by delaying sizes, dates, and types until needed, reusing listings, and reducing sorting work.

  • Large folders show rows without reading every detail up front.
  • Large folder sorts reuse ordered runs and prebuilt name keys.
  • Folder previews count items and pick pictures in one pass.
  • Sidebar lists announce only their own changes.
  • Theme rounding follows Hyprland events instead of a timer.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Open or refresh a folder] --> B[List and sort in a worker]
  B --> C[Read the first rows]
  C --> D[Show the listing]
  D --> E[View asks for an unread row]
  E --> F[Read requested rows in a worker]
  F --> G{Still the same generation?}
  G -->|Yes| H[Save values and notify the view]
  G -->|No| I[Drop the old answer]
Loading

Reviews (14) · Last reviewed commit: "Keep a row's type once it is read from i..." · Reviewed by Greptile

Comment thread src/foldermodel.cpp Resolved
Comment thread src/theme.cpp Resolved
@thisisgm
thisisgm force-pushed the large-folders branch 2 times, most recently from 9f79a89 to 47b758d Compare November 11, 2026 14:19
Comment thread src/Format.js Resolved
Building a listing asked the disk about every file a second time,
because Qt stats a file again before it reads the name. A regular file
is now typed from the name the listing already read; a link, a pipe or
a device keeps the full lookup.
A selection, a folder count or a second view of the same folder each
made the model copy its whole listing. They now read it in place, so a
large folder is held once however many views show it.
The preview's folder summary sorted every name to count them and to
pick two pictures. It now keeps the first two pictures in one pass, so
summarizing a large folder no longer holds every name.
Every entry kept its own copy of its type, kind and extension, and the
listing kept spare room for entries that never came. The entries of a
listing now share those strings, and the spare room is given back.
Opening a folder typed every file before its first row showed. The
listing now leaves a regular file's type, kind and extension unread
until a role asks, which touches no disk; a sort by kind still types
every entry in the worker.
Every refresh of Places, the 15 second poll included, told all four
sidebar lists they had changed. Each list now has its own signal and
is announced only when its contents differ.
While active, Folio ran hyprctl once a second to read the corner
rounding. It now reads it at start, on a palette change, on return to
the window and when Hyprland's event socket reports a reloaded config.
Sorting 100,000 names collated both strings on every compare. A
listing that arrives in a few ordered runs is now merged, and a name
or kind sort makes one sort key an entry first. The comparator is the
same, so every order and tie stays; the C locale compares as before.
A name or kind listing read every entry's size and date before its
first row showed. It now reads the first 320 rows, and a later row
when a view asks, one batch a turn, off the UI thread. A row not read
yet shows no size and no date, never 0 B.
A re-read that found the same names kept each folder's item count,
even when the folder's own date had moved. Such a folder's count is
now dropped and counted again, as it was before the listing learned
to keep its rows through a re-read.
The event socket was opened once. Hyprland hangs up on a reader that
fell behind, and after that a config reload no longer reached the
corners. The watch is now taken up again when Folio is back in use.
A text tile past the rows read at first asked for its page with no
date, then again when the date arrived, and a page has no cache on
disk, so the file was read and drawn twice. Like a thumbnail, a page
is now asked for only once its date is known.
A folder past the rows read at first is counted without its date
being read, so a re-read had nothing to compare and kept its old
count. Its date is now read just before it is counted and kept
with the count, and the re-read sees the folder changed since.
A burst of events was read whole into memory before any of it was
looked at, on the UI thread. One turn of the event loop now reads
64 KiB at most and looks at each piece as it comes, and a line too
long to keep is dropped to its end, its last piece included.
Hung up on by Hyprland, the theme took its watch up again only when
Folio was left and come back to, so a config reload in between left
the corners as they were. While in use it now connects again after
a pause, and reads the rounding for a reload that went by untold.
Hung up on while Folio stayed in use, the theme tried Hyprland's
event socket once more after a pause. A try that found no socket
was the last, and reloads went untold until Folio was left and
come back to. Each failed try now starts the pause again.
A re-read drops the answers on their way and reads the rows asked for itself.
A row it could not read either stayed marked as being read, so no view could
ask for it again. Such a row is now owed: a view may ask again, the folder's
next re-read tries it again, and a view is told when another view read it.
A view asks for a row's kind, MIME type and extension each time it
makes the row, and each ask read them from the name again: 48 us a
row in /usr/bin. The model now keeps up to 4,096 names' types, until
a folder is found changed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant