Skip to content

Hold less memory for a large folder - #92

Open
thisisgm wants to merge 21 commits into
omacom:mainfrom
thisisgm:memory
Open

thisisgm wants to merge 21 commits into
omacom:mainfrom
thisisgm:memory

Conversation

@thisisgm

@thisisgm thisisgm commented Nov 12, 2026 •

Copy link
Copy Markdown

Stacked on #26: its eighteen commits come first here, the last three are this pull request's, and I will rebase it as soon as #26 lands.

Three changes to what a large folder costs in memory, one commit each with its own tests. A row of a listing was 224 bytes and held its whole path; it is now 80 bytes and shares its folder's path with the rows beside it. And the grid keeps a text file's page at the size its tile draws it, where it kept the whole page it was made at. From a cold start into a folder of 100,000 files the process holds less (about 64 MiB in the grid, 29 in the list and 32 in the columns by the medians), and the time to the first window, the time to settle and the CPU time stay where they were. The suite goes from 236 to 241 tests. No library, depends or optdepends entry is added.

What you'd notice

  • A large folder takes less memory in every view. Nothing else changes: the rows, their order, the tiles and what a row answers are the same.
  • What few rows carry is kept out of every row. Each row held a kind, a type and an extension, a plugin item's glyph and details and a folder's first pictures, whether it had any or not. Those now sit in a block of their own that a row has only once it needs one, and the rows of one type in a listing share a single block. A write gives a row its own copy, so nothing a row is told leaks to its neighbours.
  • A folder's path is kept once. Each row held its whole path. The rows of a listing now share one string for the folder, and a row's path is put together from it and the name when it is asked for. A search's matches share the folder searched, and a plugin item's id, which need not end in its name, is kept whole.
  • A text file's page is kept at the size its tile draws it. The grid made the page of a text file 256 pixels tall (512 once zoomed in) and kept all of it for a tile that draws it 92 tall. The tile now says how large it draws through Image.sourceSize, which Qt hands to the provider in the screen's own pixels, and the provider hands back a picture no larger and never an enlarged one. Zoomed in, the page is kept at the zoom's whole step. Photos, videos, PDFs and every other thumbnail are left as they are made.

Numbers

Measured on a mini PC, cold start into a folder of 100,000 files, twelve runs a view, #26's head (10f84a6) against this branch's. Each cell is the median, with the lowest and highest run in brackets: the timings are the same within their spread, the memory is what moved. Settled is the time until the window is up and the process has gone quiet. Both are the CMake build.

View #26 this branch
Grid First window 954 ms (893 to 1067) 928 ms (902 to 1557)
Settled 1626 ms (1553 to 1784) 1531 ms (1497 to 2118)
Memory (PSS) 206.5 MiB (206.3 to 207.1) 142.4 MiB (142.1 to 142.7)
CPU time 1.42 s (1.36 to 1.55) 1.35 s (1.34 to 1.55)
List First window 918 ms (902 to 4205) 956 ms (928 to 1066)
Settled 1457 ms (1376 to 4753) 1460 ms (1391 to 1574)
Memory (PSS) 154.8 MiB (154.4 to 169.5) 125.6 MiB (125.2 to 126.1)
CPU time 1.22 s (1.19 to 1.26) 1.22 s (1.15 to 1.28)
Columns First window 972 ms (869 to 1052) 1010 ms (896 to 1103)
Settled 1536 ms (1450 to 1643) 1532 ms (1482 to 1700)
Memory (PSS) 169.7 MiB (165.8 to 186.4) 137.4 MiB (136.2 to 149.7)
CPU time 2.03 s (1.96 to 2.21) 2.09 s (1.98 to 2.85)

Screenshots

The grid's text pages, before above and after below. They are drawn at the same size, from a picture with an eighth of the pixels.

Text pages in the grid, before and after

Tests

Five new cases, each failing without its commit: a row is 80 bytes, a row listed by name keeps nothing aside, the rows of one type share one block and a write to one leaves the other alone, a counted folder without pictures stays small, and a name without an extension shares no block with a name that ends in a dot; the rows of a listing and the matches of a search share one string for their folder, each path is the file's own, and a plugin item's id survives whole; a row of the folder come to is not read because a row of the same name was read in the folder left; the provider hands a picture over no larger than the view draws and never enlarges one; and the grid keeps a text page at the tile's size, twice that once zoomed in, while a photo's thumbnail keeps all it has. The two picture cases pass at a display scale of 2 as well as 1.

Limits

  • Only text pages are kept at the drawn size. A PDF's page, a photo's or a video's thumbnail is still held as it is made.
  • A selected row's path is a string of its own for as long as it is selected, where it shared the row's: with all 100,000 rows selected that is about 14 MiB, given back when the selection goes.
  • A row's path is put together each time it is asked for. The passes that ask every row (keeping a selection through a re-read, listing the selected paths) reuse one buffer and allocate nothing per row.

@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issue was found.

Summary

This PR reduces memory held by large folder listings and grid text previews.

  • Large folder listings keep less repeated data in each row.
  • Sidebar sections update when their lists change.
  • Window rounding updates when the desktop setting changes.
  • Grid text pages use images sized for their tiles.

No new actionable issues were found. The earlier unnumbered finding about cancelled reads has been addressed by clearing pending requests and keeping owed rows available for retry. This review used source and test inspection; tests were not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Read folder] --> B[Build compact rows]
    B --> C[Share folder paths and optional fields]
    C --> D[Show listing]
    D --> E[View asks for unread size or date]
    E --> F[Read requested rows in background]
    F --> G[Notify view]
    D --> H[Text tile requests image size]
    H --> I[Draw page and shrink if needed]
Loading

Reviews (4) · Last reviewed commit: "Keep a folder's path once for all its ro..." · Reviewed by Greptile

Comment thread src/foldermodel.cpp Resolved
Building a listing asked the disk about every file a second time,
because Qt stats a file again before it reads the name. A regular file
is now typed from the name the listing already read; a link, a pipe or
a device keeps the full lookup.
A selection, a folder count or a second view of the same folder each
made the model copy its whole listing. They now read it in place, so a
large folder is held once however many views show it.
The preview's folder summary sorted every name to count them and to
pick two pictures. It now keeps the first two pictures in one pass, so
summarizing a large folder no longer holds every name.
Every entry kept its own copy of its type, kind and extension, and the
listing kept spare room for entries that never came. The entries of a
listing now share those strings, and the spare room is given back.
Opening a folder typed every file before its first row showed. The
listing now leaves a regular file's type, kind and extension unread
until a role asks, which touches no disk; a sort by kind still types
every entry in the worker.
Every refresh of Places, the 15 second poll included, told all four
sidebar lists they had changed. Each list now has its own signal and
is announced only when its contents differ.
While active, Folio ran hyprctl once a second to read the corner
rounding. It now reads it at start, on a palette change, on return to
the window and when Hyprland's event socket reports a reloaded config.
Sorting 100,000 names collated both strings on every compare. A
listing that arrives in a few ordered runs is now merged, and a name
or kind sort makes one sort key an entry first. The comparator is the
same, so every order and tie stays; the C locale compares as before.
A name or kind listing read every entry's size and date before its
first row showed. It now reads the first 320 rows, and a later row
when a view asks, one batch a turn, off the UI thread. A row not read
yet shows no size and no date, never 0 B.
A re-read that found the same names kept each folder's item count,
even when the folder's own date had moved. Such a folder's count is
now dropped and counted again, as it was before the listing learned
to keep its rows through a re-read.
The event socket was opened once. Hyprland hangs up on a reader that
fell behind, and after that a config reload no longer reached the
corners. The watch is now taken up again when Folio is back in use.
A text tile past the rows read at first asked for its page with no
date, then again when the date arrived, and a page has no cache on
disk, so the file was read and drawn twice. Like a thumbnail, a page
is now asked for only once its date is known.
A folder past the rows read at first is counted without its date
being read, so a re-read had nothing to compare and kept its old
count. Its date is now read just before it is counted and kept
with the count, and the re-read sees the folder changed since.
A burst of events was read whole into memory before any of it was
looked at, on the UI thread. One turn of the event loop now reads
64 KiB at most and looks at each piece as it comes, and a line too
long to keep is dropped to its end, its last piece included.
Hung up on by Hyprland, the theme took its watch up again only when
Folio was left and come back to, so a config reload in between left
the corners as they were. While in use it now connects again after
a pause, and reads the rounding for a reload that went by untold.
Hung up on while Folio stayed in use, the theme tried Hyprland's
event socket once more after a pause. A try that found no socket
was the last, and reloads went untold until Folio was left and
come back to. Each failed try now starts the pause again.
A re-read drops the answers on their way and reads the rows asked for itself.
A row it could not read either stayed marked as being read, so no view could
ask for it again. Such a row is now owed: a view may ask again, the folder's
next re-read tries it again, and a view is told when another view read it.
A view asks for a row's kind, MIME type and extension each time it
makes the row, and each ask read them from the name again: 48 us a
row in /usr/bin. The model now keeps up to 4,096 names' types, until
a folder is found changed.
A tile fits a text file's page into 128 by 92, and the page was kept
as drawn, at 256 pixels: nine times the pixels shown, in memory and as
a texture. The tile now says the most it draws, twice that once zoomed
in, and the page is handed over no larger, scaled smoothly.
Every row held room for a type, an extension, a plugin item's glyph
and details and a folder's first pictures: 224 bytes, mostly unused.
They are kept aside in one block, shared by the rows of one type, and
a row is 80 bytes: 14 MiB less with a folder of 100,000 files open.
Every row held its whole path beside its name, a string of its own
that repeats the folder's. A row now shares one string for its folder
and its path is put together when asked for: about 14 MiB less with
a folder of 100,000 files open. A plugin item keeps its id whole.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant