Skip to content

Harden Workspace: security fixes, correctness and O(change) performance - #9

Open
zac wants to merge 32 commits into
mainfrom
workspace-hardening
Open

zac wants to merge 32 commits into
mainfrom
workspace-hardening

Conversation

@zac

@zac zac commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Hardening, correctness and performance work on Workspace. Every change was reviewed and verified natively on macOS (including FSEvents and APFS/HFS+/exFAT volumes) and on Linux (swift:6.2-noble, unprivileged).

Security fixes

  • Root-jail escape through grapheme-extending scalars (a25ccea). Paths were parsed with Swift grapheme operations, so a combining mark, ZWJ or variation selector after / hid the separator from component splitting while the kernel still saw it. An intermediate symlink could then escape the root despite O_NOFOLLOW, letting a client read, overwrite, delete or create files outside the root. The same mismatch bypassed authorization rules, mounts, overlay deletions, quotas and event filters. Path syntax is now parsed on UTF-8 bytes, and invalid component names are rejected before any *at() call.
  • Case and normalization aliases bypassing policy (fdf6d39). Path identity used canonical-equivalence equality while volumes applied their own case and normalization rules. Alternate spellings could slip past deny rules on case-insensitive APFS, and canonical twins on Linux could share allow rules and quota accounting. Path identity is now byte-exact everywhere, and LocalFileSystem verifies the stored spelling on lookup.

Breaking changes

There are no existing users, so compatibility code was removed rather than kept.

  • Checkpoint store has one format. {"format":"workspace-history","version":1}. Any other or missing marker on an existing history directory is an error asking for the directory to be deleted.
  • Path identity is byte-exact. WorkspacePath and PathName compare and order by UTF-8 bytes. An alias spelling returns ENOENT (lookup) or EEXIST (creation) without revealing the stored name.
  • EILSEQ on HFS+ and exFAT. Creating a name the volume would rewrite (for example NFC on an NFD volume) fails and the entry is removed. Moves that would be rewritten are rolled back.
  • workspace_glob returns a paged object (max_matches, offset, truncation) instead of a bare array.
  • Removed FileSystemChangeSource.changes() and Workspace.watchChanges(at:recursive:); use changeSignals() and events(_:) with a filter.
  • New enum cases. Transaction.Conflict.Kind.structural and Transaction.TransactionError.committing are new, and Checkpoint.Origin.archiveRestore is removed. Change signals are now the FileSystemChangeSignal type, scoped .entry, .shallow or .recursive, or .rescan.
  • LimitedFileSystem consistency rule. Quota decisions see only change signals processed before the decision; revalidate() forces a fresh walk. accountingCacheEntryLimit caps retained accounting.
  • InMemoryFileSystem creates missing parent directories on write, matching LocalFileSystem exactly (permissions, ENOTDIR, symlink handling, quota charging).
  • WorkspaceToolError is now a struct instead of a two-case enum. The tool adapter returns structured errors with stable codes (not_found, permission_denied, checkpoint_not_found, limit_exceeded, …), a requested path where relevant, and a short actionable message. It never includes host paths, stored spellings or policy deny messages.
  • History API. Compaction now removes history records that every kept checkpoint already includes. History is synced before any checkpoint is published. Paging uses history(after:limit:) and history(before:limit:).

Other fixes

  • A FIFO operation gate serializes workspace mutations; edit capture and rollback are scoped to the paths touched.
  • The checkpoint-store deadlock, torn-log recovery and cross-instance history issues are fixed.
  • Three-way transaction merges are structurally safe.
  • Limits are enforced atomically, and accounting is exact for local disks and dangling links.
  • Tool-adapter output is bounded, and search inputs are hardened.
  • FSEvents callbacks and teardown are serialized.
  • New history directories are published atomically.
  • Store locks never block a worker thread: under contention, blocking flock waits could exhaust Linux's shared worker pool while the async lock holder waited to resume (a deadlock).
  • In-flight atomic-write temporaries no longer leak into listings, walks, recursive copies or change events.

Performance

WorkspaceBenchmarks is a new release-mode harness (swift run -c release WorkspaceBenchmarks --preset medium). Median times on an Apple-silicon Mac, origin/main vs this branch.

Local backend; "on-disk store" is a directory-persisted checkpoint store. Medium is 10k files / 200 MiB; large is 100k files / 100 MiB.

Scenario Medium before Medium after Large before Large after
Preview an edit 748 ms 0.44 ms 23.6 s 2.1 ms
Transaction (on-disk store) 2.55 s 213 ms 59.2 s 3.57 s
Restore (on-disk store) 3.15 s 103 ms 66.6 s 1.85 s
Diff (on-disk store) 1.34 s 84 ms 18.1 s 1.55 s
First checkpoint (on-disk store) 4.03 s 898 ms 45.7 s 7.85 s
Incremental checkpoint (on-disk store) 381 ms 95 ms 6.87 s 1.24 s
Search 2.98 s 274 ms 19.1 s 2.69 s
Glob 55 ms 34 ms 773 ms 406 ms
Limited write 61 ms 0.42 ms 990 ms 0.78 ms
External change event 729 ms 54 ms failed 57 ms
Write one file 0.76 ms 0.90 ms 1.19 ms 1.19 ms

Full reports, with I/O counts, come from --json / --compare.

On macOS, directory metadata is now read in bulk (getattrlistbulk) where the volume reports it exactly. In a paired comparison, this made incremental checkpoints 11–16% faster and glob 20% faster, with identical I/O counts.

Testing

  • swift test: 419 tests on macOS (native, including FSEvents) and 415 on Linux (swift:6.2-noble, as root with coverage at 2 and 4 CPUs, and unprivileged), warning-free with -warnings-as-errors.
  • Scripts/check-bulk-attributes.sh --volume passes on APFS, HFS+ and exFAT disk images.

zac added 30 commits October 8, 2026 20:05
Mutating Workspace operations (apply, checkpoint creation and restore,
transaction commit, archive restore, compaction and filesystem-observation
refresh) now run through a per-workspace FIFO operation gate, so a failed
atomic apply can no longer roll back a concurrent caller's successful edit.

Edits capture and restore only the subtrees they can affect, canonicalized
through ancestor symlinks, instead of the whole tree. A single write no
longer reads every file, and rollback no longer deletes files that other
processes created during the edit.

Rollback never chmods symlinks, only sets permissions that differ, handles
dangling and out-of-root links, rethrows the original error, and reports
both errors via WorkspaceRollbackError when recovery itself fails. Store
state is loaded before the first mutation, and recording failures roll
the edit back. Retracted mutations keep a sequence high-water mark so
numbers are never reused.

RecordingPolicy now shapes stored history only; previews, results and
events always carry diffs up to DiffOptions.default.maxTextBytes.
…date archives

workspace_search no longer traps on extreme context_lines: integers decoded
from tool JSON are validated, capped, and combined with overflow-safe
arithmetic. workspace_read pages by offset/max_bytes with nextOffset and
truncated fields, and search gains an encoded-output budget with
match_offset paging, so a single tool call can no longer return an
unbounded result. Plain Workspace.search calls stay unbounded by default.

TextDiff now splits lines on LF at the UTF-8 level, so CRLF files diff and
patch line by line with correct statistics.

Workspace.Archive.validate() checks the root, immediate-child structure and
unique names of a (possibly decoded) archive before restore touches the
workspace, so a malformed archive can no longer mutate outside its
destination.
LocalFileSystem now resolves paths descriptor-relative from a pinned root
descriptor, walking components without following symlinks implicitly and
performing each operation with *at() calls on the resolved parent, so an
ancestor swapped for an outside symlink between check and use can no longer
escape the root. Moves use an exclusive rename where the filesystem supports
it, falling back to check-then-rename on the same pinned descriptors.
resolveRealPath maps through realpath and no longer collapses to "/" when the
root has a symlinked spelling, and chunked reads are demand-driven.

AuthorizedFileSystem authorizes the resolved target as well as the lexical
path for dereferencing operations, so a symlink under an allowed prefix can
no longer reach a denied file. Deny rules now match if any endpoint of a
copy, move or hard link matches; allow rules still require all endpoints.

OverlayFileSystem resolves every path component through the merged namespace,
so whiteouts and copy-up can't be bypassed via an ancestor symlink.
…istory

FileCheckpointStore serializes in-process callers through an async gate
before touching the OS flock, takes uncontended locks inline, and waits for
contended ones off the cooperative pool, so concurrent checkpoint capture
and queries on one store no longer deadlock.

mutations.jsonl is split on raw LF bytes and decoded per record, so an
interrupted append that ends mid-UTF-8 character no longer empties history;
only an incomplete final record is tolerated.

Checkpoint reads validate negative lengths and forward ranges to the store.
Checkpoint mutation cursors come from the store's sequence high-water mark,
read under its lock before capture starts. history() refreshes persisted
records incrementally by sequence, crosses permanent gaps, and does a full
reload when another instance retracts or prunes records (tracked by a
retraction epoch stored with the high-water mark; legacy sidecars still
decode). Checkpoint event delivery is tracked separately from the query
cache.

history() now reads the store and waits for in-flight mutating operations,
so it reflects other instances and throws if the stored log is unreadable
instead of returning a stale cache.

Size+mtime blob reuse follows git's racy-entry rule, and
Workspace(checkpointCapture: .verifyContents) opts into always rehashing.
…cher edge cases

LimitedFileSystem serializes preflight and mutation through a FIFO gate, so
concurrent writes can no longer jointly exceed a limit. Accounting comes
from metadata (stat sizes, entry counts, file identity and link count)
instead of content snapshots, counts dangling symlinks, charges hard-link
aliases under a documented logical-bytes policy, resolves symlinks through
the backend so mounts can't be used to bypass quotas, and lets an
over-quota workspace shrink back under its limit.

FileInfo gains optional identity and linkCount, and FileSystemFeatures
gains replacementFollowsFinalSymlink, so wrappers can account exactly
without a separate protocol.

InMemoryFileSystem throws EISDIR instead of silently replacing a directory
with a file, and its exists/glob use lstat semantics, so dangling symlinks
survive snapshots and checkpoints. MountedFileSystem lists empty base
directories, propagates backend errors, keeps stat consistent with
listings, and includes nested mount points in directory listings.

On Linux, recursive inotify watches include hidden directories, re-register
replaced directories, retire watches in one batched pass (removing
thousands of directories no longer stalls notifications), and treat the
parent watch and unreadable subdirectories as best-effort.
A transaction that removes or replaces a directory while the parent changes
something beneath it (or the reverse) now reports a single .structural
conflict at the outermost replaced directory instead of silently dropping
the descendant. Descendant deletions converge with an ancestor's removal,
identical changes coalesce, and every merged node is checked for a
surviving directory ancestry before anything is written.

Root-directory metadata participates in conflict detection and merging, so
root permission changes are no longer silently dropped.

Transactions enter a .committing state before the first await: edits,
discard and a second commit throw TransactionError.committing while reads
of the draft still work, and a rollback failure permanently closes the
transaction.

Snapshot.restore applies a directory's final permissions after its
children, granting temporary owner access only when the filesystem actually
denies it and restoring the original mode on failure, so restrictive
directory modes no longer block their own contents.
LocalFileSystem reports file identity (device, inode) and link count for
non-directories, so LimitedFileSystem charges hard-link aliases exactly on
disk instead of falling back to a conservative bound that could reject valid
appends. Local advertises replacementFollowsFinalSymlink: replacement writes
follow in-root final links and atomically replace the target, while links
that escape the root are rejected.

LimitedFileSystem resolves a dangling final link through the backend's own
physical resolution (a new internal potential-path protocol forwarded by
Mounted, Sandbox and SecurityScoped), so relative targets reached through
symlinked parents and absolute in-root host paths are charged where the
write actually lands. InMemoryFileSystem writes resolve links the same way
its reads do.

OverlayFileSystem and AuthorizedFileSystem now give exists (and exact-path
glob) lstat semantics like the other backends, so dangling symlinks survive
snapshots and checkpoints through an overlay.
…l output

TextDiff trims common prefixes and suffixes and runs a bounded Myers search
(up to 1,000 edits or 1,000,000 steps, with a separate budget per
intra-line pair), falling back to one replacement hunk with exact
statistics only for genuinely large edit distances. Checkpointing a
100k-line new file no longer takes tens of seconds.

The in-memory store caches revision indexes per immutable snapshot, and
checkpoint pages resolve once and use ranged reads, so paging through a
checkpoint file no longer re-hashes the whole snapshot on every page.
Checkpoint reads resolve intermediate symlinks within the workspace, and
directory-backed capture keeps dangling and cyclic links.

workspace_glob, workspace_diff and workspace_apply results are bounded by
encoded-size budgets with paging and explicit truncation fields, accounted
incrementally. workspace_apply never reports failure after its edits have
committed; it falls back to a minimal summary instead. workspace_glob now
returns an object ({matches, totalMatches, ...}) rather than a bare array.
Workspace parsed POSIX paths with Swift grapheme-cluster operations. A
grapheme-extending scalar following `/` (e.g. a combining mark, ZWJ,
variation selector or spacing mark) could hide that separator from component
splitting and containment checks, while the kernel still treated its UTF-8
byte as a separator. LocalFileSystem could therefore pass several components
to one descriptor-relative syscall, so an intermediate symlink escaped the
root despite O_NOFOLLOW. Because createSymlink accepts arbitrary targets, a
workspace client could read, overwrite, delete or hard-link existing files
and create new files and directories outside the root in any directory a
symlink could name, subject to host filesystem permissions, provided the
name began with such a scalar. The same mismatch let paths bypass
authorization rules, mount routing, overlay deletions, descendant-move
protection and quotas, and broke checkpoint, archive and revision lookups
and recursive event filters.

This fix parses path syntax, including globs, on UTF-8 bytes; compares
containment component-wise with the same canonical-equivalence rules as
WorkspacePath equality; and rejects empty, `.`, `..`, slash- or
NUL-containing component names before `*at()` calls, except the explicit `.`
reference to an already-resolved directory descriptor.
Workspace used Swift canonical-equivalence equality for path identity while
host filesystems applied different case and Unicode-normalization rules.
Alternate case spellings could bypass deny rules on case-insensitive volumes
such as APFS, while distinct canonical twins on Linux could inherit another
path's allow rules and collapse quota accounting. Canonically equivalent
sibling names could also trap parented checkpoint saves and revision diffs
when String-keyed child maps rejected them as duplicates.

This fix makes path identity byte-exact throughout filesystem and history
operations. LocalFileSystem verifies spelling per lookup using stored-name
attributes or an exact directory-listing match, with known byte-exact
filesystems skipping verification. Creations that a filesystem rewrites
(HFS+, and listing-verified volumes such as exFAT) are removed and rejected
with EILSEQ; rewritten moves are rolled back. Lookup aliases remain ENOENT
and conflicting creation or mutation destinations remain EEXIST, without
disclosing the stored spelling.
WorkspaceBenchmarks (swift run -c release WorkspaceBenchmarks) generates
deterministic small/medium/large fixtures and runs each scenario in a fresh
child process against LocalFileSystem and InMemoryFileSystem with memory and
directory persistence. It records wall time, exact FileSystem-boundary I/O
counts from a counting wrapper, checkpoint-store bytes written, and memory
(setup high-water, per-operation increase, and held RSS), writes JSON, and
compares against a baseline with --compare.

The potential-path resolver protocol and its witnesses move from internal to
package access so the counting wrapper can forward it without changing
LimitedFileSystem's resolution path; the public API is unchanged.
FileCheckpointStore moves to format v2. Each directory is a
content-addressed tree of name-only, byte-sorted entries, so an incremental
checkpoint writes only the trees on changed paths and identical subtrees
are shared. Small blobs go into immutable pack files, synced in batches
before the snapshot root publishes, and publication uses F_FULLFSYNC on
Darwin where supported. Reads decode only the trees on the requested path,
through a bounded LRU of decoded trees and catalog entries. The in-memory
store carries content hashes over from capture instead of re-hashing
snapshots.

Existing v1 stores stay readable and appendable after an on-open marker
migration. v2 roots use a key that v1 code cannot decode, so an old process
still running during an upgrade fails closed instead of treating the new
trees as empty. Compaction handles both formats, packs, interrupted
relocation and orphan sweeping.

On the medium benchmark fixture, the first directory checkpoint drops from
3.9 s to 1.1 s, an incremental checkpoint writes about 29 KB instead of
4.8 MB, and a warm checkpoint read takes 0.4 ms instead of 61 ms.
LocalFileSystem gains a package-internal bulk traversal: one walk per call
holds directory descriptors for its duration (capped at 32 levels, then an
identity-checked per-path fallback), enumerates entries with lstat
semantics relative to the parent descriptor, and reads files relative to
it. Snapshot capture, search, glob and LimitedFileSystem accounting use it;
wrappers forward it or fall back, and AuthorizedFileSystem keeps per-entry
authorization. readdir names are the stored spelling, so walks take them
without a spelling recheck, and children are ordered by UTF-8 bytes to
match WorkspacePath ordering. The resolve fast path opens ancestors
directly through spelling-verified directory descriptors.

Search compiles its include/exclude selection into the single content walk,
detects binary content with memchr, matches ASCII literals on bytes with an
ASCII case-fold path, runs certified line-local regexes in one pass, and
builds line strings only for matches and context; results are identical to
the previous algorithm. Large reads allocate from the observed size and
grow geometrically.

On the medium benchmark fixture, search drops from 2.9 s to 0.25 s and
preview from 670 ms to 270 ms.
Transactions freeze an indexed base and give the draft a copy-on-write
overlay. Commit finds parent-side changes from metadata against that base,
applying the store's racy-mtime rule (re-reading anything modified within
two seconds of the previous capture, and always under .verifyContents), and
merges and rolls back only the changed paths. A cold begin keeps up to
256 MiB of the base in memory and spools the rest privately, releasing it on
commit, discard or failure; later transactions re-seed from the committed
checkpoint.

Preview captures only the edits' affected roots and applies them to an
overlay that follows the backing filesystem's semantics, so its ChangeSets
match apply's.

Filesystem observation keeps a metadata index built with the bulk traversal
instead of a full-content snapshot, plus an LRU content cache for text
diffs (ObservationPolicy). Change sources report scoped signals: an entry, a
directory listing, or a recursive subtree. FSEvents and inotify map their
events to the narrowest correct scope, so one external write is handled
without walking the tree. Failed refreshes keep their pending changes and
retry a bounded number of times, and unreadable files are reported as
changes with a content-read omission.

On the medium benchmark fixture, a steady transaction drops from 613 ms to
121 ms, preview from 287 ms to under 1 ms, and handling one external change
from 279 ms to 58 ms.
FileCheckpointStore's capture now walks the source with the bulk traversal
when the filesystem provides one, falling back to the per-path walk
otherwise. Both routes share the same reuse and persistence logic and
produce byte-identical trees and pack layouts; content is read only for
entries that can't be reused under the racy-mtime rule. Entries that vanish
between listing and lookup are skipped on both routes instead of failing
the capture.

Prior-directory state is prepared lazily from cached tree entries with a
binary lookup, capture-time tree-cache inserts are scan-resistant within the
existing budget, and the visitor tracks ancestry with a stack instead of
full-path hashing.

On the medium benchmark fixture an incremental directory checkpoint drops
from about 220 ms to 89 ms, and on the large fixture from 6.1 s to 1.5 s.
restore(to:) and diff(from: .checkpoint, to: .current) no longer capture
the whole current tree. They compare the checkpoint's persisted capture
metadata with one bulk metadata walk of the current tree, applying the
same racy-mtime and identity rules as transactions, and read and hash only
entries whose metadata differs or can't be trusted. Restore writes only the
differences, records sparse before/after snapshots for history and failure
recovery, and publishes the restored checkpoint by reusing stored blobs
and trees. diff(.current, .current) returns without touching the
filesystem.

Snapshot capture keeps a per-entry stat on its per-path route, so
AuthorizedFileSystem's per-entry checks still apply, and InMemoryFileSystem
gains a native actor-owned bulk walk that takes the fast route instead.

On the medium benchmark fixture a directory-persistence restore drops from
about 1.3 s to 106 ms (reading 12 KiB instead of 604 MiB), and a checkpoint
diff from about 310 ms to 89 ms.
LocalFileSystem's FSEvents observation delivered callbacks on a concurrent
global queue while cancellation stopped, invalidated and released the
stream and its context box on whatever thread ended the subscription. A
callback still in flight could then read the freed box (EXC_BAD_ACCESS in
the event mapper), which crashed roughly one in four runs of a probe that
tore down an observation while its tree was changing.

The stream is now scheduled on a private serial queue and teardown is
queued on that same queue, so it can never overlap a callback. The Linux
inotify path, whose teardown was already serialized, now handles watch
exhaustion and recovers watches once a directory becomes readable. A stress
test creates and tears down observations in a loop while generating events.
LimitedFileSystem keeps its accounting (totals plus per-path kind, size,
identity and link count, and alias indexes) between mutations instead of
walking the whole tree on every preflight. Successful mutations reconcile
the affected paths under the FIFO gate; failures and uncertain alias
effects invalidate. When the base declares complete change coverage
(changeSignalsAreComplete: LocalFileSystem and the forwarding wrappers;
MountedFileSystem only when every layer does; never AuthorizedFileSystem,
which filters signals), signalled scopes are invalidated and re-walked
lazily. InMemoryFileSystem exposes an exact mutation generation instead.
Other backends, and trees past an entry cap, keep walking as before.

Accounting is as of the change signals processed before a projection
finishes; revalidate() forces a fresh walk.

On the medium benchmark fixture a write through LimitedFileSystem drops
from about 21 ms to 0.19 ms, and on the large fixture from about 277 ms to
0.36 ms.
The racy-window tests stamped files with the current second and relied on
the next capture starting within two seconds, so they failed under load.
They now use timestamps that are always inside the window and overwrite
content in place, so the racy rule, not an identity change, is what forces
the rehash.
If the shared fixture's file count or total bytes change during a run, the
harness now stops with an error naming the last scenario that ran, instead
of letting later scenarios fail on a partial tree.
FileIdentity includes the device number, which can change across reboots
or remounts for external volumes, container overlays, NFS and some btrfs
setups. A device-only change made every persisted identity mismatch, so the
next checkpoint rewrote every tree and re-read every file, and restore,
diff and transactions lost their fast path.

When a persisted or cached identity differs from the current one only in
device number, reuse and change detection now treat identity as unknown
and fall back to size, mtime and the racy-window rule. A real replacement
(different inode) still forces a read. Hard-link accounting and hashing
keep exact device-and-inode identity.
Nothing has shipped, so earlier on-disk formats, serialized shapes and API
spellings are no longer supported. The checkpoint store has a single format
marker ({"format":"workspace-history","version":1}); any other or missing
marker on an existing history directory is an error that asks for the
directory to be deleted. Format migration and its nested locks, legacy
inline-children manifests, persisted .missing children, the fail-closed
treeRoot key, integer sequence sidecars and silent field defaults are gone;
missing required fields now raise storageCorrupted.

Search request and result types use synthesized Codable. The Void
FileSystemChangeSource.changes() stream and Workspace.watchChanges(at:recursive:)
are removed in favour of changeSignals() and events(_:) with a filter, along
with several internal aliases, the v1 store fixture and compatibility-only
tests.
A new workspace history directory is now staged with its synchronized
format marker and renamed into place, followed by a sync of the parent
directory, so a crash during first open leaves only disposable staging
instead of an unmarked directory that later opens reject. Materialized
saves drop .missing children the same way capture does, two unused
persistence helpers are removed, and the capture-route parity test pins
in-memory timestamps so its racy-window expectations no longer depend on
scheduling.
Checkpoint-store Node and TreeEntry and MetadataIndex.Entry compare exactly
again, matching what they encode. Reuse decisions that tolerate device
renumbering (directory and tree reuse, compareNodes, publishRevision,
compaction relocation, observation) use a named isReusable(as:) instead.

LimitedFileSystem takes an accountingCacheEntryLimit initializer parameter:
nil keeps the platform default (250,000 entries, 25,000 on iOS and
watchOS), and 0 never retains accounting between mutations.
The README is reorganised around what an adopter needs: when to use the
package, the core model, the main tasks, and a single section of
guarantees and limits. Internal storage, locking and syscall detail is
removed, repeated explanations are merged, every example is checked to
compile and run against the public API, and benchmark detail moves to
Benchmarks/README.md.
Local traversal now reads each directory's entries and their metadata with
getattrlistbulk instead of readdir plus one fstatat per child. Regular
files and symlinks take their kind, size, mode, mtime, identity and link
count from the bulk record; directories, mount points and special files
keep fstatat. Bulk names are only trusted under spelling policies where the
returned name is the stored name, and they still pass entry validation.

The access mask can report an effective rather than stored mode: APFS
drops set-ID bits when the owner is unknown, as on nosuid/noowners mounts.
Mounts with MNT_NOSUID, MNT_IGNORE_OWNERSHIP or MNT_QUARANTINE, and entries
whose owner is unknown, fall back to fstatat. Partial or malformed batches
are discarded in favour of the fallback, and a strict walk still reports a
child that vanishes after listing.

Scripts/check-bulk-attributes.sh runs the parity test on a mounted volume;
it passes on APFS, HFS+ and exFAT images. On a native paired A/B against
the previous commit, directory-persisted incremental checkpoints are
11-16% faster and glob 20% faster, with identical I/O counts; search is
unchanged.
Compaction now removes history records that every retained checkpoint
already includes: those at or before the smallest retained mutation cursor,
also with `.all`. With no checkpoints, history is kept. The log is
validated before compaction changes any checkpoint or blob, then rewritten
under the mutations lock into a synced replacement that is renamed into
place. The sequence high-water mark and a cache epoch are durably bumped
first, so sequence numbers are never reused and other instances drop
stale caches. CompactionReport gains removedMutationCount and
removedMutationBytes; dry runs project them.

Every checkpoint publication path (create, transaction commit, restore and
compaction rebases) syncs the history log and sequence sidecar first, with
F_FULLFSYNC on Darwin, so a published checkpoint never refers to history a
crash can lose. Ordinary appends stay unsynced.

history(after:limit:) returns records oldest first and
history(before:limit:) newest first, with exclusive cursors. Pages
binary-search the sorted log on sequence numbers alone and decode only the
records they return: a five-record page of a 1,000-record log does five
full decodes. Negative limits throw.
…case

InMemoryFileSystem now creates missing parent directories on write, copy,
move and symlink creation, matching LocalFileSystem: the same physical
resolution (including writes through dangling symlinks and missing
components before `..`), permissions from the umask, ENOTDIR for a file in
an intermediate position, and quota charging for created parents through
Limited. Copying or moving a directory into its own subtree fails with
EINVAL on every backend before any parent is created. A parameterized
matrix runs each route against Local first and requires identical trees
and errors from memory and overlay. Edit capture follows the created
parents; it falls back to whole-tree capture only for symlink targets
with a `..` after another component.

WorkspaceToolAdapter maps failures to WorkspaceToolError, a Codable error
with a stable code (not_found, not_a_directory, already_exists,
permission_denied, operation_not_permitted, checkpoint_not_found,
limit_exceeded, conflict, invalid_arguments, ...), the requested path or
scope for path-scoped failures, and a short message a model can act on.
Argument errors name the field and the problem; limit errors name the
limit; a read that races a write asks for the page to be retried. Messages
are fixed strings: host paths, stored spellings, error userInfo and policy
deny messages are never forwarded.

Checkpoint.Origin.archiveRestore is removed; nothing produced it.
zac added 2 commits October 20, 2026 23:52
…aries

Store locks waited in a blocking flock on Dispatch's global queue. On Linux,
where Swift's cooperative pool and the global workers are shared, enough
contending stores in one process could occupy every worker while the
current holder, an async task, waited for a thread to resume and release
the lock. Locks are now taken with LOCK_NB, retrying after asynchronous
10 ms delays; uncontended acquisition is immediate, and lock order,
cancellation and durability barriers are unchanged. A 256-store contention
test hangs on the old code and finishes in about two seconds.

LocalFileSystem's atomic writes stage a process-private temporary beside the
destination. Change streams already ignored those names, but directory
listings, walks and recursive copies did not. A shallow refresh racing a
write could report the temporary as a user change, a listing could fail
with ENOENT when it was renamed, and a recursive copy could duplicate it
into the destination as a stray file. Listings, walks and copies now skip
this process's in-flight temporaries; removal still sees them.
On a 4-CPU runner with the full suite in parallel, a 10 ms sleep was
measured at 3.9 s, so 2-5 s deadlines for expected events expired before
their waiters ran. Positive waits now allow 30 s and return as soon as the
condition holds, and the timeout helper keeps an element that arrives
alongside the timeout. Short "nothing else arrives" windows now start after
a delivery marker, which makes them stricter. Retry tests wait for the
retry state machine to finish, Limited tests wait for both byte and entry
counts, and a 2 ms permission-cache race is gone. The bulk watch-retirement
check compares process CPU time against same-run setup cost instead of a
wall-clock bound, and still requires the removal to be reported. A
quadratic implementation fails it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant