Repository navigation
Query provider databases in process and stop blocking cooperative threads on subprocess waits - #38
Merged
Merged
Conversation
ProfileStatsScanner and LocalActivityReader launched /usr/bin/sqlite3 for every query, about five processes per Codex profile per refresh. They now use the SQLite C API through SQLiteReadOnly.query: - the existing path validation still runs first, and immutable=1 is used only when no -wal or -shm sidecar exists; - connections open READONLY | NOFOLLOW, set trusted_schema=OFF and query_only=ON, and refuse statements SQLite does not report read-only; - a progress handler interrupts at the timeout or on task cancellation; - output bytes and rows are counted as values are copied, keeping the 8 MiB stats budget and the activity byte budget plus a row limit at its LIMIT; - rows arrive typed and in order, so values containing newlines or the separator no longer split rows, and the readabilityHandler / readDataToEndOfFile ordering race in queryRows is gone with the pipe; - non-UTF-8 text, or a missing or mistyped required activity column, fails that source instead of being repaired. The fake-sqlite3 timeout test is replaced by real slow views, and the benchmark runner counts sqlite3 launches only where they still happen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BoundedSubprocess.run busy-polled waitpid and a semaphore with Thread.sleep, and AppServerRateLimitClient polled semaphores and isRunning every 100 ms, on Swift concurrency's cooperative threads. BoundedSubprocess now drives each process from dispatch sources on a private serial queue: root exit, pipe output, the deadline, and the SIGTERM-then-SIGKILL group escalation. An async overload bridges completion to a continuation with a cancellation handler, so async callers suspend; the synchronous overload waits on one semaphore for synchronous callers. The exit event can precede the zombie state, so its handler reaps with a blocking waitpid on that private queue rather than WNOHANG. The app-server quota read is async end to end. It awaits initialize, response, output-closed, and process-exit milestones from the readability handler and a process source, and GroupedSubprocess.terminate suspends between group checks instead of sleeping. Timeouts, process-group kill, cancellation, and output limits are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CI toolchain's region-based isolation checker rejects a dynamic Self capture inside the Task.detached closure; read the query into a local first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lastActiveOrg lookup was the last provider-database sqlite3 launch on the refresh path. It now uses SQLiteReadOnly.query, which gains an `under:` root forwarded to databaseArgument so the profile-root containment check is kept. The query returns host_key, the plain value, and the encrypted value as typed text and blob columns instead of a hex-and-pipe string; decryption, host-hash and UUID validation are unchanged, as are the 2 s timeout and 4 KiB budget, now with a one-row limit. Any query failure still yields no organization. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Desktop status refreshes blocked a launcher actor's cooperative thread for up to 3 s per `ps` call. The read-only status reads (`statuses`, `statusBatch`, `stockStatus`, and Claude's `status(for:appURL:)`) now await an async snapshot, so the wait suspends. `CodexProcessSnapshotProviding` gains `suspendingProcessSnapshot()`, whose default calls the synchronous snapshot, so existing synchronous conformers still conform. The system Codex and Claude providers implement it with the async `BoundedSubprocess.run` and share one set of `ps` arguments, timeout, and output bound with their synchronous reads. Open, close, and focus keep synchronous snapshots and gain no suspension point. Official Codex open/close use a private synchronous stock status. Both actors hold only immutable state. Status reads derive results from the snapshot alone (Codex), or run every helper and main-process identity check after it returns with no suspension between them (Claude). Status results never authorize a lifecycle action. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…evisions The runner's baseline branch calls the older per-provider status APIs, so its status rows cannot compare two revisions that both have `statusBatch`. `AGENTDOCK_BENCHMARK_BASELINE_DEFINE=CANDIDATE` opts a baseline into the candidate's calls; the default is unchanged. Record the installed-status comparison for suspending snapshots: no regression, same `ps` launches and validator entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes that make refreshes cheaper without removing any safety check.
1. Codex stats and activity query SQLite in process (241ac6b)
ProfileStatsScanner(queryRows,tableColumns,profileStateRows,querySingleRow) andLocalActivityReaderused to launch/usr/bin/sqlite3for every query, about five processes per Codex profile per refresh. They now go through a newSQLiteReadOnly.query, which uses the SQLite C API:SQLiteReadOnly.databaseArgumentstill validates the path first (parent resolved, symlinked or non-regular database and sidecars rejected), andimmutable=1is still used only when no-wal/-shmexists.SQLITE_OPEN_READONLY | SQLITE_OPEN_NOFOLLOW. They also settrusted_schema=OFFandquery_only=ON, and refuse any statement SQLite doesn't report as read-only.sqlite3_progress_handlerstops a query at the existing timeout or whenTask.isCancelledbecomes true.LIMIT.readabilityHandler/readDataToEndOfFileordering race inqueryRowsis gone, because there is no pipe any more.2. Subprocess waits no longer hold cooperative threads (ae88f7b)
BoundedSubprocessnow runs each process from dispatch sources on a private serial queue: root exit, pipe output, the deadline, and the group SIGTERM → 1 s → SIGKILL escalation. A newasyncoverload hands the result back through a continuation with a cancellation handler. Async callers suspend instead of blocking. Sync callers keep the sync overload, which now waits on one semaphore instead of polling every 10 ms.WNOHANGreap in the exit handler can miss it. The handler now does a blockingwaitpidon the private queue, which returns promptly once the exit event has fired.AppServerRateLimitClientis async end to end. It waits on initialize, response, output-closed and process-exit milestones, which come from the readability handler and a process source.GroupedSubprocess.terminate()suspends between group checks instead of callingThread.sleep. Timeout, process-group kill, cancellation and output-limit behavior are unchanged.Verification
swift test: 416 tests, 0 failures, 20 opt-in skips.PerformanceRegressionTestspasses../script/benchmark_resources.sh 9af151b <out>, SQLite scenario (nine samples per revision):sqlite3launches 5 → 0 per refresh. Wall time 287.6 → 247.9 ms; child CPU 271.7 → 0 ms and parent CPU 4.8 → 244.9 ms, so the work moves into the app. 50,000 sessions and zero query errors in both. The status rows also differ, but only because the runner's baseline branch calls the older status API on purpose; they don't measure this change. Details are indocs/resource-usage-benchmarks.md.git merge-tree); the merged tree keeps itsPOSIX_SPAWN_CLOEXEC_DEFAULTchanges.Notes for review
sqlite3process. The read-only, no-follow,trusted_schema=OFF, timeout and output bounds limit what a hostile database can do, but a SQLite parser bug would now hit the app process.pssnapshots still use the sync runner on the launcher actors, behind sync snapshot protocols.LocalChatSessionandClaudeUsageClientstill launchsqlite3.🤖 Generated with Claude Code