Skip to content

perf: prepare recording sources during background uploads - #2255

Merged
richiemcilroy merged 4 commits into
mainfrom
codex/recording-background-preparation
Sep 9, 2026
Merged

perf: prepare recording sources during background uploads#2255
richiemcilroy merged 4 commits into
mainfrom
codex/recording-background-preparation

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Sep 9, 2026

Copy link
Copy Markdown
Member

Desktop Instant recordings previously waited until Stop to preserve every uploaded source fragment. Both desktop uploaders now prepare closed, successfully uploaded fragments in bounded background batches so finalization can reuse those validated copies.

Preparation is optional, owner-scoped, and cancelled without delaying completion. It retries each fragment at most twice, backs off outages, stops scheduling after finalization starts, and falls back to the existing source commitment path when preparation is missing or invalid. Source identity, audio, completion proof, and final media verification requirements remain unchanged. Initialization files are still captured after Stop because they can change during recording.

Copies stay within S3 or Google Drive. Vercel receives metadata only: at most 32 references per request, one request every 30 seconds, four concurrent copies, and bounded execution. No schema migration is required; older clients and servers remain compatible. Final muxing and full verification still happen on the media worker, so this does not promise constant-time finalization.

Live rollout checks exposed S3 connection exhaustion during source copying. Storage access previously constructed fresh clients and keep-alive pools for every checkpoint. The change reuses default clients and a scoped connection pool for default and custom buckets, bounded to 50 connections per host and 128 per protocol. Idle connections expire without adding an active-copy timeout, and custom credentials are still resolved on every access. A real HTTP regression test reproduced 320 connections across 40 checkpoints before the fix and at most 16 afterward; concurrent recording and credential-rotation checks also pass.

Validation:

  • 586 web and storage regression tests, scoped TypeScript checking, Biome, and workflow YAML parsing passed.
  • Seven real production recordings supplied 1,623 private source files. Four complete recordings (S3/Drive, 16 seconds to 27.5 minutes, with and without audio) passed Linux muxing, packet/timing comparison, full decode, and source-preservation verification: 53,776 video frames. Three additional long/retried/source-blocked recordings supplied partial-source refusal cases. No customer objects or database rows were modified.
  • Eight production-corpus replay tests, 45 Tauri upload tests, 28 GPUI upload tests, and five shared preparation tests passed. The existing 50 ms Tauri scheduler assertion passed alone and with serial tests but was sensitive to parallel CPU contention.
  • Clippy passed for Tauri and the recording library; Rust checks passed for both desktop clients. GPUI used installed Rust 1.98 because an existing dependency requires Rust 1.89 or later.
  • Final Greptile CLI review 27c05759-b6f1-4b94-bbf3-36d86cf1cc42 includes the connection-pool fix: 5/5, no actionable findings.

Updated desktop binaries still require packaged native recording/upload verification before release. Production replay timings exclude provider/network latency; post-release readiness and cost measurements remain necessary.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; the latest connection-pool changes address the reported resource exhaustion without introducing an actionable regression.

Summary

  • Adds optional preparation scheduling to both desktop upload implementations.
  • Adds an owner-scoped preparation API and source-validation logic.
  • Reuses bounded HTTP and HTTPS agents across default and custom S3 bucket clients.
  • Adds recording replay, preparation, source-preservation, and real HTTP connection-reuse regression coverage.
  • Extends the recording-reliability workflow to run the new tests.

@richiemcilroy
richiemcilroy merged commit 30dd00e into main Sep 9, 2026
30 checks passed
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant