Skip to content

perf: stop rewriting unchanged sessions every pass and share one app snapshot per poll - #86

Merged
vishr merged 2 commits into
mainfrom
perf/collector-pass
Oct 30, 2026
Merged

vishr merged 2 commits into
mainfrom
perf/collector-pass

Conversation

@vishr

@vishr vishr commented Oct 30, 2026

Copy link
Copy Markdown
Member

What changed

The collector no longer rewrites every session on every pass. It keeps the committed rows in memory, writes only sessions that changed, and links parents only among sessions that can still change. The app and driga dev serve share one data snapshot per poll interval across concurrent page requests, and they gain --cpu-profile and --profile-for like service run.

Why

On the owner's Mac, Driga's own service and app each used about one CPU core, and the app's memory grew to about 760 MB. A 60 s profile of the live service (23 live agents, 1,616 stored sessions) put 31% of samples in the collector pass transaction:

  • 12% rewriting unchanged sessions;
  • 11% relinking parents across all history;
  • 22% in garbage collection from the churn.

The rewrites also grew the WAL to 54 MB.

Risk and rollout

  • No migration, and nothing written to agent settings.
  • The cached rows are invalidated by SQLite data_version for writes from other connections and by a shared revision for writes through the same store. A failed pass discards them.
  • The snapshot cache serves data at most one poll interval old, which is the same freshness the page already polls at.

Contract changes

  • CLI: driga app and driga dev serve gain --cpu-profile and --profile-for.

Verified

  • Live service, same Mac, same agents, profiled for 60 s each: 39.3% CPU before and 18.5% after (23.60 s of samples against 11.09 s), with no errors.
  • Benchmark, 1,600 stored and 25 live sessions: 274 ms down to 48.8 ms per pass, and 587k down to 166k allocations.
  • App, 60 s of the real poll set on a copy of the real database: 20% less CPU, and retained heap down from 67.6 to 24.8 MiB.
  • TestPassUpsertsOnlyChangedSessions uses a SQLite trigger to confirm a pass with no changes makes no session UPDATE. It failed before the fix with unchanged session updated 1 times.
  • Tests: go test for collector, store, derive, web and viewer, plus -race for web and collector. GOOS=linux go build and GOOS=windows go vet pass.
  • I have accepted the CLA, or will when the bot asks on my first pull request.

vishr added 2 commits October 30, 2026 11:47
…snapshot per poll

Compare session facts independently of elapsed mixer presentation and reuse
committed rows between passes. Restrict inferred parent work to live and
recently ended sessions, and resolve explicit edges from one local index.

Keep dashboard single-flight sharing while evicting superseded snapshots.
Batch MCP history clock reads, and expose CPU, heap and memory profiling
for the desktop app and development server.

The 1600-session, 25-live benchmark drops from 274 ms and 587374 allocations
to 49 ms and 166243 allocations per pass. Native and UTC package tests,
web cache race tests, and native, Linux and Windows vet checks pass.
@vishr
vishr merged commit 6f6c0bc into main Oct 30, 2026
7 checks passed
@vishr
vishr deleted the perf/collector-pass branch October 30, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant