Skip to content

perf(table): reuse RowScanner across Iter rows - #57

Merged
VojtechVitek merged 2 commits into
masterfrom
iter-scanner-reuse
Jul 31, 2026
Merged

perf(table): reuse RowScanner across Iter rows#57
VojtechVitek merged 2 commits into
masterfrom
iter-scanner-reuse

Conversation

@VojtechVitek

Copy link
Copy Markdown
Member

What

Table.Iter created a fresh scany RowScanner on every row via pgxscan.API.ScanRow, recomputing the column-to-field mapping each iteration. This creates one RowScanner before the loop and reuses it, so the reflection work happens once per query instead of once per row.

Motivated by this write-up on pgx.CollectRows — the same per-row reflection anti-pattern it describes was present in Iter. GetAll/List/pagination already reuse a single RowScanner internally, so this only affects the Iter streaming path.

Commits

  1. test(table) — extracts the Iter scan loop into iterRows and adds BenchmarkTableIter, driven by an in-memory fake pgx.Rows (no DB needed, runs in CI). Baseline.
  2. perf(table) — the one-line-idea change: reuse the RowScanner.

Result

BenchmarkTableIter via benchstat, count=8, all deltas p=0.000:

Metric (1000 rows) Baseline Improved Δ
time 394.8µs 177.5µs −55%
B/op 406.6 KiB 63.2 KiB −84%
allocs/op 6002 1008 −83% (~6/row → ~1/row)

Testing

  • go test ./tests/... (Postgres-backed) passes, including TestIter and TestTable.
  • go build ./... and go vet ./ clean.

🤖 Generated with Claude Code

VojtechVitek and others added 2 commits July 31, 2026 15:24
Extract the Iter scan loop into iterRows so it can be driven directly,
and add a DB-independent benchmark using an in-memory fake pgx.Rows.

This measures the per-row scan cost of the current implementation, which
creates a fresh scany RowScanner (recomputing the column-to-field mapping)
on every row via pgxscan.API.ScanRow. Serves as the baseline for the
scanner-reuse optimization in the next commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Iter created a fresh scany RowScanner per row via pgxscan.API.ScanRow,
recomputing the column-to-field mapping on every iteration. Create one
RowScanner before the loop and reuse it so the reflection work happens
once per query instead of once per row.

Benchmarked via BenchmarkTableIter (in-memory fake pgx.Rows), count=8:

  rows=1000   time  394.8µs -> 177.5µs  (-55%)
              B/op  406.6Ki ->  63.2Ki  (-84%)
              allocs   6002 ->    1008  (-83%, ~6/row -> ~1/row)

All deltas p=0.000. GetAll/List/pagination already reuse a single
RowScanner internally, so this only affects the Iter streaming path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VojtechVitek
VojtechVitek merged commit 494ee1b into master Jul 31, 2026
1 check passed
@VojtechVitek
VojtechVitek deleted the iter-scanner-reuse branch July 31, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants