perf(queries): skip unchanged tables with change summaries

ROADMAP.md

@@ -1,6 +1,6 @@
 # Archevolt performance plan
 
-Status: steps 0, 1, 2, 3, 4, 5, 6, and 8 done. Step 7 remains. Step 0's baselines are in `benchmarks/baseline.md`, recorded on .NET 10. Step 3's before and after numbers, on .NET 11, are in `benchmarks/step3-table-storage.md`. Later steps record fresh before and after comparisons on the same SDK. Step 3 met its gate except for named queries, whose regression came from enumerator codegen. Step 2 rewrote that enumerator and met its gate. Its numbers are in `benchmarks/step2-named-query-enumerator.md`. Step 4 replaced directory references with integers and met the median/range gate after an unchanged-code drift control. Its measurements and the remaining variance in large random reads are in `benchmarks/step4-integer-entity-directory.md`. Step 5 made reservation lock-free, added batch spawn, and met its gate. Its numbers are in `benchmarks/step5-lock-free-reservation.md`. Step 6 cached column maps, interned archetypes by bitmask, and met its gate. Its numbers are in `benchmarks/step6-cached-column-maps.md`. Step 8 made both immediate spawns write bundles in place and met its gate. Its numbers are in `benchmarks/step8-in-place-spawn.md`.
+Status: every step is done. Step 0's baselines are in `benchmarks/baseline.md`, recorded on .NET 10. Step 3's before and after numbers, on .NET 11, are in `benchmarks/step3-table-storage.md`. Later steps record fresh before and after comparisons on the same SDK. Step 3 met its gate except for named queries, whose regression came from enumerator codegen. Step 2 rewrote that enumerator and met its gate. Its numbers are in `benchmarks/step2-named-query-enumerator.md`. Step 4 replaced directory references with integers and met the median/range gate after an unchanged-code drift control. Its measurements and the remaining variance in large random reads are in `benchmarks/step4-integer-entity-directory.md`. Step 5 made reservation lock-free, added batch spawn, and met its gate. Its numbers are in `benchmarks/step5-lock-free-reservation.md`. Step 6 cached column maps, interned archetypes by bitmask, and met its gate. Its numbers are in `benchmarks/step6-cached-column-maps.md`. Step 8 made both immediate spawns write bundles in place and met its gate. Its numbers are in `benchmarks/step8-in-place-spawn.md`. Step 7 added per-column change summaries, met its gate, and kept 64-bit ticks. Its numbers are in `benchmarks/step7-block-change-summaries.md`.
 
 The v1 implementation plan is complete. Every stage passed its acceptance gate, and the evidence is in `.audit/acceptance.md`. The [v1 plan](https://github.com/Stephanvs/archevolt/blob/ebacf07059fefbf9b182e40a290812be584dd9ee/PLAN.md) remains in history as the record of the original design, its alternatives, and its gates.
 
@@ -243,4 +243,14 @@ Follow-up, not yet a numbered step: [step3-partition-granularity.md](benchmarks/
 - Step 8 says the `BundleWriter` points at the row's destination columns. The writer keeps its staging form for `Commands` and `InsertBundle` and gains a table form. `BundleBinding.Columns` holds each member's column, so the table form writes without a lookup. The amendment after step 6 said the binding did not need its columns. It needs them now, because `WriteBundle` no longer runs for immediate spawns.
 - The new row sits in the table, but not yet in the directory, while user code in the bundle's `Write` runs. The world counts as running for that window, so a `Write` that reaches the same world through other state cannot move or truncate the row. Step 8's text did not anticipate this.
 - [The step 8 measurements](benchmarks/step8-in-place-spawn.md) show `Spawn` at 0.65 and 0.74 of its time, and `SpawnBatch` at 0.60 and 0.69, at 1,000 and 100,000 entities. Both are now below Bevy's recorded spawn and `spawn_batch` figures at both sizes.
-- Step 7 is the only step left. "Sequence" still says it comes last.
+- Step 7 was the only step left. It landed last, as "Sequence" says.
+
+## Amendments after step 7
+
+- Step 7 says one summary tick per tracked column "per table or block". Each table keeps two per tracked column, the newest added stamp and the newest changed stamp. Partitioned runs share the table's summaries.
+- Every stamp raises its summary. World operations and command playback store their new tick, row copies between tables raise the target to the row's ticks, and system writes raise it only when it is behind. Rows leaving a table never lower it, so a summary may cover rows that are gone. That costs a scan, not a wrong result.
+- The skip lives in `QueryCursor.NextChanged`, which the generator calls only for queries with `Added<T>` or `Changed<T>` terms. A first version checked inside `Next`. Inlined through every generated enumerator, the extra code used up the JIT's inlining budget for system bodies, so `TickWindow.Contains` became a call in the change-filtered row loop, and the 16-archetype named query at 1,000 entities ran 14 to 17% slower. `[MethodImpl(AggressiveInlining)]` on `Contains` did not bring the inline back. This is the same sensitivity to inlining and PGO that steps 2 and 3 recorded.
+- `[EachBlock]` accepts change filters at block granularity, as step 7 proposed. The method receives every row of a block whose table passes, and `AV011` is retired.
+- Generated `[Each]` raises the summary once per block. Raising it through `TrackedColumn.Write` on every row was measurable on the tracked write case. `TrackedColumn.Write`, `StampedRef.Value`, and generated named-query row properties still check it per write. `TrackedColumn.Summary` is public for generated rows, like `Values` and `Changed`.
+- 32-bit ticks were measured with a proxy that runs the generated loops over native columns and leaves out Bevy's clamping pass. They saved nothing at 100,000 rows and 3 to 4% at 1,000,000. Ticks stay 64-bit. Revisit only if tracked iteration over millions of rows becomes a target workload.
+- Runs taken while the machine was in normal use moved by up to 40% in cases whose code did not change, and suggested an 8 to 13% gain for 32-bit ticks that an idle rerun did not reproduce. The step 7 numbers come from an otherwise idle machine. Later gates should be measured the same way.

benchmarks/Archevolt.Benchmarks/MatrixWorkloads.cs

@@ -29,6 +29,10 @@ public static void Mark(scoped ref TrackedPosition position, scoped in Velocity
             position.Y + velocity.Y * delta, position.Z + velocity.Z * delta);
     }
 
+    // Writes the one archetype of sixteen that carries every tag, so a change scan has one table to read.
+    [Each, With<MatrixTag0>, With<MatrixTag1>, With<MatrixTag2>, With<MatrixTag3>]
+    public static void MarkSparse(scoped ref TrackedPosition position, scoped in Velocity velocity) => Mark(ref position, in velocity);
+
     [EcsSystem]
     public static void Changed(scoped ChangedPositions query, scoped ResMut<ScanTotal> total)
     {
@@ -83,6 +87,8 @@ public static partial class MatrixModule;
 
 [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Mark), Order = 0)]
 public sealed partial class TrackedMovement;
+[Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.MarkSparse), Order = 0)]
+public sealed partial class SparseTrackedMovement;
 [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Changed), Order = 0)]
 public sealed partial class ChangedScan;
 [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Optional), Order = 0)]

benchmarks/Archevolt.Benchmarks/PerformanceMeasurement.cs

@@ -71,6 +71,17 @@ public static void Run()
             }
             finally { filters.Cleanup(); }
         }
+        foreach (int count in new[] { 1_000, 100_000 })
+        {
+            var sparse = new SparseChangeBenchmarks { Count = count };
+            sparse.Setup();
+            try
+            {
+                Measure("sparse-write-then-changed", count, sparse.SparseWriteThenChanged, () => sparse.NativeBytes);
+                sparse.RequireScan();
+            }
+            finally { sparse.Cleanup(); }
+        }
         var lifecycle = new LifecycleBenchmarks();
         lifecycle.Setup();
         try

benchmarks/Archevolt.Benchmarks/ScenarioBenchmarks.cs

@@ -63,6 +63,52 @@ public void Setup()
     public void Cleanup() { _writer?.Dispose(); _changed?.Dispose(); _optional?.Dispose(); _world?.Dispose(); }
 }
 
+// Sixteen archetypes of tracked particles. Each update changes one archetype, then scans for changes.
+[MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn]
+public class SparseChangeBenchmarks
+{
+    private World _world = null!;
+    private BoundSchedule _writer = null!;
+    private BoundSchedule _changed = null!;
+    [Params(1_000, 100_000)] public int Count { get; set; }
+    public long NativeBytes => _world.NativeBytes;
+    public long LastScanCount => _world.GetResource<ScanTotal>().Count;
+    public int ExpectedScanCount => Count / 16;
+
+    [GlobalSetup]
+    public void Setup()
+    {
+        _world = MatrixPopulation.Build();
+        _world.InsertResource(new ScanTotal());
+        for (int i = 0; i < Count; i++)
+        {
+            Entity entity = _world.Spawn(new TrackedParticle(new TrackedPosition(i % 97, 0, 0), new Velocity(1, 2, 3)));
+            int mask = i % 16;
+            if ((mask & 1) != 0) _world.Insert(entity, default(MatrixTag0));
+            if ((mask & 2) != 0) _world.Insert(entity, default(MatrixTag1));
+            if ((mask & 4) != 0) _world.Insert(entity, default(MatrixTag2));
+            if ((mask & 8) != 0) _world.Insert(entity, default(MatrixTag3));
+        }
+        _writer = SparseTrackedMovement.Create(_world);
+        _changed = ChangedScan.Create(_world);
+        _changed.Run();
+        SparseWriteThenChanged();
+        RequireScan();
+    }
+    [Benchmark] public void SparseWriteThenChanged() { _writer.Run(); _changed.Run(); }
+    public void RequireScan()
+    {
+        if (LastScanCount != ExpectedScanCount)
+            throw new InvalidOperationException($"Changed scan saw {LastScanCount} rows, not {ExpectedScanCount}.");
+    }
+    [GlobalCleanup]
+    public void Cleanup()
+    {
+        try { if (_world is not null) RequireScan(); }
+        finally { _writer?.Dispose(); _changed?.Dispose(); _world?.Dispose(); }
+    }
+}
+
 [MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn]
 public class LifecycleBenchmarks
 {

benchmarks/Archevolt.Benchmarks/TickWidthBenchmarks.cs

@@ -0,0 +1,117 @@
+using System.Runtime.InteropServices;
+using BenchmarkDotNet.Attributes;
+
+namespace Archevolt.Benchmarks;
+
+// ROADMAP step 7 asks whether 32-bit change ticks would speed up tracked iteration. These cases run the per-row work
+// of the generated tracked write and Changed scan over native columns, once with the runtime's 64-bit ticks and once
+// with 32-bit ticks compared the way Bevy compares them. Only the tick width and comparison differ, so the gap bounds
+// what 32-bit table rows could gain. Not part of the baseline set; run with --filter *TickWidthBenchmarks*.
+[MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn]
+public unsafe class TickWidthBenchmarks
+{
+    // Bevy's MAX_CHANGE_AGE: ticks older than this are clamped by a periodic pass, so wrapping comparisons stay exact.
+    private const uint MaxChangeAge = uint.MaxValue - (2 * (uint.MaxValue / 3) - 1);
+    private const float Delta = 1f / 60;
+    private TrackedPosition* _positions;
+    private Velocity* _velocities;
+    private ulong* _wide;
+    private uint* _narrow;
+    private ulong _now;
+    [Params(100_000, 1_000_000)] public int Count { get; set; }
+    public long LastScanCount { get; private set; }
+
+    [GlobalSetup]
+    public void Setup()
+    {
+        _positions = Allocate<TrackedPosition>();
+        _velocities = Allocate<Velocity>();
+        _wide = Allocate<ulong>();
+        _narrow = Allocate<uint>();
+        for (int i = 0; i < Count; i++)
+        {
+            _positions[i] = new TrackedPosition(i % 97, 0, 0);
+            _velocities[i] = new Velocity(1, 2, 3);
+        }
+        // Start near the 32-bit wrap so the narrow comparisons cross it during the run.
+        _now = uint.MaxValue - 1_000;
+        new Span<ulong>(_wide, Count).Fill(_now);
+        new Span<uint>(_narrow, Count).Fill((uint)_now);
+    }
+
+    private T* Allocate<T>() where T : unmanaged =>
+        (T*)NativeMemory.AlignedAlloc(((nuint)Count * (nuint)sizeof(T) + 63) & ~(nuint)63, 64);
+
+    [Benchmark] public void TrackedWrite64() => Write(_wide, ++_now);
+    [Benchmark] public void TrackedWrite32() => Write(_narrow, (uint)++_now);
+
+    [Benchmark]
+    public void WriteThenChanged64()
+    {
+        ulong last = _now;
+        ulong now = ++_now;
+        Write(_wide, now);
+        ulong scan = ++_now;
+        long count = 0;
+        double sum = 0;
+        for (int row = 0; row < Count; row++)
+        {
+            ulong stamp = _wide[row];
+            if (stamp > last && stamp <= scan)
+            {
+                count++;
+                sum += _positions[row].X;
+            }
+        }
+        Publish(count, sum);
+    }
+
+    [Benchmark]
+    public void WriteThenChanged32()
+    {
+        uint last = (uint)_now;
+        uint now = (uint)++_now;
+        Write(_narrow, now);
+        uint scan = (uint)++_now;
+        uint sinceLast = Math.Min(scan - last, MaxChangeAge);
+        long count = 0;
+        double sum = 0;
+        for (int row = 0; row < Count; row++)
+        {
+            if (sinceLast > Math.Min(scan - _narrow[row], MaxChangeAge))
+            {
+                count++;
+                sum += _positions[row].X;
+            }
+        }
+        Publish(count, sum);
+    }
+
+    private void Write<TTick>(TTick* ticks, TTick now) where TTick : unmanaged
+    {
+        for (int row = 0; row < Count; row++)
+        {
+            ref TrackedPosition position = ref _positions[row];
+            Velocity velocity = _velocities[row];
+            ticks[row] = now;
+            position = new TrackedPosition(position.X + velocity.X * Delta,
+                position.Y + velocity.Y * Delta, position.Z + velocity.Z * Delta);
+        }
+    }
+
+    private void Publish(long count, double sum)
+    {
+        if (count != Count || double.IsNaN(sum))
+            throw new InvalidOperationException($"Changed scan saw {count} rows, not {Count}.");
+        LastScanCount = count;
+    }
+
+    [GlobalCleanup]
+    public void Cleanup()
+    {
+        NativeMemory.AlignedFree(_positions);
+        NativeMemory.AlignedFree(_velocities);
+        NativeMemory.AlignedFree(_wide);
+        NativeMemory.AlignedFree(_narrow);
+    }
+}