diff --git a/ROADMAP.md b/ROADMAP.md index 8d2699d..cf25c64 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,6 @@ # Archevolt performance plan -Status: steps 0, 1, 2, 3, 4, 5, 6, and 8 done. Step 7 remains. Step 0's baselines are in `benchmarks/baseline.md`, recorded on .NET 10. Step 3's before and after numbers, on .NET 11, are in `benchmarks/step3-table-storage.md`. Later steps record fresh before and after comparisons on the same SDK. Step 3 met its gate except for named queries, whose regression came from enumerator codegen. Step 2 rewrote that enumerator and met its gate. Its numbers are in `benchmarks/step2-named-query-enumerator.md`. Step 4 replaced directory references with integers and met the median/range gate after an unchanged-code drift control. Its measurements and the remaining variance in large random reads are in `benchmarks/step4-integer-entity-directory.md`. Step 5 made reservation lock-free, added batch spawn, and met its gate. Its numbers are in `benchmarks/step5-lock-free-reservation.md`. Step 6 cached column maps, interned archetypes by bitmask, and met its gate. Its numbers are in `benchmarks/step6-cached-column-maps.md`. Step 8 made both immediate spawns write bundles in place and met its gate. Its numbers are in `benchmarks/step8-in-place-spawn.md`. +Status: every step is done. Step 0's baselines are in `benchmarks/baseline.md`, recorded on .NET 10. Step 3's before and after numbers, on .NET 11, are in `benchmarks/step3-table-storage.md`. Later steps record fresh before and after comparisons on the same SDK. Step 3 met its gate except for named queries, whose regression came from enumerator codegen. Step 2 rewrote that enumerator and met its gate. Its numbers are in `benchmarks/step2-named-query-enumerator.md`. Step 4 replaced directory references with integers and met the median/range gate after an unchanged-code drift control. Its measurements and the remaining variance in large random reads are in `benchmarks/step4-integer-entity-directory.md`. Step 5 made reservation lock-free, added batch spawn, and met its gate. Its numbers are in `benchmarks/step5-lock-free-reservation.md`. Step 6 cached column maps, interned archetypes by bitmask, and met its gate. Its numbers are in `benchmarks/step6-cached-column-maps.md`. Step 8 made both immediate spawns write bundles in place and met its gate. Its numbers are in `benchmarks/step8-in-place-spawn.md`. Step 7 added per-column change summaries, met its gate, and kept 64-bit ticks. Its numbers are in `benchmarks/step7-block-change-summaries.md`. The v1 implementation plan is complete. Every stage passed its acceptance gate, and the evidence is in `.audit/acceptance.md`. The [v1 plan](https://github.com/Stephanvs/archevolt/blob/ebacf07059fefbf9b182e40a290812be584dd9ee/PLAN.md) remains in history as the record of the original design, its alternatives, and its gates. @@ -243,4 +243,14 @@ Follow-up, not yet a numbered step: [step3-partition-granularity.md](benchmarks/ - Step 8 says the `BundleWriter` points at the row's destination columns. The writer keeps its staging form for `Commands` and `InsertBundle` and gains a table form. `BundleBinding.Columns` holds each member's column, so the table form writes without a lookup. The amendment after step 6 said the binding did not need its columns. It needs them now, because `WriteBundle` no longer runs for immediate spawns. - The new row sits in the table, but not yet in the directory, while user code in the bundle's `Write` runs. The world counts as running for that window, so a `Write` that reaches the same world through other state cannot move or truncate the row. Step 8's text did not anticipate this. - [The step 8 measurements](benchmarks/step8-in-place-spawn.md) show `Spawn` at 0.65 and 0.74 of its time, and `SpawnBatch` at 0.60 and 0.69, at 1,000 and 100,000 entities. Both are now below Bevy's recorded spawn and `spawn_batch` figures at both sizes. -- Step 7 is the only step left. "Sequence" still says it comes last. +- Step 7 was the only step left. It landed last, as "Sequence" says. + +## Amendments after step 7 + +- Step 7 says one summary tick per tracked column "per table or block". Each table keeps two per tracked column, the newest added stamp and the newest changed stamp. Partitioned runs share the table's summaries. +- Every stamp raises its summary. World operations and command playback store their new tick, row copies between tables raise the target to the row's ticks, and system writes raise it only when it is behind. Rows leaving a table never lower it, so a summary may cover rows that are gone. That costs a scan, not a wrong result. +- The skip lives in `QueryCursor.NextChanged`, which the generator calls only for queries with `Added` or `Changed` terms. A first version checked inside `Next`. Inlined through every generated enumerator, the extra code used up the JIT's inlining budget for system bodies, so `TickWindow.Contains` became a call in the change-filtered row loop, and the 16-archetype named query at 1,000 entities ran 14 to 17% slower. `[MethodImpl(AggressiveInlining)]` on `Contains` did not bring the inline back. This is the same sensitivity to inlining and PGO that steps 2 and 3 recorded. +- `[EachBlock]` accepts change filters at block granularity, as step 7 proposed. The method receives every row of a block whose table passes, and `AV011` is retired. +- Generated `[Each]` raises the summary once per block. Raising it through `TrackedColumn.Write` on every row was measurable on the tracked write case. `TrackedColumn.Write`, `StampedRef.Value`, and generated named-query row properties still check it per write. `TrackedColumn.Summary` is public for generated rows, like `Values` and `Changed`. +- 32-bit ticks were measured with a proxy that runs the generated loops over native columns and leaves out Bevy's clamping pass. They saved nothing at 100,000 rows and 3 to 4% at 1,000,000. Ticks stay 64-bit. Revisit only if tracked iteration over millions of rows becomes a target workload. +- Runs taken while the machine was in normal use moved by up to 40% in cases whose code did not change, and suggested an 8 to 13% gain for 32-bit ticks that an idle rerun did not reproduce. The step 7 numbers come from an otherwise idle machine. Later gates should be measured the same way. diff --git a/benchmarks/Archevolt.Benchmarks/MatrixWorkloads.cs b/benchmarks/Archevolt.Benchmarks/MatrixWorkloads.cs index e82541c..96c1ad9 100644 --- a/benchmarks/Archevolt.Benchmarks/MatrixWorkloads.cs +++ b/benchmarks/Archevolt.Benchmarks/MatrixWorkloads.cs @@ -29,6 +29,10 @@ public static void Mark(scoped ref TrackedPosition position, scoped in Velocity position.Y + velocity.Y * delta, position.Z + velocity.Z * delta); } + // Writes the one archetype of sixteen that carries every tag, so a change scan has one table to read. + [Each, With, With, With, With] + public static void MarkSparse(scoped ref TrackedPosition position, scoped in Velocity velocity) => Mark(ref position, in velocity); + [EcsSystem] public static void Changed(scoped ChangedPositions query, scoped ResMut total) { @@ -83,6 +87,8 @@ public static partial class MatrixModule; [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Mark), Order = 0)] public sealed partial class TrackedMovement; +[Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.MarkSparse), Order = 0)] +public sealed partial class SparseTrackedMovement; [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Changed), Order = 0)] public sealed partial class ChangedScan; [Schedule, Run(typeof(MatrixSystems), nameof(MatrixSystems.Optional), Order = 0)] diff --git a/benchmarks/Archevolt.Benchmarks/PerformanceMeasurement.cs b/benchmarks/Archevolt.Benchmarks/PerformanceMeasurement.cs index 3553dd0..764ee7f 100644 --- a/benchmarks/Archevolt.Benchmarks/PerformanceMeasurement.cs +++ b/benchmarks/Archevolt.Benchmarks/PerformanceMeasurement.cs @@ -71,6 +71,17 @@ public static void Run() } finally { filters.Cleanup(); } } + foreach (int count in new[] { 1_000, 100_000 }) + { + var sparse = new SparseChangeBenchmarks { Count = count }; + sparse.Setup(); + try + { + Measure("sparse-write-then-changed", count, sparse.SparseWriteThenChanged, () => sparse.NativeBytes); + sparse.RequireScan(); + } + finally { sparse.Cleanup(); } + } var lifecycle = new LifecycleBenchmarks(); lifecycle.Setup(); try diff --git a/benchmarks/Archevolt.Benchmarks/ScenarioBenchmarks.cs b/benchmarks/Archevolt.Benchmarks/ScenarioBenchmarks.cs index ed2cb01..89d1505 100644 --- a/benchmarks/Archevolt.Benchmarks/ScenarioBenchmarks.cs +++ b/benchmarks/Archevolt.Benchmarks/ScenarioBenchmarks.cs @@ -63,6 +63,52 @@ public void Setup() public void Cleanup() { _writer?.Dispose(); _changed?.Dispose(); _optional?.Dispose(); _world?.Dispose(); } } +// Sixteen archetypes of tracked particles. Each update changes one archetype, then scans for changes. +[MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn] +public class SparseChangeBenchmarks +{ + private World _world = null!; + private BoundSchedule _writer = null!; + private BoundSchedule _changed = null!; + [Params(1_000, 100_000)] public int Count { get; set; } + public long NativeBytes => _world.NativeBytes; + public long LastScanCount => _world.GetResource().Count; + public int ExpectedScanCount => Count / 16; + + [GlobalSetup] + public void Setup() + { + _world = MatrixPopulation.Build(); + _world.InsertResource(new ScanTotal()); + for (int i = 0; i < Count; i++) + { + Entity entity = _world.Spawn(new TrackedParticle(new TrackedPosition(i % 97, 0, 0), new Velocity(1, 2, 3))); + int mask = i % 16; + if ((mask & 1) != 0) _world.Insert(entity, default(MatrixTag0)); + if ((mask & 2) != 0) _world.Insert(entity, default(MatrixTag1)); + if ((mask & 4) != 0) _world.Insert(entity, default(MatrixTag2)); + if ((mask & 8) != 0) _world.Insert(entity, default(MatrixTag3)); + } + _writer = SparseTrackedMovement.Create(_world); + _changed = ChangedScan.Create(_world); + _changed.Run(); + SparseWriteThenChanged(); + RequireScan(); + } + [Benchmark] public void SparseWriteThenChanged() { _writer.Run(); _changed.Run(); } + public void RequireScan() + { + if (LastScanCount != ExpectedScanCount) + throw new InvalidOperationException($"Changed scan saw {LastScanCount} rows, not {ExpectedScanCount}."); + } + [GlobalCleanup] + public void Cleanup() + { + try { if (_world is not null) RequireScan(); } + finally { _writer?.Dispose(); _changed?.Dispose(); _world?.Dispose(); } + } +} + [MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn] public class LifecycleBenchmarks { diff --git a/benchmarks/Archevolt.Benchmarks/TickWidthBenchmarks.cs b/benchmarks/Archevolt.Benchmarks/TickWidthBenchmarks.cs new file mode 100644 index 0000000..acfed50 --- /dev/null +++ b/benchmarks/Archevolt.Benchmarks/TickWidthBenchmarks.cs @@ -0,0 +1,117 @@ +using System.Runtime.InteropServices; +using BenchmarkDotNet.Attributes; + +namespace Archevolt.Benchmarks; + +// ROADMAP step 7 asks whether 32-bit change ticks would speed up tracked iteration. These cases run the per-row work +// of the generated tracked write and Changed scan over native columns, once with the runtime's 64-bit ticks and once +// with 32-bit ticks compared the way Bevy compares them. Only the tick width and comparison differ, so the gap bounds +// what 32-bit table rows could gain. Not part of the baseline set; run with --filter *TickWidthBenchmarks*. +[MemoryDiagnoser, MinColumn, MaxColumn, MedianColumn] +public unsafe class TickWidthBenchmarks +{ + // Bevy's MAX_CHANGE_AGE: ticks older than this are clamped by a periodic pass, so wrapping comparisons stay exact. + private const uint MaxChangeAge = uint.MaxValue - (2 * (uint.MaxValue / 3) - 1); + private const float Delta = 1f / 60; + private TrackedPosition* _positions; + private Velocity* _velocities; + private ulong* _wide; + private uint* _narrow; + private ulong _now; + [Params(100_000, 1_000_000)] public int Count { get; set; } + public long LastScanCount { get; private set; } + + [GlobalSetup] + public void Setup() + { + _positions = Allocate(); + _velocities = Allocate(); + _wide = Allocate(); + _narrow = Allocate(); + for (int i = 0; i < Count; i++) + { + _positions[i] = new TrackedPosition(i % 97, 0, 0); + _velocities[i] = new Velocity(1, 2, 3); + } + // Start near the 32-bit wrap so the narrow comparisons cross it during the run. + _now = uint.MaxValue - 1_000; + new Span(_wide, Count).Fill(_now); + new Span(_narrow, Count).Fill((uint)_now); + } + + private T* Allocate() where T : unmanaged => + (T*)NativeMemory.AlignedAlloc(((nuint)Count * (nuint)sizeof(T) + 63) & ~(nuint)63, 64); + + [Benchmark] public void TrackedWrite64() => Write(_wide, ++_now); + [Benchmark] public void TrackedWrite32() => Write(_narrow, (uint)++_now); + + [Benchmark] + public void WriteThenChanged64() + { + ulong last = _now; + ulong now = ++_now; + Write(_wide, now); + ulong scan = ++_now; + long count = 0; + double sum = 0; + for (int row = 0; row < Count; row++) + { + ulong stamp = _wide[row]; + if (stamp > last && stamp <= scan) + { + count++; + sum += _positions[row].X; + } + } + Publish(count, sum); + } + + [Benchmark] + public void WriteThenChanged32() + { + uint last = (uint)_now; + uint now = (uint)++_now; + Write(_narrow, now); + uint scan = (uint)++_now; + uint sinceLast = Math.Min(scan - last, MaxChangeAge); + long count = 0; + double sum = 0; + for (int row = 0; row < Count; row++) + { + if (sinceLast > Math.Min(scan - _narrow[row], MaxChangeAge)) + { + count++; + sum += _positions[row].X; + } + } + Publish(count, sum); + } + + private void Write(TTick* ticks, TTick now) where TTick : unmanaged + { + for (int row = 0; row < Count; row++) + { + ref TrackedPosition position = ref _positions[row]; + Velocity velocity = _velocities[row]; + ticks[row] = now; + position = new TrackedPosition(position.X + velocity.X * Delta, + position.Y + velocity.Y * Delta, position.Z + velocity.Z * Delta); + } + } + + private void Publish(long count, double sum) + { + if (count != Count || double.IsNaN(sum)) + throw new InvalidOperationException($"Changed scan saw {count} rows, not {Count}."); + LastScanCount = count; + } + + [GlobalCleanup] + public void Cleanup() + { + NativeMemory.AlignedFree(_positions); + NativeMemory.AlignedFree(_velocities); + NativeMemory.AlignedFree(_wide); + NativeMemory.AlignedFree(_narrow); + } +} diff --git a/benchmarks/step7-block-change-summaries.md b/benchmarks/step7-block-change-summaries.md new file mode 100644 index 0000000..4551b44 --- /dev/null +++ b/benchmarks/step7-block-change-summaries.md @@ -0,0 +1,116 @@ +# Step 7: block change summaries and 32-bit ticks + +This file records [ROADMAP.md](../ROADMAP.md) step 7. Change-filtered queries now skip whole tables in which nothing was added or changed since the system last ran. The 32-bit tick experiment did not justify a change, and ticks stay 64-bit. + +## Change + +- **One summary per tracked column per table.** Each table keeps the newest added stamp and the newest changed stamp of every tracked column. Partitioned runs split a table into row ranges that share its summaries. +- **Every stamp raises its summary.** World operations and command playback store their new tick, which is the newest in the world. A row copied into another table raises the target's summaries to the row's own ticks. System writes raise a summary only when it is behind, so partitioned workers do not trade its cache line. Rows leaving a table never lower a summary, so it may cover rows that are gone. That costs a scan and does not change any result. +- **`QueryCursor.NextChanged`.** It wraps `Next` and skips a block when a filtered column's summary is at or before the system's last run. The generator calls it only for queries with `Added` or `Changed` terms, in `[Each]`, `[EachBlock]`, and named-query enumerators. `Next` is unchanged. +- **`[EachBlock]` accepts change filters.** The method receives every block whose table passes the summary check, including rows in it that did not change. `AV011` is retired. +- **Generated `[Each]` raises the summary once per block.** It stamps each row through the column's spans, then calls the method. `TrackedColumn.Write`, `StampedRef.Value`, and generated named-query row properties check the summary on every write. `TrackedColumn.Summary` is public for generated rows, as `Values` and `Changed` already were. + +## Conditions + +| Setting | Value | +| --- | --- | +| CPU | AMD Ryzen 9 5950X 16-Core Processor | +| OS | Windows 11 Pro 10.0.26300 | +| .NET | SDK 11.0.100-rc.3.26522.148, RyuJIT x86-64-v3, Release | +| BenchmarkDotNet | 0.16.0-preview.2, default job, one launch per process run | +| Power plan | BenchmarkDotNet selects High performance and restores the previous plan | +| Before | `0a4c07f`: `bc62d14`, the tip of step 8, plus this step's benchmark changes only | +| After | `d0da09c`: a working-tree snapshot of this change on `bc62d14` | + +Both versions ran in the same dedicated worktree. Each process run used this command: + +```powershell +./scripts/baseline.ps1 -Runs 1 -Classes FilterBenchmarks,SparseChangeBenchmarks,TraversalMatrixBenchmarks,ReferenceComparisonBenchmarks +``` + +The six process runs alternated in the order before, after, after, before, before, after, between 03:46 and 05:10 UTC on 31 October 2026, with the machine otherwise unused. Total CPU use, sampled over 10 seconds before each run, was 8 to 12%. Each timing is the median of three per-process means in microseconds, with their range. `scripts/compare-baselines.ps1` classifies an after median outside the before range as faster or slower. Builds and tests did not run alongside the measurements. Bevy was not rerun. + +Runs on the previous evening, with the machine in normal use, took more than twice as long per process and moved by up to 40% in cases whose code did not change. They are not reported. + +## Measurements + +| Case | Before µs [min, max] | After µs [min, max] | After/before | Verdict | +| --- | ---: | ---: | ---: | --- | +| `FilterBenchmarks.ChangedNone(Count=1000)` | 0.610 [0.609, 0.627] | 0.163 [0.162, 0.179] | 0.27 | faster | +| `FilterBenchmarks.ChangedNone(Count=100000)` | 47.02 [46.90, 48.11] | 0.162 [0.156, 0.166] | 0.00 | faster | +| `FilterBenchmarks.OptionalHalfPresent(Count=1000)` | 1.262 [1.259, 1.284] | 1.258 [1.258, 1.260] | 1.00 | faster | +| `FilterBenchmarks.OptionalHalfPresent(Count=100000)` | 134.4 [125.4, 136.0] | 135.8 [135.5, 135.9] | 1.01 | within | +| `FilterBenchmarks.TrackedWrite(Count=1000)` | 1.153 [1.152, 1.153] | 1.164 [1.157, 1.189] | 1.01 | slower | +| `FilterBenchmarks.TrackedWrite(Count=100000)` | 96.48 [95.85, 96.57] | 96.61 [96.21, 97.33] | 1.00 | slower | +| `FilterBenchmarks.WriteThenChangedAll(Count=1000)` | 2.129 [2.041, 2.311] | 2.120 [2.109, 2.126] | 1.00 | within | +| `FilterBenchmarks.WriteThenChangedAll(Count=100000)` | 169.9 [169.3, 177.6] | 169.2 [168.8, 170.1] | 1.00 | faster | +| `ReferenceComparisonBenchmarks.ArchInline(Count=100000)` | 96.17 [95.06, 109.6] | 105.0 [96.49, 106.9] | 1.09 | within | +| `ReferenceComparisonBenchmarks.GeneratedBlock(Count=100000)` | 84.36 [84.15, 85.34] | 84.52 [84.37, 84.71] | 1.00 | within | +| `ReferenceComparisonBenchmarks.GeneratedBlockVector(Count=100000)` | 20.73 [20.62, 20.94] | 20.68 [20.56, 20.92] | 1.00 | within | +| `ReferenceComparisonBenchmarks.GeneratedEach(Count=100000)` | 97.97 [96.95, 98.00] | 95.96 [95.59, 96.17] | 0.98 | faster | +| `ReferenceComparisonBenchmarks.GeneratedQuery(Count=100000)` | 83.85 [83.59, 84.47] | 83.99 [83.77, 84.58] | 1.00 | within | +| `ReferenceComparisonBenchmarks.Handwritten(Count=100000)` | 84.41 [84.28, 84.83] | 83.89 [83.63, 84.15] | 0.99 | faster | +| `SparseChangeBenchmarks.SparseWriteThenChanged(Count=1000)` | 1.034 [1.026, 1.039] | 0.600 [0.575, 0.638] | 0.58 | faster | +| `SparseChangeBenchmarks.SparseWriteThenChanged(Count=100000)` | 79.57 [58.24, 80.55] | 12.79 [11.50, 12.90] | 0.16 | faster | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000, Archetypes=1, Wide=False)` | 1.082 [1.082, 1.095] | 1.092 [1.080, 1.096] | 1.01 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000, Archetypes=1, Wide=True)` | 1.085 [1.074, 1.116] | 1.098 [1.075, 1.116] | 1.01 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000, Archetypes=16, Wide=False)` | 1.317 [1.298, 1.358] | 1.309 [1.302, 1.334] | 0.99 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000, Archetypes=16, Wide=True)` | 1.341 [1.335, 1.342] | 1.330 [1.326, 1.337] | 0.99 | faster | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=100000, Archetypes=1, Wide=False)` | 96.38 [96.37, 98.53] | 96.19 [96.06, 96.20] | 1.00 | faster | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=100000, Archetypes=1, Wide=True)` | 98.89 [98.40, 100.5] | 97.08 [95.83, 102.3] | 0.98 | faster | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=100000, Archetypes=16, Wide=False)` | 97.64 [96.97, 98.09] | 97.16 [97.10, 97.29] | 1.00 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=100000, Archetypes=16, Wide=True)` | 96.63 [96.59, 97.21] | 97.11 [96.61, 97.55] | 1.00 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000000, Archetypes=1, Wide=False)` | 1,008 [977.9, 1,015] | 997.7 [992.5, 1,053] | 0.99 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000000, Archetypes=1, Wide=True)` | 996.5 [979.8, 1,037] | 983.2 [979.0, 1,018] | 0.99 | within | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000000, Archetypes=16, Wide=False)` | 997.9 [997.8, 1,000] | 996.3 [986.9, 997.8] | 1.00 | faster | +| `TraversalMatrixBenchmarks.GeneratedEach(Count=1000000, Archetypes=16, Wide=True)` | 993.0 [989.6, 994.9] | 989.5 [989.2, 990.0] | 1.00 | faster | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000, Archetypes=1, Wide=False)` | 0.998 [0.997, 1.003] | 1.012 [0.995, 1.013] | 1.01 | slower | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000, Archetypes=1, Wide=True)` | 1.009 [0.968, 1.011] | 0.997 [0.968, 1.165] | 0.99 | within | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000, Archetypes=16, Wide=False)` | 1.234 [1.215, 1.250] | 1.232 [1.221, 1.261] | 1.00 | within | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000, Archetypes=16, Wide=True)` | 1.266 [1.253, 1.276] | 1.252 [1.247, 1.254] | 0.99 | faster | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=100000, Archetypes=1, Wide=False)` | 100.1 [84.06, 104.4] | 84.16 [83.71, 100.00] | 0.84 | within | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=100000, Archetypes=1, Wide=True)` | 84.05 [83.66, 84.06] | 84.57 [83.90, 89.61] | 1.01 | slower | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=100000, Archetypes=16, Wide=False)` | 100.8 [85.66, 101.6] | 101.4 [100.5, 101.6] | 1.01 | within | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=100000, Archetypes=16, Wide=True)` | 86.23 [85.82, 101.4] | 86.06 [85.97, 101.1] | 1.00 | within | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000000, Archetypes=1, Wide=False)` | 867.0 [862.0, 878.8] | 860.5 [852.3, 903.4] | 0.99 | faster | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000000, Archetypes=1, Wide=True)` | 864.5 [844.8, 866.5] | 885.3 [876.1, 890.2] | 1.02 | slower | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000000, Archetypes=16, Wide=False)` | 875.1 [873.1, 885.5] | 889.8 [880.8, 890.3] | 1.02 | slower | +| `TraversalMatrixBenchmarks.GeneratedQuery(Count=1000000, Archetypes=16, Wide=True)` | 880.9 [879.3, 894.0] | 888.6 [873.1, 890.5] | 1.01 | within | + +`SparseChangeBenchmarks` spreads tracked particles over 16 archetypes. Each iteration writes the one archetype that carries all four tags, then runs the `Changed` scan. Before this change the scan read every row's tick. Now it reads one table of 16. At 100,000 entities that takes 0.16 of the time, and what remains is the write and the scan of 6,250 rows. At 1,000 entities it takes 0.58, because per-table and dispatch costs are a larger share there. `ChangedNone`, which scans with nothing changed, no longer reads any row: 0.16 µs at both sizes, against 47 µs before at 100,000 entities. The sparse case's before range is wide, 58.2 to 80.6 µs, because one process ran fast. The after range lies well below it either way. + +Untracked and dense-change iteration holds. Every `TraversalMatrixBenchmarks`, `ReferenceComparisonBenchmarks`, `TrackedWrite`, `WriteThenChangedAll`, and `OptionalHalfPresent` case is within 2% of its before median, and each of their "faster" and "slower" verdicts sits within that 2%. Unfiltered queries run the same `Next` as before, so those verdicts are run-to-run variance. `ArchInline` is a third-party comparison that runs no Archevolt code. Its 1.09 lies within its before range. + +Managed allocation per operation is zero in every measured case, before and after. + +## Two regressions found and fixed + +The first version of this change checked the summaries inside `QueryCursor.Next`, and generated `[Each]` raised the summary through `TrackedColumn.Write` on every row. Interleaved runs on the idle machine showed two regressions: + +- `GeneratedQuery` over 16 archetypes at 1,000 entities was 14 to 17% slower. Every run after was slower than the slowest run before. +- `WriteThenChangedAll` at 1,000 entities was bimodal, about 2.1 µs in some processes and 3.0 µs in others, against a steady 2.1 µs before. + +JIT disassembly of the generated `Changed` system body explained both. `Next` is inlined, through the enumerator's `NextBlock`, into every system body that runs a query. Its extra code used up the JIT's inlining budget for the body, so `TickWindow.Contains` stopped being inlined into the row loop. The loop then called it once per row and kept the enumerator's fields on the stack. Fast and slow processes produced the same instructions; only the loop's alignment differed, 32-byte aligned in the fast processes and not in the slow ones. + +Moving the check into `NextChanged`, which only change-filtered generated code calls and which the JIT does not inline, restored the old row loop. `[MethodImpl(AggressiveInlining)]` on `Contains` did not: the call stayed. Raising the summary once per block in generated `[Each]` took the per-row summary check out of the tracked write loop. Its Tier1 row loop has one tick store and no bounds checks, and the summary check sits before the loop. + +## 32-bit ticks + +`TickWidthBenchmarks` runs the per-row work of the generated tracked write and `Changed` scan over native columns. One version uses 64-bit ticks. The other uses 32-bit ticks compared the way Bevy compares them, by wrapping subtraction clamped to `MAX_CHANGE_AGE`, and starts close to the 32-bit wrap. Only the tick width and the comparison differ. The proxy leaves out the periodic clamping pass that real 32-bit ticks need, so it overstates their gain. + +Three process runs on the idle machine, at `1e73e43`, between 03:00 and 03:08 UTC on 31 October 2026: + +| Case | 64-bit µs [min, max] | 32-bit µs [min, max] | 32/64 | +| --- | ---: | ---: | ---: | +| Tracked write, 100,000 rows | 133.0 [132.7, 134.9] | 133.7 [133.6, 134.0] | 1.01 | +| Write then `Changed` scan, 100,000 rows | 211.7 [207.7, 212.3] | 210.4 [210.4, 211.5] | 0.99 | +| Tracked write, 1,000,000 rows | 1,413 [1,411, 1,414] | 1,371 [1,350, 1,402] | 0.97 | +| Write then `Changed` scan, 1,000,000 rows | 2,261 [2,223, 2,266] | 2,161 [2,157, 2,178] | 0.96 | + +At 100,000 rows the two widths are indistinguishable. At 1,000,000 rows the narrower ticks save at most 3 to 4%. Adopting them would add a periodic pass over every tracked column and change the public `ulong` tick type, which this gain does not justify, so ticks stay 64-bit. A run the previous evening, with the machine in use, showed 8 to 13% at 1,000,000 rows. The idle rerun did not reproduce it. + +Reproduce with: + +```powershell +./scripts/baseline.ps1 -Runs 3 -Classes TickWidthBenchmarks +``` diff --git a/docs/guides/change-tracking.mdx b/docs/guides/change-tracking.mdx index c10144e..2ce1d08 100644 --- a/docs/guides/change-tracking.mdx +++ b/docs/guides/change-tracking.mdx @@ -22,6 +22,12 @@ A system observes stamps satisfying `lastSuccessfulRun < stamp <= thisRun`. A sk A generated writable row property stamps before returning its ref. `[Each]` stamps tracked writable refs before the method call. `[EachBlock]` stamps every row of a tracked writable span before the method call. `ResMut.Value` follows the same rule. +## Skipping unchanged archetypes + +Each archetype keeps the newest added and changed stamp of every tracked component. An `Added` or `Changed` filter skips an archetype whose newest stamp predates the system's last successful run, without reading its rows. A scan in a world where little changed therefore costs time in proportion to the archetypes that did change, not to every entity. + +Removing or moving an entity does not lower an archetype's newest stamp. The filter then reads that archetype's rows and finds no match, so the result is still exact. `[EachBlock]` is the exception: it applies the filter [per block](/guides/systems#eachblock), not per row. + :::note Taking a writable ref counts as a change even when the method leaves the value unchanged. Use `Read` or `scoped in` when you do not intend to write. ::: diff --git a/docs/guides/systems.mdx b/docs/guides/systems.mdx index e07c0ac..75dd624 100644 --- a/docs/guides/systems.mdx +++ b/docs/guides/systems.mdx @@ -49,7 +49,8 @@ public static partial class MotionKernel - Use `scoped Span` for writes and `scoped ReadOnlySpan` for reads. Every span in one call has the same length. - An optional `scoped EntityColumn` parameter supplies identity. `entities[i]` returns the `Entity` for row `i`. -- `With` and `Without` apply to the method. `Added` and `Changed` are rejected ([`AV011`](/reference/diagnostics)), because the method receives whole blocks. Use `[Each]` for change-filtered traversal. +- `With` and `Without` apply to the method. +- `Added` and `Changed` select whole blocks, not rows. The method receives a block when its archetype has an addition or change since the system last ran, and every row of that block comes with it, including rows that did not change. Use `[Each]` or a named query when the body must see only the changed rows. - A writable span of a [tracked component](/guides/change-tracking) stamps every row in the block as changed. ## Partitioning diff --git a/docs/performance.mdx b/docs/performance.mdx index 89e868e..227e217 100644 --- a/docs/performance.mdx +++ b/docs/performance.mdx @@ -30,6 +30,7 @@ Measured on an AMD Ryzen 9 5950X, Windows 11, .NET 11.0.0-rc.2, Release, Benchma - Structural churn allocates. - Small workloads can cost more to dispatch in parallel than to run serially. - Writable access stamps a change on [tracked components](/guides/change-tracking) even when the value is equal. +- `Added` and `Changed` filters [skip archetypes](/guides/change-tracking#skipping-unchanged-archetypes) with no stamp since the system last ran, but read every row of an archetype that has one. Changes spread thinly across many archetypes cost more to scan than the same changes kept in a few. ## Reproduce diff --git a/docs/reference/diagnostics.mdx b/docs/reference/diagnostics.mdx index ccccc09..9de04b4 100644 --- a/docs/reference/diagnostics.mdx +++ b/docs/reference/diagnostics.mdx @@ -19,7 +19,6 @@ All generator diagnostics are errors except `AV008`, which is a warning. Runtime | `AV008` | Conflicting systems have no explicit dependency. The planner supplies a conservative order. | | `AV009` | Missing or incompatible module or system metadata, registration, or export. | | `AV010` | Unsafe partition request or condition capability. | -| `AV011` | Change filter on an `[EachBlock]` method. | ## Common fixes @@ -29,4 +28,3 @@ All generator diagnostics are errors except `AV008`, which is a warning. Runtime - **AV006.** Add `[TrackChanges]` to the component or resource. See [Change tracking](/guides/change-tracking). - **AV007.** Check `Before`, `After`, `Order`, and instance keys on your [`Run` declarations](/guides/schedules). - **AV008.** Add an explicit `Before` or `After` dependency between the two systems. -- **AV011.** Move the change filter to an `[Each]` method. See [Systems and traversal](/guides/systems). diff --git a/scripts/baseline.ps1 b/scripts/baseline.ps1 index eae320a..6286f5c 100644 --- a/scripts/baseline.ps1 +++ b/scripts/baseline.ps1 @@ -26,7 +26,7 @@ param( [switch] $IncludeBevy, [ValidateNotNullOrEmpty()] [string[]] $Classes = @('EntityPopulationBenchmarks', 'EntityStorageBenchmarks', 'TraversalMatrixBenchmarks', - 'FilterBenchmarks', 'ReferenceComparisonBenchmarks'), + 'FilterBenchmarks', 'SparseChangeBenchmarks', 'ReferenceComparisonBenchmarks'), # Requires -IncludeBevy. Empty runs all Criterion bench targets in bevy-reference. [string[]] $Benches = @() ) diff --git a/src/Archevolt.Generators/Models.cs b/src/Archevolt.Generators/Models.cs index 1a3f987..269c693 100644 --- a/src/Archevolt.Generators/Models.cs +++ b/src/Archevolt.Generators/Models.cs @@ -32,7 +32,7 @@ internal sealed record ParameterModel(string Type, string Name, string Kind, str internal static class Diagnostics { - static readonly Dictionary Descriptors = Enumerable.Range(1, 11).ToDictionary( + static readonly Dictionary Descriptors = Enumerable.Range(1, 10).ToDictionary( static index => $"AV{index:000}", static index => new DiagnosticDescriptor($"AV{index:000}", "Invalid Archevolt declaration", "{0}", "Archevolt", index == 8 ? DiagnosticSeverity.Warning : DiagnosticSeverity.Error, isEnabledByDefault: true)); diff --git a/src/Archevolt.Generators/QueryGeneration.cs b/src/Archevolt.Generators/QueryGeneration.cs index 217b611..5bb0a35 100644 --- a/src/Archevolt.Generators/QueryGeneration.cs +++ b/src/Archevolt.Generators/QueryGeneration.cs @@ -58,7 +58,7 @@ static string ColumnType(TermModel term) => term.Mode == "Write" : $"global::System.ReadOnlySpan<{term.Component.Type}>"; static string ColumnName(TermModel term) => "_column" + term.Index; static string Element(TermModel term, string row) => term.Mode == "Write" && term.Component.Tracked - ? $"{ColumnName(term)}.Write({row})" : $"{ColumnName(term)}[{row}]"; + ? $"{ColumnName(term)}.Values[{row}]" : $"{ColumnName(term)}[{row}]"; static string BlockColumn(TermModel term) => term.Mode == "Read" ? $"block.Read<{term.Component.Type}>({term.Index})" : term.Component.Tracked ? $"block.Tracked<{term.Component.Type}>({term.Index}).WriteAll()" : $"block.Write<{term.Component.Type}>({term.Index})"; @@ -103,6 +103,8 @@ static GeneratedUnit GenerateQuery(INamedTypeSymbol symbol, CancellationToken ca const string Unsafe = "global::System.Runtime.CompilerServices.Unsafe"; const string BaseOf = "global::System.Runtime.InteropServices.MemoryMarshal.GetReference"; static bool Stamped(TermModel term) => term.Mode == "Write" && term.Component.Tracked; + // Change-filtered queries skip blocks whose table has no stamp since the last run. Only they pay for that check. + static string NextBlock(IEnumerable terms) => terms.Any(static term => term.Mode is "Added" or "Changed") ? "NextChanged" : "Next"; // The enumerator holds a ref to each column's current element and the rows left in the block. MoveNext advances // every ref, then tests the row count, so the per-row path is one block that ends in the loop's branch. With a @@ -127,7 +129,11 @@ void Ref(string type, string field, string stride) // An absent optional column binds a null ref and a stride of zero, so it stays null without a per-row branch. if (term.Mode == "Optional") text.Append("private int _stride").Append(term.Index).AppendLine(";"); Ref(term.Component.Type, ColumnName(term), term.Mode == "Optional" ? "_stride" + term.Index : "1"); - if (Stamped(term)) Ref("ulong", "_changed" + term.Index, "1"); + if (Stamped(term)) + { + Ref("ulong", "_changed" + term.Index, "1"); + text.Append("private ref ulong _summary").Append(term.Index).AppendLine(";"); + } } for (int i = 0; i < filters.Count; i++) Ref("ulong", "_ticks" + i, "1"); text.Append("internal Enumerator(global::Archevolt.QueryCursor cursor) { this = default; _cursor = cursor;").Append(window ? " _window = cursor.Window;" : "").AppendLine(" }"); @@ -140,7 +146,7 @@ void Ref(string type, string field, string stride) // QueryCursor.Next never returns an empty block, and every column is sliced to block.Count rows. The refs step // one element past a column's end before NextBlock rebinds them. Columns live in native table slabs, so the GC // never sees those refs. Once the cursor is exhausted the refs are nulled, so Current cannot read past the end. - text.AppendLine(Inline).Append("private bool NextBlock()\n{\nvar cursor = _cursor;\nbool found = cursor.Next(out var block);\n_cursor = cursor;\nif (!found)\n{\n_remaining = 0;\n") + text.AppendLine(Inline).Append("private bool NextBlock()\n{\nvar cursor = _cursor;\nbool found = cursor.").Append(NextBlock(filters)).Append("(out var block);\n_cursor = cursor;\nif (!found)\n{\n_remaining = 0;\n") .Append(clear).AppendLine("return false;\n}\n_count = block.Count;\n_remaining = _count - 1;\n_entities = block.Entities;"); foreach (var term in values) { @@ -153,7 +159,8 @@ void Ref(string type, string field, string stride) else if (Stamped(term)) text.Append("var tracked").Append(term.Index).Append(" = block.Tracked<").Append(type).Append(">(").Append(term.Index).AppendLine(");") .Append(column).Append(" = ref ").Append(BaseOf).Append("(tracked").Append(term.Index).AppendLine(".Values);") - .Append("_changed").Append(term.Index).Append(" = ref ").Append(BaseOf).Append("(tracked").Append(term.Index).AppendLine(".Changed);"); + .Append("_changed").Append(term.Index).Append(" = ref ").Append(BaseOf).Append("(tracked").Append(term.Index).AppendLine(".Changed);") + .Append("_summary").Append(term.Index).Append(" = ref tracked").Append(term.Index).AppendLine(".Summary;"); else text.Append(column).Append(" = ref ").Append(BaseOf).Append("(block.").Append(term.Mode).Append('<').Append(type).Append(">(").Append(term.Index).AppendLine("));"); } @@ -164,7 +171,9 @@ void Ref(string type, string field, string stride) foreach (var term in values) { arguments.Add("ref " + ColumnName(term)); - if (Stamped(term)) arguments.Add("ref _changed" + term.Index); + if (!Stamped(term)) continue; + arguments.Add("ref _changed" + term.Index); + arguments.Add("ref _summary" + term.Index); } arguments.Add("_entities"); arguments.Add("_count - 1 - _remaining"); @@ -185,8 +194,10 @@ static void GenerateRow(StringBuilder text, IReadOnlyList values) text.Append("readonly ").Append(reference).Append(term.Component.Type).Append(" _value").Append(term.Index).AppendLine(";"); parameters.Add(reference + term.Component.Type + " value" + term.Index); if (!Stamped(term)) continue; - text.Append("readonly ref ulong _changed").Append(term.Index).AppendLine(";"); + text.Append("readonly ref ulong _changed").Append(term.Index).AppendLine(";") + .Append("readonly ref ulong _summary").Append(term.Index).AppendLine(";"); parameters.Add("ref ulong changed" + term.Index); + parameters.Add("ref ulong summary" + term.Index); } text.AppendLine("readonly global::Archevolt.EntityColumn _entities;\nreadonly int _row;"); parameters.Add("global::Archevolt.EntityColumn entities"); @@ -200,7 +211,9 @@ static void GenerateRow(StringBuilder text, IReadOnlyList values) foreach (var term in values) { text.Append("_value").Append(term.Index).Append(" = ref value").Append(term.Index).AppendLine(";"); - if (Stamped(term)) text.Append("_changed").Append(term.Index).Append(" = ref changed").Append(term.Index).AppendLine(";"); + if (Stamped(term)) + text.Append("_changed").Append(term.Index).Append(" = ref changed").Append(term.Index).AppendLine(";") + .Append("_summary").Append(term.Index).Append(" = ref summary").Append(term.Index).AppendLine(";"); } text.Append("_entities = entities;\n_row = row;\n").AppendLine(stamped ? "_now = now;\n}" : "}"); text.AppendLine("public global::Archevolt.Entity Entity => _entities[_row];"); @@ -209,7 +222,9 @@ static void GenerateRow(StringBuilder text, IReadOnlyList values) if (term.Mode == "Optional") text.Append("public bool Has").Append(term.Alias).Append(" => !").Append(Unsafe).Append(".IsNullRef(in _value").Append(term.Index).AppendLine(");"); text.Append("public ref ").Append(term.Mode == "Write" ? "" : "readonly ").Append(term.Component.Type).Append(' ').Append(Escape(term.Alias)).AppendLine("\n{\nget\n{"); if (term.Mode == "Optional") text.Append("if (!Has").Append(term.Alias).AppendLine(") throw new global::System.InvalidOperationException(\"The optional component is absent.\");"); - if (Stamped(term)) text.Append("_changed").Append(term.Index).AppendLine(" = _now;"); + if (Stamped(term)) + text.Append("_changed").Append(term.Index).AppendLine(" = _now;") + .Append("if (_summary").Append(term.Index).Append(" < _now) _summary").Append(term.Index).AppendLine(" = _now;"); text.Append("return ref _value").Append(term.Index).AppendLine(";\n}\n}"); } text.AppendLine("}"); diff --git a/src/Archevolt.Generators/SystemGeneration.cs b/src/Archevolt.Generators/SystemGeneration.cs index cd147f7..cfcdf62 100644 --- a/src/Archevolt.Generators/SystemGeneration.cs +++ b/src/Archevolt.Generators/SystemGeneration.cs @@ -135,8 +135,6 @@ static SystemModel System(IMethodSymbol method, List problems, Cancella components.AddRange(terms.Select(static term => term.Component)); if (form == SystemForm.EachBlock) { - foreach (var term in terms.Where(static term => term.Mode is "Added" or "Changed")) - Error(problems, method, "AV011", $"EachBlock receives whole blocks and cannot filter rows by {term.Mode}<{term.Component.Name}>; use [Each] until block change summaries land."); if (!parameters.Any(static parameter => parameter.Kind == "Column")) Error(problems, method, "AV005", "EachBlock requires at least one component span."); } else if (!parameters.Any(static parameter => parameter.Kind == "Component")) Error(problems, method, "AV005", "Each requires at least one component ref."); @@ -204,13 +202,21 @@ static GeneratedUnit GenerateSystem(IMethodSymbol method, CancellationToken canc text.Append("var ").Append(parameter.Name).Append(" = _message").Append(parameter.Index).Append(parameter.Kind == "MessageWriter" ? ".Write(run);" : ".Read(run);").AppendLine(); if (model.Form == SystemForm.Each) { - text.AppendLine("var cursor = _query0.Begin(run);\nwhile (cursor.Next(out var block))\n{"); - BindColumns(text, model.Queries[0], declarations: true); + // Every visited row stamps each tracked writable column, so the column summary is raised once per block, or + // once per passing row when a change filter may skip every row. TrackedColumn.Write would raise it per row. + var stamped = model.Queries[0].Where(Stamped).ToArray(); + text.AppendLine("var cursor = _query0.Begin(run);"); + if (stamped.Length != 0) text.AppendLine("ulong __now = run.Window.Now;"); var filters = model.Queries[0].Where(static term => term.Mode is "Added" or "Changed").ToArray(); + text.Append("while (cursor.").Append(NextBlock(filters)).AppendLine("(out var block))\n{"); + BindColumns(text, model.Queries[0], declarations: true); for (int index = 0; index < filters.Length; index++) text.Append("var ticks").Append(index).Append(" = block.").Append(filters[index].Mode).Append('(').Append(filters[index].Index).AppendLine(");"); if (model.Parameters.Any(static parameter => parameter.Kind == "Entity")) text.AppendLine("var entities = block.Entities;"); + var raise = string.Concat(stamped.Select(static term => $"if ({ColumnName(term)}.Summary < __now) {ColumnName(term)}.Summary = __now;\n")); + if (filters.Length == 0) text.Append(raise); text.AppendLine("int count = block.Count;\nfor (int row = 0; row < count; row++)\n{"); - if (filters.Length != 0) text.Append("if (!(").Append(string.Join(" && ", filters.Select((_, index) => $"cursor.Window.Contains(ticks{index}[row])"))).AppendLine(")) continue;"); + if (filters.Length != 0) text.Append("if (!(").Append(string.Join(" && ", filters.Select((_, index) => $"cursor.Window.Contains(ticks{index}[row])"))).AppendLine(")) continue;").Append(raise); + foreach (var term in stamped) text.Append(ColumnName(term)).AppendLine(".Changed[row] = __now;"); text.Append(Escape(method.Name)).Append('(').Append(string.Join(", ", model.Parameters.Select(parameter => parameter.Kind switch { "Component" => parameter.RefKind + " " + Element(model.Queries[0].First(term => term.Index == parameter.Index), "row"), @@ -219,7 +225,7 @@ static GeneratedUnit GenerateSystem(IMethodSymbol method, CancellationToken canc } else if (model.Form == SystemForm.EachBlock) { - text.AppendLine("var cursor = _query0.Begin(run);\nwhile (cursor.Next(out var block))\n{"); + text.Append("var cursor = _query0.Begin(run);\nwhile (cursor.").Append(NextBlock(model.Queries[0])).AppendLine("(out var block))\n{"); text.Append(TypeName(method.ContainingType)).Append('.').Append(Escape(method.Name)).Append('(').Append(string.Join(", ", model.Parameters.Select(parameter => parameter.Kind switch { "Column" => BlockColumn(model.Queries[0].First(term => term.Index == parameter.Index)), diff --git a/src/Archevolt/Queries/QueryShape.cs b/src/Archevolt/Queries/QueryShape.cs index 614c451..4c3d034 100644 --- a/src/Archevolt/Queries/QueryShape.cs +++ b/src/Archevolt/Queries/QueryShape.cs @@ -81,4 +81,6 @@ public readonly struct TickWindow public ulong Now { get; } internal TickWindow(ulong last, ulong now) { _last = last; Now = now; } public bool Contains(ulong stamp) => stamp > _last && stamp <= Now; + // A summary at or before the last run rules out every stamp it covers, whatever Now is. + internal bool Precedes(ulong summary) => summary <= _last; } diff --git a/src/Archevolt/Queries/QueryState.cs b/src/Archevolt/Queries/QueryState.cs index 23c494d..0c1bdac 100644 --- a/src/Archevolt/Queries/QueryState.cs +++ b/src/Archevolt/Queries/QueryState.cs @@ -1,3 +1,5 @@ +using System.Runtime.CompilerServices; + namespace Archevolt; public sealed class QueryState @@ -7,6 +9,8 @@ public sealed class QueryState internal readonly int[] ComponentIds; private readonly int[] _required; private readonly int[] _excluded; + // Terms with an Added or Changed filter. QueryCursor.NextChanged skips a table whose summary rules out every row. + internal readonly QueryFilter[] Filters; internal readonly List Matches = []; private int _seen; @@ -17,6 +21,7 @@ internal QueryState(SystemInstance owner, QueryShape shape) ComponentIds = shape.Terms.Select(t => owner.World.Registry.Id(t.TypeKey)).ToArray(); _required = shape.RequiredTypes.Select(owner.World.Registry.Id).ToArray(); _excluded = shape.ExcludedTypes.Select(owner.World.Registry.Id).ToArray(); + var filters = new List(); for (int i = 0; i < shape.Terms.Length; i++) { var component = owner.World.Registry.Components[ComponentIds[i]]; @@ -25,7 +30,12 @@ internal QueryState(SystemInstance owner, QueryShape shape) throw new ArgumentException("Tags support membership and tick filters, not value access."); if (!component.Tracked && (access & (QueryAccess.Added | QueryAccess.Changed)) != 0) throw new ArgumentException($"Change filter on untracked component {component.Name}."); + if ((access & QueryAccess.Added) != 0) + filters.Add(new QueryFilter(i, Added: true)); + if ((access & QueryAccess.Changed) != 0) + filters.Add(new QueryFilter(i, Added: false)); } + Filters = filters.ToArray(); } internal void Refresh() @@ -83,7 +93,22 @@ internal int TotalRows } } -internal sealed record QueryMatch(Archetype Archetype, int[] Columns); +internal readonly record struct QueryFilter(int Term, bool Added); + +internal sealed record QueryMatch(Archetype Archetype, int[] Columns) +{ + internal bool RulesOut(QueryFilter[] filters, TickWindow window) + { + var table = Archetype.Table; + foreach (var filter in filters) + { + int column = Columns[filter.Term]; + if (window.Precedes(filter.Added ? table.AddedSummary[column] : table.ChangedSummary[column])) + return true; + } + return false; + } +} public ref struct QueryCursor { @@ -134,6 +159,18 @@ public bool Next(out BlockView block) } return false; } + + // Like Next, but skips a block when a change-filtered term's summary rules out every row of its table. Generated + // code calls it only for queries with Added or Changed terms. It stays out of line, because the extra code inlined + // through Next exhausted the JIT's inlining budget in generated systems and left their row loops on the stack. + [MethodImpl(MethodImplOptions.NoInlining)] + public bool NextChanged(out BlockView block) + { + while (Next(out block)) + if (!block.RulesOut()) + return true; + return false; + } } public readonly ref struct BlockView @@ -186,7 +223,7 @@ public TrackedColumn Tracked(int term) where T : unmanaged if (!_table!.Components[column].Tracked) throw new InvalidOperationException("The component is not tracked."); return new TrackedColumn(_table.Values(column).Slice(_start, _count), - _table.Changed(column).Slice(_start, _count), _run.Window.Now); + _table.Changed(column).Slice(_start, _count), ref _table.ChangedSummary[column], _run.Window.Now); } public ReadOnlySpan Added(int term) => Ticks(term, QueryAccess.Added); @@ -230,6 +267,8 @@ private ReadOnlySpan Ticks(int term, QueryAccess kind) return (kind == QueryAccess.Added ? _table!.Added(column) : _table!.Changed(column)).Slice(_start, _count); } + internal bool RulesOut() => _match!.RulesOut(_state!.Filters, _run.Window); + private void Validate() { if (_state is null) @@ -251,32 +290,64 @@ public readonly ref struct EntityColumn { private readonly Span _values; private readonly Span _changed; + private readonly ref ulong _summary; private readonly ulong _now; - internal TrackedColumn(Span values, Span changed, ulong now) + internal TrackedColumn(Span values, Span changed, ref ulong summary, ulong now) { _values = values; _changed = changed; + _summary = ref summary; _now = now; } public int Length => _values.Length; // Generated named-query rows stamp one row per access, so they take both spans and bounds-check once per block. + // A row stamped through Changed must also raise Summary to the stamp, or change filters can skip the row. public Span Values => _values; public Span Changed => _changed; - public ref T Write(int row) { _changed[row] = _now; return ref _values[row]; } - public Span WriteAll() { _changed.Fill(_now); return _values; } - public StampedRef At(int row) => new(ref _values[row], ref _changed[row], _now); + public ref ulong Summary => ref _summary; + public ref T Write(int row) + { + _changed[row] = _now; + Raise(ref _summary, _now); + return ref _values[row]; + } + public Span WriteAll() + { + _changed.Fill(_now); + Raise(ref _summary, _now); + return _values; + } + public StampedRef At(int row) => new(ref _values[row], ref _changed[row], ref _summary, _now); + + // Partitioned workers stamp one column of one table, so the summary is written only when it is behind. After the + // first row every worker only reads it, and its cache line is not passed between cores. + internal static void Raise(ref ulong summary, ulong now) + { + if (summary < now) + summary = now; + } } public readonly ref struct StampedRef where T : unmanaged { private readonly ref T _value; private readonly ref ulong _changed; + private readonly ref ulong _summary; private readonly ulong _now; - internal StampedRef(ref T value, ref ulong changed, ulong now) + internal StampedRef(ref T value, ref ulong changed, ref ulong summary, ulong now) { _value = ref value; _changed = ref changed; + _summary = ref summary; _now = now; } - public ref T Value { get { _changed = _now; return ref _value; } } + public ref T Value + { + get + { + _changed = _now; + TrackedColumn.Raise(ref _summary, _now); + return ref _value; + } + } } diff --git a/src/Archevolt/Storage/Descriptors.cs b/src/Archevolt/Storage/Descriptors.cs index 3c09b92..ce155e3 100644 --- a/src/Archevolt/Storage/Descriptors.cs +++ b/src/Archevolt/Storage/Descriptors.cs @@ -126,10 +126,7 @@ public void Put(in T value) where T : unmanaged if (!component.IsTag) _table.Values(column)[_row] = value; if (component.Tracked) - { - _table.Added(column)[_row] = _tick; - _table.Changed(column)[_row] = _tick; - } + _table.Stamp(column, _row, _tick, added: true); } _member++; } diff --git a/src/Archevolt/Storage/Tables.cs b/src/Archevolt/Storage/Tables.cs index 70cecf7..b0e23a5 100644 --- a/src/Archevolt/Storage/Tables.cs +++ b/src/Archevolt/Storage/Tables.cs @@ -11,6 +11,10 @@ internal sealed unsafe class Table : IDisposable private nint _slab; private nint _entities; internal readonly ComponentDescriptor[] Components; + // Per tracked column, a tick no older than any added or changed stamp in the column. A change-filtered query + // skips the table when the summary predates the system's last run. Rows leaving the table do not lower it. + internal readonly ulong[] AddedSummary; + internal readonly ulong[] ChangedSummary; internal int Count; internal int Capacity { get; private set; } internal long Bytes { get; private set; } @@ -21,6 +25,8 @@ internal Table(ComponentDescriptor[] components) _values = new nint[components.Length]; _added = new nint[components.Length]; _changed = new nint[components.Length]; + AddedSummary = new ulong[components.Length]; + ChangedSummary = new ulong[components.Length]; } internal Span Entities => new((void*)_entities, Count); @@ -79,12 +85,26 @@ internal void CopyRowTo(int source, Table target, int destination, int[] targetC ValueBytes(i, source).CopyTo(target.ValueBytes(targetColumn, destination)); if (Components[i].Tracked) { - target.Added(targetColumn)[destination] = Added(i)[source]; - target.Changed(targetColumn)[destination] = Changed(i)[source]; + ulong added = target.Added(targetColumn)[destination] = Added(i)[source]; + ulong changed = target.Changed(targetColumn)[destination] = Changed(i)[source]; + target.AddedSummary[targetColumn] = Math.Max(target.AddedSummary[targetColumn], added); + target.ChangedSummary[targetColumn] = Math.Max(target.ChangedSummary[targetColumn], changed); } } } + // Stamps one row of a tracked column. tick is the newest tick in the world, so it raises both summaries. + internal void Stamp(int column, int row, ulong tick, bool added) + { + if (added) + { + Added(column)[row] = tick; + AddedSummary[column] = tick; + } + Changed(column)[row] = tick; + ChangedSummary[column] = tick; + } + internal void Trim() { if (Count != Capacity) @@ -139,6 +159,8 @@ public void Dispose() Array.Clear(_values); Array.Clear(_added); Array.Clear(_changed); + Array.Clear(AddedSummary); + Array.Clear(ChangedSummary); Count = 0; Capacity = 0; Bytes = 0; diff --git a/src/Archevolt/World.cs b/src/Archevolt/World.cs index 6138beb..833e89d 100644 --- a/src/Archevolt/World.cs +++ b/src/Archevolt/World.cs @@ -318,11 +318,8 @@ private void Move(Entity entity, ArchetypeEdge edge) private static void Stamp(Table table, int column, int row, ulong tick, bool added) { - if (!table.Components[column].Tracked) - return; - if (added) - table.Added(column)[row] = tick; - table.Changed(column)[row] = tick; + if (table.Components[column].Tracked) + table.Stamp(column, row, tick, added); } public void Dispose() diff --git a/tests/Archevolt.Generators.Tests/GeneratorTests.cs b/tests/Archevolt.Generators.Tests/GeneratorTests.cs index 136ad8a..33ccbcb 100644 --- a/tests/Archevolt.Generators.Tests/GeneratorTests.cs +++ b/tests/Archevolt.Generators.Tests/GeneratorTests.cs @@ -681,6 +681,55 @@ public static int Run() Execute(assembly).Should().Be(1000); } + [Fact] + public void EachBlock_change_filters_skip_blocks_whose_table_has_no_change_since_the_last_run() + { + var assembly = CompileAndLoad(""" + using Archevolt; + using System; + [Component, TrackChanges] public readonly record struct Position(int Value); + [Component] public readonly struct Marked { } + [Bundle] public readonly partial record struct Particle(Position Position); + [Query, Write] public readonly ref partial struct Positions { } + public static partial class Systems + { + public static int Blocks, Rows, Target; + [Each, With] public static void Nudge(scoped ref Position position) => position = new(position.Value + 1); + [EcsSystem] public static void Poke(scoped Positions positions) { foreach (var row in positions) if (row.Entity.Index == Target) row.Position = new(-1); } + [EachBlock, Changed] public static void Observe(scoped ReadOnlySpan positions) { Blocks++; Rows += positions.Length; } + } + [EcsModule, IncludeSystems(typeof(Systems)), IncludeBundle, IncludeComponent] public static partial class Game { } + [Schedule, Run(typeof(Systems), nameof(Systems.Nudge))] public partial class Nudger { } + [Schedule, Run(typeof(Systems), nameof(Systems.Poke))] public partial class Poker { } + [Schedule, Run(typeof(Systems), nameof(Systems.Observe))] public partial class Observer { } + public static class Acceptance + { + static int Observed(BoundSchedule observer) { Systems.Blocks = Systems.Rows = 0; observer.Run(); return Systems.Blocks * 1000 + Systems.Rows; } + public static int Run() + { + using var world = new WorldBuilder().AddModule(Game.Module).Build(); + var entities = new Entity[100]; + for (int i = 0; i < entities.Length; i++) entities[i] = world.Spawn(new Particle(new(i))); + for (int i = 0; i < 10; i++) world.Insert(entities[i], default(Marked)); + using var nudger = Nudger.Create(world); using var poker = Poker.Create(world); using var observer = Observer.Create(world); + if (Observed(observer) != 2100) return -1; + if (Observed(observer) != 0) return -2; + nudger.Run(); + if (Observed(observer) != 1010) return -3; + Systems.Target = entities[50].Index; + poker.Run(); + if (Observed(observer) != 1090) return -4; + // The moved row carries its change into a table whose summary predates the observer's last run. + nudger.Run(); + world.Remove(entities[0]); + if (Observed(observer) != 2100) return -5; + return Observed(observer); + } + } + """); + Execute(assembly).Should().Be(0); + } + [Fact] public void EachBlock_failure_preserves_partial_writes_and_marks_untouched_rows_changed() { @@ -1005,8 +1054,6 @@ public static int Run() [InlineData("public static partial class S { [EcsSystem] public static void A() {} } [Schedule, Stage(\"update\"), Run(typeof(S), \"A\")] public partial class Simulation { }", "AV007")] [InlineData("public static partial class S { [EcsSystem] public static void A() {} } [Schedule, Run(typeof(S), \"A\", Set = \"missing\")] public partial class Simulation { }", "AV007")] [InlineData("[Schedule, Stage(\"same\", Order = 0), Stage(\"same\", Order = 1)] public partial class Simulation { }", "AV007")] - [InlineData("[Component, TrackChanges] public readonly record struct P(int Value); public static partial class S { [EachBlock, Changed

] public static void Run(scoped System.Span

values) {} }", "AV011")] - [InlineData("[Component, TrackChanges] public readonly record struct P(int Value); public static partial class S { [EachBlock, Added

] public static void Run(scoped System.Span

values) {} }", "AV011")] [InlineData("[Component] public readonly record struct P(int Value); [Component] public readonly record struct Q(int Value); public static partial class S { [EachBlock] public static void Run(scoped System.Span

values, scoped ref Q other) {} }", "AV005")] [InlineData("[Component] public readonly record struct P(int Value); public static partial class S { [EachBlock] public static void Run(System.Span

values) {} }", "AV005")] [InlineData("[Component] public readonly record struct P(int Value); public static partial class S { [EachBlock] public static void Run(scoped System.Span

values, EntityColumn entities) {} }", "AV005")] diff --git a/tests/Archevolt.Tests/QueryRuntimeTests.cs b/tests/Archevolt.Tests/QueryRuntimeTests.cs index a9232cd..44df480 100644 --- a/tests/Archevolt.Tests/QueryRuntimeTests.cs +++ b/tests/Archevolt.Tests/QueryRuntimeTests.cs @@ -188,6 +188,54 @@ public void Tracking_observes_equal_writes_preserves_added_and_ignores_structura Assert.Equal(1, changedCount); } + [Fact] + public void Change_filters_skip_tables_without_a_stamp_since_the_last_run() + { + using var world = CreateWorld(); + Entity plain = world.Spawn(new Particle(new Position(1), new Velocity(1))); + Entity extra = world.Spawn(new Particle(new Position(2), new Velocity(1))); + world.Insert(extra, new Extra(0)); + int changedBlocks = 0, addedBlocks = 0; + QueryAction countChanged = (run, query) => { changedBlocks = Blocks(query.Begin(run)); }; + QueryAction countAdded = (run, query) => { addedBlocks = Blocks(query.Begin(run)); }; + QueryAction touchExtra = (run, query) => + { + var cursor = query.Begin(run); + while (cursor.Next(out var block)) + _ = block.Tracked(0).At(0).Value; + }; + using var changes = BoundSchedule.Bind(world, SystemDescriptor.Create("changed", b => new QueryBody(b, countChanged)) + .Query(QueryShape.Create("changed").Read(0).Changed(0).Build()).Build()); + using var additions = BoundSchedule.Bind(world, SystemDescriptor.Create("added", b => new QueryBody(b, countAdded)) + .Query(QueryShape.Create("added").Read(0).Added(0).Build()).Build()); + using var writer = BoundSchedule.Bind(world, SystemDescriptor.Create("write", b => new QueryBody(b, touchExtra)) + .Query(QueryShape.Create("write").Write(0).With().Build()).Build()); + changes.Run(TestContext.Current.CancellationToken); + additions.Run(TestContext.Current.CancellationToken); + Assert.Equal(2, changedBlocks); + Assert.Equal(2, addedBlocks); + changes.Run(TestContext.Current.CancellationToken); + additions.Run(TestContext.Current.CancellationToken); + Assert.Equal(0, changedBlocks); + Assert.Equal(0, addedBlocks); + world.Insert(plain, new Position(3)); + changes.Run(TestContext.Current.CancellationToken); + additions.Run(TestContext.Current.CancellationToken); + Assert.Equal(1, changedBlocks); + Assert.Equal(0, addedBlocks); + writer.Run(TestContext.Current.CancellationToken); + changes.Run(TestContext.Current.CancellationToken); + Assert.Equal(1, changedBlocks); + } + + private static int Blocks(QueryCursor cursor) + { + int count = 0; + while (cursor.NextChanged(out _)) + count++; + return count; + } + private static int Count(QueryCursor cursor) { int count = 0;