fix(mem_wal): run grace eviction on its own task, not the flush handler's ticker

[xuanyu-z]

A frozen memtable's bytes are reclaimed only by the grace sweep. It ran as a ticker on the memtable-flush handler, and TaskDispatcher biases a handler's mailbox ahead of its ticker — deliberately, because a tick that only duplicates work a message would have done anyway must never starve a freeze completion or close()'s final append.

That reasoning does not hold for this tick. While handle() is awaiting, the select! is not evaluated at all, so the ticker's only opportunity is the instant after a flush returns — and at any write rate where sealing outpaces flushing, the mailbox is never empty at that instant. The sweep then never runs for as long as the load lasts, so frozen memtables accumulate without bound and resident memory grows until the process is killed.

The change

Eviction runs through a new TaskExecutor::add_periodic, which spawns work on its own task with the same cancellation and shutdown semantics as add_handler.

sweep_expired_frozen becomes a free function over the shared state and memory view, so a task that does not own the handler can run it.

The periodic timer uses the same missed-tick behaviour as the dispatcher: Burst would replay every tick missed while the work ran, which for slow work leaves the timer permanently ready and starves the cancellation arm.

TriggerMemTableFlush::SweepExpired is removed. The ticker was its only sender, so the variant, its Debug arm and the handler arm became unreachable; doc references to it now name sweep_expired_frozen. Note for reviewers: that variant is reachable from outside the crate, so strictly this drops a public enum variant. Nothing constructs it, but say the word and I will keep it with an explicit sender instead.

Testing

test_saturated_handler_starves_its_ticker_but_not_a_periodic_task asserts both halves — that a saturated handler starves its own ticker, and that a periodic task keeps its cadence beside it — so the distinction holds. Without the second assertion the test passes on the old code: a local-store flush is fast enough that the mailbox drains and the ticker runs. It runs under start_paused, so the cadence costs no wall time.

cargo fmt --all --check, ci/check_proto_comments.py, and cargo test -p lance --lib dataset::mem_wal (664 passing) are clean, as is cargo doc for intra-doc links. The two findings cargo clippy surfaces are pre-existing, in lance-io, which this PR does not touch.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The dedicated periodic worker addresses sweeper starvation without weakening mailbox priority or the grace/backpressure contract. This revision safely removes the obsolete SweepExpired message path that no longer had a sender, leaving one eviction mechanism; scheduler cadence and real grace eviction remain verified.