Skip to content

Fix spec prefix capture on compressor-less Flash layers - #648

Open
gilbert-barajas wants to merge 1 commit into
antirez:ds4f-mxfp4from
gilbert-barajas:fix-flash-prefix-capture
Open

Fix spec prefix capture on compressor-less Flash layers#648
gilbert-barajas wants to merge 1 commit into
antirez:ds4f-mxfp4from
gilbert-barajas:fix-flash-prefix-capture

Conversation

@gilbert-barajas

Copy link
Copy Markdown

metal_graph_capture_prefix_attn_state() and metal_graph_capture_prefix_index_state() return false when the layer's spec_prefix1_* buffers are NULL.

Those buffers are allocated only for layers that have a compressor: attention state under ratio != 0, index state under ratio == 4 (ds4.c:17059-17139). For the Flash variant, ds4_expected_layer_compress_ratio() returns 0 for layers 0 and 1:

case DS4_VARIANT_FLASH:
    if (il < 2) return 0;

So on any Flash model the buffers for layer 0 are NULL, and the strict decode2 verifier — which sets spec_capture_prefixes and loops every layer with no ratio gating (ds4.c:34495, :34530) — fails at il=0 and breaks out. The verifier can't run on Flash at all, independently of SSD streaming.

spec_frontier_snapshot(), spec_frontier_restore() and spec_frontier_commit_prefix() already skip these layers with if (ratio == 0) continue and gate index work on if (ratio == 4). The two capture functions are the only ones in that path that don't. This mirrors them.

Returning true for a skipped layer is safe because the write-gate and the read-gate are the same: the only reader of the prefix1 buffers and spec_prefix_n_comp is spec_frontier_commit_prefix(), and each of its reads is already behind the matching predicate (:49754, :49763). The rollback path, spec_frontier_restore(), reads the main frontier buffers rather than the prefix1 ones. No consumer can observe uninitialised state for the layers now skipped.

Six lines, two functions. Builds clean under -Wall -Wextra.

Found while testing MTP on DeepSeek-V4-Flash; the bug is independent of that work.

metal_graph_capture_prefix_attn_state() and _index_state() return false when
the layer's prefix buffers are NULL. For the Flash variant the first two layers
have an attention-compressor ratio of 0 (ds4_expected_layer_compress_ratio:
"if (il < 2) return 0"), so those buffers are never allocated and the capture
fails at il=0.

The effect is that the strict decode2 verifier cannot run on any Flash model,
with or without SSD streaming.

spec_frontier_snapshot() and spec_frontier_commit_prefix() already skip these
layers with an explicit "if (ratio == 0) continue". The two capture functions
are the only places in the same code path that do not. This mirrors them:
ratio 0 carries no attention frontier, and only ratio-4 layers carry an index
frontier.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant