Repository navigation
Conversation
launch_gemm_a8w8 launches ceil(N / tile_n) N-tiles, but the B copy and the C store of the last tile were not bounded in N. When N % tile_n != 0, the last tile read B rows past N and stored all tile_n columns from its start. With ldc == N, the columns past N landed at the start of the next row of C, and past the end of C for the last row. The upstream tests only ran shapes with N % tile_n == 0. Bound the B copy (in 16-row blocks) and the C store by the columns left from the tile start: the TDM zero-fills B rows past N and drops C columns past N. The blockscale B-scale copy is now bounded for every tile size, since a 256-wide tail tile would otherwise read past the scales. N must stay a multiple of 16 (B is preshuffled in 16-row blocks), and blockscale still needs N % 128 == 0. Tests add per-token/per-channel cases with N = 2112 (the DeepSeek-V3 fused qkv_a width) and a blockscale case with N = 1152 and 256-wide tiles; the checks reject N % 16 != 0 instead of N % tile_n != 0. Signed-off-by: Jin Pan <jin.pan@amd.com>
With ldc == N, the N-tail cases failed on main only when a stray tail store landed after the owner tile's store. The test runner now fills the C padding with NaN and asserts that it stays NaN, and two padded N-tail cases put every stray column into the padding, so they fail on main on every run. The launcher's docstring states its N and K requirements. Signed-off-by: Jin Pan <jin.pan@amd.com>
jhinpan
changed the title
The padding check filled the columns past N with NaN and asserted that they were still NaN, so a stray store of a NaN passed. The stray columns of main are computed from B rows past N, whose contents are undefined, so such a store can be a NaN. Fill the padding with a signaling-NaN bit pattern that no computed output carries and compare the raw bits. Signed-off-by: Jin Pan <jin.pan@amd.com>
|
The bad results in the summary are shapes this kernel does not support. The tests skip
Clamping B and C in the descriptor changes that contract, and #1228 / #1233 then have to apply the same bound again in |
Per review, the caller pads N to a multiple of tile_n. Revert the B and C clamps and the N-tail test cases, and restore the test skip. The launcher docstring now states the shape contract, and _run_and_check still checks that no store reaches the C columns past N. Signed-off-by: Jin Pan <jin.pan@amd.com>
jhinpan
changed the title
|
Done in 2fd1236: the B and C clamps and the N-tail cases are gone, and the |
launch_gemm_a8w8 requires N % tile_n == 0, K % tile_k == 0 and K // tile_k >= num_buffers - 1 (blockscale also N % 128 == 0); its docstring now says so. For a padded C, _run_and_check fills the columns past N with a signaling-NaN bit pattern and checks that they keep it, so any store past N fails the test, a NaN store included. Folded in from #1229. Signed-off-by: Jin Pan <jin.pan@amd.com>
3 participants
launch_gemm_a8w8(the gfx1250 FP8 GEMM) requiresN % tile_n == 0; the caller pads N to a multiple oftile_n. This PR states the shape contract in the launcher docstring:N % tile_n == 0,K % tile_k == 0andK // tile_k >= num_buffers - 1(blockscale alsoN % 128 == 0). A partial last M tile is clamped.The tests keep skipping shapes that break the contract. For the cases with a padded C (
ldc > N),_run_and_checknow fills the columns past N with a signaling-NaN bit pattern before the launch and checks that the bits do not change, so a store past N fails the test, even one that stores a NaN.No kernel change.