Skip to content

[gfx1250] Use TDM for MXFP8 and FP8 quantization - #778

Draft
matthiasdiener wants to merge 5 commits into
devfrom
mdiener/gfx1250-tdm-mxfp8-quant
Draft

[gfx1250] Use TDM for MXFP8 and FP8 quantization#778
matthiasdiener wants to merge 5 commits into
devfrom
mdiener/gfx1250-tdm-mxfp8-quant

Conversation

@matthiasdiener

@matthiasdiener matthiasdiener commented Oct 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

Addresses part of https://github.com/AMD-ROCm-Internal/frameworks-internal/issues/172.

Performance: https://github.com/AMD-ROCm-Internal/frameworks-internal/issues/172#issuecomment-6353557004

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

Please list the changes introduced in this PR:

  • Change A
  • Change B

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@matthiasdiener
Add gfx1250 Tensor Data Mover helper
3be50bc 
util/rocm_tdm.cuh wraps the gfx1250 TDM builtins for asynchronous 2D tile
copies between global memory and LDS (tdm::load_2d, tdm::store_2d,
tdm::wait). It packs the descriptors itself instead of using
hip/amd_detail/amd_gfx1250_TDM.h, whose stride setters truncate, and is
only defined when compiling for gfx1250 with a compiler that provides the
unified TDM builtins (NVTE_ROCM_HAS_TDM).

test_rocm_tdm covers per-wave load/store round trips for 1-8 byte
elements, partial edge tiles and a row pitch different from the width.
@matthiasdiener
use TDM for MXFP8 quant
38c0d8d
@matthiasdiener
add fp8
deaafd8
@matthiasdiener matthiasdiener self-assigned this Oct 22, 2026
@matthiasdiener matthiasdiener added the ci-level 1 CI test level 1 label Oct 22, 2026
@matthiasdiener matthiasdiener changed the title [gfx1250] Use the TDM for MXFP8 and FP8 quantization [gfx1250] Use TDM for MXFP8 and FP8 quantization Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wenchenvincent wenchenvincent Awaiting requested review from wenchenvincent wenchenvincent will be requested when the pull request is marked ready for review wenchenvincent is a code owner
@ipanfilo ipanfilo Awaiting requested review from ipanfilo ipanfilo will be requested when the pull request is marked ready for review ipanfilo is a code owner
@wangye805 wangye805 Awaiting requested review from wangye805 wangye805 will be requested when the pull request is marked ready for review wangye805 is a code owner
@Micky774 Micky774 Awaiting requested review from Micky774 Micky774 will be requested when the pull request is marked ready for review Micky774 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@matthiasdiener matthiasdiener

Labels

ci-level 1 CI test level 1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@matthiasdiener