Skip to content

Fix the Qwen GQA test failure.#4385

Open
SandSnip3r wants to merge 1 commit into
pytorch:mainfrom
SandSnip3r:dev-fix-gqa-bug
Open

Fix the Qwen GQA test failure.#4385
SandSnip3r wants to merge 1 commit into
pytorch:mainfrom
SandSnip3r:dev-fix-gqa-bug

Conversation

@SandSnip3r

Copy link
Copy Markdown
Collaborator

Description

Fix the Qwen GQA test failure.

The Qwen GQA test failed in the decomposed variant due to the intemediate between the matmuls being downcast to fp16. Plumb use_fp32_acc through the dynamo backend.
Pick a specific seed for the GQA test.
Reenable GQA test for fp16.

Type of change

Please delete options that are not relevant and/or add your own.

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

The Qwen GQA test failed in the decomposed variant due to the intemediate between the matmuls being downcast to fp16.
Plumb use_fp32_acc through the dynamo backend.
Pick a specific seed for the GQA test.
Reenable GQA test for fp16.
@meta-cla meta-cla Bot added the cla signed label Jul 6, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: lowering Issues re: The lowering / preprocessing passes component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: torch_compile labels Jul 6, 2026
@github-actions github-actions Bot requested a review from narendasan July 6, 2026 22:20
@narendasan narendasan requested a review from zewenli98 July 6, 2026 22:25

@zewenli98 zewenli98 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SandSnip3r The idea looks good! The decomposed attention path needs the entire computation in FP32, not just matmul.

My only concern is that the definition of use_fp32_acc currently suggests FP32 applies only to matmul, whereas this implementation extends it to the whole attention path. It may help to clarify that in the definition, for example by noting that [op1, op2, ...] will use FP32 for the full computation rather than matmul alone for accuracy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests component: torch_compile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants