Repository navigation
feat(quantization): add qparam_qspec for double-scale quantization - #161
Open
dengqiaoyu wants to merge 3 commits into
Open
dengqiaoyu wants to merge 3 commits into
dengqiaoyu wants to merge 3 commits into
Conversation
- Double-scale formats store each block's qparams as low-precision codes under shared parent scales (e.g. NVFP4, GGUF-style scales and mins), so elements must be quantized against the decoded qparams; `qparam_qspec` is a nested `QuantizationSpec` that quantizes the grid of per-block qparams like any other tensor - Quantize the scale, and under `qformulation="minval"` the minval on its non-negative grid, with separate nested fake quantizers, and re-derive an asymmetric zero point from the decoded scale - Fit nested granularities to the outer grid: a nested block axis left at `None` follows the outer one, and a per-tensor parent's single scale is quantized per tensor - Carry `qparam_qspec` through the graph-mode field maps, apply `qat_schedule` transitions to the nested fake quantizers, and count nested codes in bits per weight - `Quantizer.finalize` raises `NotImplementedError` for Core AI and Core ML until export can store quantized qparams
- Reuse existing helpers (`FakeQuantizeImplBase.get_block_size`, `get_parent_module_and_attr_name`) instead of re-deriving block sizes and parent modules - Run the `qparam_qspec` export guard before the dynamic-calculator guard in `Quantizer.finalize`, so a nested dynamic calculator is reported under its `qparam_qspec` instead of by its internal module name - Consolidate test setup (shared spec helpers, `_minval_with_block_1`, `_EXECUTION_MODES`) and drop a test that re-implemented the zero-point formula, which the hand-computed zero-point test covers - Fix docstrings and the config docs example that no longer matched the code
- Split the `qparam_qspec` config example into GGUF double scale and NVFP4 subsections, introduced as popular formats that double scale can specify - Drop the "special case" framing of NVFP4 from the overview, the config field list, the `qparam_qspec` docstring and a test section header - Note that exact Q4_K needs 6-bit codes under FP16 parents, which are not supported yet, so the GGUF example uses uint8 codes under FP32 parents
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
qparam_qspecis a nestedQuantizationSpecthat quantizes the grid of per-block qparams like any other tensorqformulation="minval"the minval on its non-negative grid, with separate nested fake quantizers, and re-derive an asymmetric zero point from the decoded scaleNonefollows the outer one, and a per-tensor parent's single scale is quantized per tensorqparam_qspecthrough the graph-mode field maps, applyqat_scheduletransitions to the nested fake quantizers, and count nested codes in bits per weightQuantizer.finalizeraisesNotImplementedErrorfor Core AI and Core ML until export can store quantized qparams