docs: add datatype and hardware support matrix#3195
Open
CodersAcademy006 wants to merge 1 commit into
Open
Conversation
This was referenced Jul 9, 2026
Add a page documenting which low-precision quantization formats (FP8 per-tensor, FP8 block scaling, MXFP8, NVFP4) are supported on which NVIDIA compute capabilities, along with the default recipe selected per architecture. The tables mirror the runtime capability checks in transformer_engine.pytorch.quantization and transformer_engine.pytorch.utils so behavior and docs stay aligned. The NVFP4 entry for compute capability 12.0 notes that the default recipe's stochastic rounding is architecture specific to 10.0 and 10.3, so round-to-nearest is required on 12.0. Closes NVIDIA#2512 Closes NVIDIA#2304 Refs NVIDIA#2668 Refs NVIDIA#2255 Signed-off-by: Srijan Upadhyay <srjnupadhyay@gmail.com>
823e1f1 to
3881e2c
Compare
Contributor
Greptile SummaryThis PR adds a documentation page for low-precision datatype support. The main changes are:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (1): Last reviewed commit: "docs: add datatype and hardware support ..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
There is no single place in the docs that states which low-precision formats
are supported on which GPU architecture. The information is spread across the
README, inline compute-capability checks, and test skip conditions. Several
issues ask for exactly this (#2512, #2304), and related threads keep hitting the
same confusion about SM120 support (#2668, #2255).
Solution
Add
docs/support_matrix.rst, a page that documents format support keyed bycompute capability, since that is the value Transformer Engine checks at runtime.
It covers BF16, FP8 per-tensor, FP8 block scaling, MXFP8, and NVFP4, plus the
default recipe selected per architecture.
The tables are derived directly from the runtime capability checks so docs and
behavior stay aligned:
_compute_fp8_support,_compute_mxfp8_support,_compute_nvfp4_support,_compute_fp8_block_scaling_support, andget_default_fp8_recipeintransformer_engine/pytorch/quantization.pytransformer_engine/pytorch/utils.pyTwo nuances are called out explicitly because they are common sources of
confusion:
limited to 10.0 through 10.x.
stochastic rounding whose FP4 conversion instructions are architecture specific
to 10.0 and 10.3. On 12.0 the default recipe raises an architecture-specific
error and round-to-nearest is required.
The page is linked under the Getting Started section in
docs/index.rst.Testing
Validated the reStructuredText structure with docutils. I was not able to run a
full Sphinx build locally because that requires a compiled Transformer Engine,
which needs a CUDA GPU I do not have access to. A reviewer running the docs build
in CI can confirm the page renders as expected.
Limitations
This page documents the current state of the runtime capability checks. If those
checks change, the tables need to be updated to match. A note in the page states
this so the two stay in sync.
Closes #2512
Closes #2304
Refs #2668
Refs #2255