Skip to content

docs: add datatype and hardware support matrix#3195

Open
CodersAcademy006 wants to merge 1 commit into
NVIDIA:mainfrom
CodersAcademy006:docs/datatype-hardware-support-matrix
Open

docs: add datatype and hardware support matrix#3195
CodersAcademy006 wants to merge 1 commit into
NVIDIA:mainfrom
CodersAcademy006:docs/datatype-hardware-support-matrix

Conversation

@CodersAcademy006

Copy link
Copy Markdown

Problem

There is no single place in the docs that states which low-precision formats
are supported on which GPU architecture. The information is spread across the
README, inline compute-capability checks, and test skip conditions. Several
issues ask for exactly this (#2512, #2304), and related threads keep hitting the
same confusion about SM120 support (#2668, #2255).

Solution

Add docs/support_matrix.rst, a page that documents format support keyed by
compute capability, since that is the value Transformer Engine checks at runtime.
It covers BF16, FP8 per-tensor, FP8 block scaling, MXFP8, and NVFP4, plus the
default recipe selected per architecture.

The tables are derived directly from the runtime capability checks so docs and
behavior stay aligned:

  • _compute_fp8_support, _compute_mxfp8_support, _compute_nvfp4_support,
    _compute_fp8_block_scaling_support, and get_default_fp8_recipe in
    transformer_engine/pytorch/quantization.py
  • the BF16 check in transformer_engine/pytorch/utils.py

Two nuances are called out explicitly because they are common sources of
confusion:

  • MXFP8 is not currently supported on compute capability 12.0 and higher; it is
    limited to 10.0 through 10.x.
  • NVFP4 is reported as available on 10.0 and higher, but the default recipe uses
    stochastic rounding whose FP4 conversion instructions are architecture specific
    to 10.0 and 10.3. On 12.0 the default recipe raises an architecture-specific
    error and round-to-nearest is required.

The page is linked under the Getting Started section in docs/index.rst.

Testing

Validated the reStructuredText structure with docutils. I was not able to run a
full Sphinx build locally because that requires a compiled Transformer Engine,
which needs a CUDA GPU I do not have access to. A reviewer running the docs build
in CI can confirm the page renders as expected.

Limitations

This page documents the current state of the runtime capability checks. If those
checks change, the tables need to be updated to match. A note in the page states
this so the two stay in sync.

Closes #2512
Closes #2304
Refs #2668
Refs #2255

@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Jul 9, 2026
Add a page documenting which low-precision quantization formats
(FP8 per-tensor, FP8 block scaling, MXFP8, NVFP4) are supported on
which NVIDIA compute capabilities, along with the default recipe
selected per architecture. The tables mirror the runtime capability
checks in transformer_engine.pytorch.quantization and
transformer_engine.pytorch.utils so behavior and docs stay aligned.

The NVFP4 entry for compute capability 12.0 notes that the default
recipe's stochastic rounding is architecture specific to 10.0 and 10.3,
so round-to-nearest is required on 12.0.

Closes NVIDIA#2512
Closes NVIDIA#2304
Refs NVIDIA#2668
Refs NVIDIA#2255

Signed-off-by: Srijan Upadhyay <srjnupadhyay@gmail.com>
@CodersAcademy006 CodersAcademy006 force-pushed the docs/datatype-hardware-support-matrix branch from 823e1f1 to 3881e2c Compare July 9, 2026 05:20
@CodersAcademy006 CodersAcademy006 marked this pull request as ready for review July 9, 2026 05:25
@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a documentation page for low-precision datatype support. The main changes are:

  • A new support matrix page for GPU compute capabilities.
  • Tables for BF16, FP8, FP8 block scaling, MXFP8, NVFP4, and default recipes.
  • A main docs index entry linking the new page.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
docs/index.rst Adds the new support matrix page to the main documentation toctree.
docs/support_matrix.rst Adds datatype support tables and caveats that match the documented runtime capability checks.

Reviews (1): Last reviewed commit: "docs: add datatype and hardware support ..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Docs] Add datatype × hardware support matrix Document Supported Datatypes Per Compute Capability

1 participant