Add SiFive U74 target with a scalar 4×4 register-tiled GEMM kernel#5903
Open
hmeiland wants to merge 6 commits into
Open
Add SiFive U74 target with a scalar 4×4 register-tiled GEMM kernel#5903hmeiland wants to merge 6 commits into
hmeiland wants to merge 6 commits into
Conversation
The SiFive U74 (RV64GC; e.g. StarFive JH7110 / VisionFive 2) is a scalar,
in-order core with no RVV, so today it falls back to RISCV64_GENERIC whose
S/D GEMM uses the generic 2x2 C micro-kernel.
Per the U74 Core Complex Manual (Table 169) fmadd.d has a 7-cycle latency
at repeat rate 1 (fully pipelined). A 2x2 tile exposes only 4 independent
accumulator chains -- fewer than the FMA latency -- so the FP pipe stalls
on the accumulator dependency, and the 1:1 load:FMA ratio saturates the
single load/store pipe ("only one outstanding line fill", manual 8.2).
This adds a portable 4x4 GEMM micro-kernel and a dedicated U74 target:
- kernel/generic/gemmkernel_4x4.c: 16-accumulator 4x4 register tile. 16
independent chains exceed the 7-cycle latency, and the load:FMA ratio
drops to 1:2. 16 acc + 4 A + 4 B fit RV64G's 32 FP registers without
spilling. Full 4/2/1 edge handling in both M and N.
- U74 target wiring: getarch.c (FORCE_U74, 32 KiB/64 B L1D, 2 MiB L2),
param.h (S/D UNROLL 4/4; complex stays 2/2), kernel/riscv64/KERNEL.U74
(S/D GEMM -> gemmkernel_4x4 + gemm_[nt]copy_4; S/D TRMM -> existing
trmmkernel_4x4), Makefile.prebuild + Makefile.riscv64 (-mtune=sifive-u74),
TargetList.txt, cpuid_riscv64.c.
The 4x4 kernel was verified numerically against a naive reference GEMM,
driven through the real gemm_tcopy_4 / gemm_ncopy_4 packing routines,
across 27,436 M/N/K x alpha combinations covering every 4/2/1 tail case:
worst absolute error 0.
Build with: make TARGET=U74
The StarFive JH7110's U74 cores implement rv64imafdc_..._zba_zbb, so the U74 target now compiles with -march=rv64imafdc_zba_zbb. The generic RISCV64_GENERIC target stays at bare rv64imafdc for portability across unknown RV64GC cores. Measured on the VisionFive 2 (GCC 13.3, -mtune=sifive-u74 held constant): GCC emits Zba shift-add instructions in the packing routines, but DGEMM is unchanged -- the 4x4 kernel holds 1.533 GF either way and packing 1.74 vs 1.75 GB/s. The 4x4 micro-kernel is FMA-bound (fused fmadd.d with immediate-offset loads) and packing is LPDDR4-bandwidth-bound, so integer address generation is not on the critical path. The flag is nonetheless the correct -march for the silicon, is free, and can only help address-gen-bound code elsewhere in the library. The remaining GEMM headroom on the U74 is microarchitectural scheduling (a hand-written assembly micro-kernel), not the ISA.
A same-board controlled HPL A/B (VisionFive 2, TARGET=U74, N=10000, NB=192, 2x2 grid, both libraries clean-built, residual PASSED) measures 4.97 vs 4.86 GFLOPS (+2.25%) for DGEMM_Q=256 over the inherited 128; at a memory-filling N=27456 the tuned build sustains 5.41 vs 5.24 GFLOPS (+3.24%). With HPL's NB=192, KC=256>=192 lets each trailing-update K panel run in a single KC block, halving the C read-modify-write traffic versus KC=128 (which splits K=192 into 128+64). A single-core P/Q sweep over the real packing + micro-kernel objects confirms the GEMM is compute-bound (whole grid within 3%), so P and R are left unchanged; SGEMM_Q was already 240.
Adds a hand-written RV64GC scalar 4x4 DGEMM micro-kernel (kern_u74.S) and wires it as the U74 DGEMMKERNEL via a hybrid dispatcher (gemmkernel_4x4_u74.c): the fast path (bm,bn multiples of 4, even bk, non-TRMM) runs the asm; all other shapes, odd bk, and the TRMM builds fall back to the portable C 4x4 kernel. The asm uses a 4x4 register tile (16 accumulators), full operand double- buffering (P/Q ping-pong) with one-iteration lookahead, and load-before-FMA issue ordering matched to the U74's dual-issue in-order front end. Probes show this reaches the FP-pipe peak (~16.5 cycles / 16 fmadd.d = 2.9 GF on L1-resident data); the streaming plateau is memory-latency-bound, not the schedule. Measured on a VisionFive 2 (single-core, KC=256): micro-kernel 1.88 vs 1.54 GF (+22%), full blocked DGEMM 1.77 vs 1.48 GF (+20%); the advantage holds under 4-core contention (+17%). End-to-end HPL N=10000 (4 cores, Q=256): 5.17 vs 4.97 GF (+4.0%), residual PASSED. Correctness validated against the full BLAS Level-3 test suite (DGEMM 17,496 computational calls, 0 failures). kern_u74.S is the readable source; the .c embeds it via top-level __asm__ so it builds as a single OpenBLAS kernel object with no build-system changes.
Full-memory HPL N=27456 (asm hybrid + Q=256) measured at 5.99 GFLOPS (residual PASSED, ~50% of the 12 GF peak, +10.7% over the tuned C kernel, 1.81x the stock 2x2) - the best clean figure. Doc-only header update.
Mirror the RISCV64_GENERIC target blocks for U74 in cmake/prebuild.cmake (cache defines: 32 KiB L1D / 2 MiB L2, 64 B lines, matching getarch), cmake/system.cmake and cmake/cc.cmake (-march=rv64imafdc_zba_zbb -mabi=lp64d -mtune=sifive-u74), so 'cmake -DTARGET=U74' builds alongside the Makefile path. Verified on a VisionFive 2: configure recognizes the target and a full 'cmake --build' produces libopenblas.a with the U74 march/mtune flags applied to the kernel objects (0 errors).
Author
|
Update — addressed two things a reviewer would likely raise:
Both the Makefile and CMake builds pass the full BLAS Level-3 test suite (DGEMM 17,496 calls, 0 failures) and HPL (residual PASSED) on physical U74 hardware. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
RV64GC application cores without the vector extension (SiFive U74/U54, Andes AX45MP, SiFive P550, …) currently fall back to the
RISCV64_GENERICtarget, whose GEMM uses a portable 2×2 C micro-kernel. On the StarFive VisionFive 2 (JH7110, quad-core U74) that leaves HPL at ≈3.1 GFLOPS — only ~26 % of the 12 GFLOPS double-precision peak.The bottleneck is the micro-kernel, not the memory system. The U74's
fmadd.dhas a 7-cycle latency at 1/cycle throughput on a single FP pipe; a 2×2 tile exposes only four independent accumulator chains — fewer than the latency — so the pipe stalls on the accumulator recurrence, while the 1:1 load:FMA ratio saturates the lone load/store port.What this adds
A dedicated
U74target and a scalar 4×4 register-tiled GEMM micro-kernel:kernel/generic/gemmkernel_4x4.c— portable C 4×4 kernel (16 accumulators). Raises in-flight chains above the FMA latency and lowers the load:FMA ratio to 1:2, fitting RV64G's 32 FP registers with no spill; handles all 4/2/1 edge cases.kernel/generic/kern_u74.S+kernel/generic/gemmkernel_4x4_u74.c— a hand-scheduled scalar assembly DGEMM micro-kernel (full operand double-buffering / P·Q ping-pong, load-before-FMA issue ordering matched to the dual-issue in-order front end), wired through a hybrid dispatcher: the asm serves full 4×4 / even-bkDGEMM blocks and falls back to the C kernel for ragged edges, oddbk, and TRMM.kern_u74.Sis the readable source; the.cembeds it via top-level__asm__so it builds as a single kernel object with no build-system changes.getarch.c,cpuid_riscv64.c,TargetList.txt,KERNEL.U74,param.h,Makefile.riscv64,Makefile.prebuild. Block sizes tuned for the 32 KiB L1D / 2 MiB shared L2 (DGEMM_Q = 256, aligning KC with a typical HPLNB = 192).Measured results (physical 8 GiB VisionFive 2, quad U74 @ 1.5 GHz)
Single-core DGEMM micro-kernel: 1.63 vs 1.00 GFLOPS over the generic 2×2 (≈1.6×). Micro-benchmark probes show the assembly reaches the FP-pipe peak — 2.90 GFLOPS on L1-resident data (~16.5 cycles per 16
fmadd.d).Four-core HPL, same board, matched N / NB=192 / 2×2 grid:
End-to-end +81 % over the generic 2×2 at the memory-filling size; every run residual-PASSED.
Correctness / validation
TARGET=U74build passes the OpenBLAS BLAS Level-3 test suite —DGEMM17,496 computational calls, 0 failures (DSYMM/DTRMM/DTRSM/DSYRK/DSYR2K all PASSED) — and HPL residual-PASSED.Notes
Zicbopsoftware-prefetch), not the kernel schedule.develop, or drop the U74-specific assembly and keep only the portable C kernel if preferred.