Skip to content

ARM64: Cortex-A72 DGEMM 6×8 microkernel and blocking - #5970

Open
hugomeiland wants to merge 4 commits into
OpenMathLib:developfrom
hugomeiland:cortexa72-dgemm-6x8
Open

ARM64: Cortex-A72 DGEMM 6×8 microkernel and blocking#5970
hugomeiland wants to merge 4 commits into
OpenMathLib:developfrom
hugomeiland:cortexa72-dgemm-6x8

Conversation

@hugomeiland

Copy link
Copy Markdown

Summary

TARGET=CORTEXA72 / DYNAMIC_ARCH MIDR 0xd08 currently aliases Cortex-A57 (KERNEL.CORTEXA57dgemm_kernel_8x4.S, shared param.h including the NUM_CORES>8 “server” DGEMM_Q=512 path). That blocking is a poor fit for A72’s ~2 MiB L2 shared by a 4-core cluster (e.g. AWS Graviton1).

This PR gives CORTEXA72 a dedicated DGEMM 6×8 path (BLIS-like MR×NR) plus measured panel blocking P=120 / Q=240 / R=4096 (R=768 when NUM_CORES≤2).

Measured HPL (AWS a1.metal, 16×1 MPI, grid 4×4, N=52000)

Backend Best NB GFLOPS
BLIS cortexa57 (archived) 192 ~107.4
This PR: 6×8 + P120/Q240/R4096 192 103.40
Prior 6×8 + P168/Q128 192 ~100
Custom 8×4 A72 blocking 192 ~90
Stock OpenBLAS (A57 alias) 128 ~95

Residuals PASSED on all HPL runs used for the table.

Explored and not shipped as default:

  • 8×8 NEON ukernel: assembles and is numerically OK, but spills 32 C accumulators → ~58 GFLOPS HPL on the same box.
  • k/4 + free-v7 pipeline variant of 6×8: slight single-core gain, but hurt multicore HPL (~89 @ N=20k) → rejected.

What changed

  • kernel/arm64/dgemm_kernel_6x8_cortexa72.S — MR=6 NR=8; C in v8v31; A v0v2, B v3v6
  • gemm_{t,n}copy_6_panel.c — contiguous MR=6 Goto packs (required; stock generic/gemm_*copy_6.c is a 4+2 clone and yields wrong residuals with a 6-wide ukernel)
  • trsm_kernel_*_UNROLLM6.c — stock TRSM assumes power-of-2 UNROLL_M via GEMM_UNROLL_M_SHIFT / bit masks; with M=6 that overruns packed buffers (free(): invalid next size in HPL / cblas_dtrsm). Same idea as loongarch64 *_UNROLLN6.c
  • KERNEL.CORTEXA72 — wire DGEMM/pack/TRSM; DTRMM falls back to generic/trmmkernel_2x2.c (no 6×8 TRMM yet); NR=8 reuses dgemm_{n,t}copy_8.S
  • param.h — CORTEXA72 split out of the A57 shared block; DGEMM_DEFAULT_UNROLL_{M,N}=6,8, P=120, Q=240, R=4096 / 768
  • Makefile.system / cmake/arch.cmake — add CORTEXA72 to DYNAMIC_CORE
  • driver/others/dynamic_arm64.cextern gotoblas_CORTEXA72 instead of #define … gotoblas_CORTEXA57

SGEMM/CGEMM/ZGEMM remain on the A57 kernels for now.

Test plan

  • make TARGET=CORTEXA72 USE_THREAD=0 NO_FORTRAN=1 NO_LAPACK=1 libs smoke build on arm64 (Apple host) — 6×8 .S, panel packers, and UNROLLM6 TRSM objects compile and link into libopenblas_cortexa72-*.a
  • HPL residual checks on a1.metal with the winning config (see table)
  • Single-core DGEMM residual checks for 6×8 after panel packers
  • OpenBLAS CI (DYNAMIC_ARCH / arm64) once this PR is opened
  • Optional: TARGET=CORTEXA72 cblas/utest on an A72 host

Notes for reviewers

Tuning context from the arm-benchmarks research tree (private); happy to paste more kernel-cycle / NB sweeps if useful. Prefetch distances kept at the proven A57 values (A_PRE=2560 B_PRE=448 C_PRE=128); schedule deviations measured on A72 mostly lost.

Made with Cursor

hugomeiland and others added 2 commits August 9, 2026 10:31
TARGET=CORTEXA72 previously reused the A57 8x4 path. Add a dedicated
6x8 NEON ukernel, contiguous MR=6 panel packers (stock gemm_*copy_6 is
4+2), and UNROLL_M=6-aware TRSM kernels so HPL/dtrsm does not corrupt
the heap. DTRMM falls back to generic 2x2 until a matching kernel exists.

Co-authored-by: Cursor <cursoragent@cursor.com>
Give CORTEXA72 its own param.h block (UNROLL 6x8, P=120 Q=240, R=4096
shared-L2 / R=768 single-core), add it to DYNAMIC_CORE, and stop
aliasing gotoblas_CORTEXA72 to A57 so DYNAMIC_ARCH can select the new
kernels on MIDR 0xd08.

Co-authored-by: Cursor <cursoragent@cursor.com>
DYNAMIC_ARCH builds CORTEXA72 as a separate kernel and pull
dneg_tcopy from generic/neg_tcopy_$(DGEMM_UNROLL_M).c. Width 6 was
missing (only 1/2/4/8/16 existed), which broke the arm64 Graviton
Cirun and Azure DYNAMIC_ARM64 jobs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hugomeiland

Copy link
Copy Markdown
Author

Pushed a fix for the arm64 Graviton Cirun / DYNAMIC_ARCH build break:

DGEMM_UNROLL_M=6 made the build look for generic/neg_tcopy_6.c (dneg_tcopy_CORTEXA72), which did not exist (only 1/2/4/8/16). Added a contiguous MR=6 negating packer matching the Goto layout of neg_tcopy_4/_8, with m-remainder cascade 4→2→1 after m/6 panels.

Commit: 2b69faf

Darwin's DYNAMIC_CORE only builds ARMV8/NEOVERSEN1/ARMV9SME/VORTEXM4,
so an unconditional extern gotoblas_CORTEXA72 left Apple M builds with
an undefined symbol. Mirror the CORTEXA57 Darwin alias.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hugomeiland

Copy link
Copy Markdown
Author

Follow-up for the Apple M / Darwin DYNAMIC_ARCH linker failure (_gotoblas_CORTEXA72 undefined):

Darwin's DYNAMIC_CORE does not build CORTEXA72 (only ARMV8 / NEOVERSEN1 / ARMV9SME / VORTEXM4). Restored a Darwin #define gotoblas_CORTEXA72 gotoblas_ARMV8 and keep the real extern on Linux, matching how CORTEXA57 is handled.

Commit: after this push on cortexa72-dgemm-6x8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant