As discussed prior,
AMD has recently upstreamed preliminary gfx90c support with native ROCm & PyTorch + ROCm both available via theRock / ROCm 7.x. You can download and use them from here. Unfortunately, its conceptually identical to spoofing HSA_OVERRIDE_GFX_VERSION=9.0.0. gfx900 is vega10, gfx90c is Green Sardine.
The following milestones would thus be duplicating upstream work:
- Finish triage of the remaining librocsparse failure modes
- Port
COMPILER_HAS_TARGET_ID_gfx90c into the librocsparse build configuration
- Port
/usr/lib64/librocblas.so.4.4
- Port
/usr/lib64/librocsolver.so.0.4
- Resolve the remaining rocm-
examples failure classes
- Resolve
applications_monte_carlo_pi
- Resolve
hip_opengl_interop
- Compile ROCm 6.4.4 from source / get TheRock working
- Add automatic GPU detection and target selection to the
rocm-examples build system
- Benchmark real-world HIP workloads natively on
gfx90c vs gfx900 spoofing
- Compare source-built
gfx90c results against the Fedora-packaged binaries
- Benchmark PyTorch on the recovered stack
- Rebuild PyTorch against the patched ROCm stack
- Port PyTorch Docker images to
gfx90c
- Repeat the recovery process on
gfx902
- Evaluate whether the same methodology generalises across additional unsupported AMD GPUs
- Test bleeding-edge ROCm 7.1
- Investigate LLVM support boundaries for GPUs outside the currently supported target set
- Build the automated CI/CD pipeline for ROCm / PyTorch / Dockerised PyTorch builds
The upstream change does not erase the educational work from the first 14 articles. Taken together, they form a progression through ROCm installation and compatibility, HIP execution, GPU architecture, execution and memory hierarchy, AMDGPU assembly, LLVM IR and register allocation, target triples and CMake, ELF relocation and PIC/PIE, benchmark methodology, experimental failure analysis, automation, and source-built ROCm components. The final articles then connect those concepts to an actual 200k+ test library validation. These articles document the mechanisms, experiments and debugging decisions required to understand the stack from first principles.
However, the crux of the project remained unsolved by upstream:
- Generate
TensileLibrary.dat for gfx90c
I haven’t had the time to write an article about my work the past 15 days yet, so I’m providing a high level explanation here.
Tensile is the benchmark-driven kernel-library generator behind rocBLAS. Rather than providing one fixed kernel for a GEMM operation, it generates a large set of candidate solutions by varying parameters such as MacroTile, ThreadTile, WorkGroup, DepthU, VectorWidth, prefetching, GlobalSplitU and WorkGroupMapping. Those candidates are compiled for the target ISA, benchmarked on the hardware, and reduced into library logic which records which solution should be selected for each problem shape. TensileCreateLibrary then packages that logic and the corresponding code objects into the final library artifact.
Adding gfx90c to the compiler and runtime establishes that the architecture can execute native kernels. The shipped gfx900 Tensile library contains 28 family/pattern libraries and 3,066 highly specialized kernels, with extensive shape coverage and per-kernel feature configurations. This is missing for gfx90c, gfx902 and all of the other GPUs this project has targeted from day 1. The goal has been: match the shipped gfx900 footprint of 3,066 kernels across 28 families and four contraction patterns with native gfx90c equivalents.
For tackling this, I initially designed a prototype shell script (ended up being ~4k lines of AI slop) to run experiments and figure out the appropriate approach from 1 September through 12 September. It started as an experimental benchmark runner and progressively I implemented all the necessary features. The final script evolved into a full experimental harness for exploring the Tensile search space and for identifying the failure modes that a persistent tuning system would have to handle. This helped me establish several constraints for the new design.
I derived the gfx90c realistic and theoretical ceilings (based on ALU/DDR limitations) from the hardware specifications:
- 7 CUs
- 448 FP32 lanes
- Wave64 execution
- 280 maximum wavefront slots
- 64 KiB LDS per CU
- no MFMA.
The approximate theoretical ceilings are:
- 1613 GFLOPS for FP32, BF16, Complex FP32, and the HPA families
- 3226 GFLOPS for packed FP16
- 101 GFLOPS for FP64 and Complex FP64
- 1613 GOPS for INT8.
These are the baseline for my TensileLibrary.dat. There was approximately ±23–31% fluctuation across ALU-bound runs due to the APU’s thermal and clock variation. This means raw GFLOPS are unsuitable for comparing experiments. I designed a normalised benchmarking system which helps isolate tuning gains from thermal/DVFS noise. All code/logs/benchmark results from the initial prototype and its supporting experiments are available on GitHub.
After finalising the design on 13 September, I built the v0 Python implementation. The flow is divided into manifest execution, candidate generation, benchmark execution, adaptive repetition, analytics and registry management, library synthesis, and final acceptance reporting. Semantic selectors allow the runner to target a specific family, contraction pattern, shape or stage without modifying the underlying manifest.
The kernel tuning search space here is too big to explore blindly.
Adding an agent makes sense, hear me out :D
The LLM acts as a hypothesis engine, consuming the experiment database, current winners, failed configurations, profiling/telemetry and hardware constraints. It proposes a small set of specific parameter mutations for the next generation of candidates. This collapses the enormous Tensile search space into evidence-driven local neighbourhoods rather than enumerating combinations blindly. I believe its a suitable constrained application for agentic AI being inherently neurosymbolic. The neural component supplies learned, probabilistic hypotheses which can be cheaply evaluated and iterated against a symbolic system which supplies explicit parameter semantics, validity rules, compilation, correctness tests, benchmarking, deduplication and selection. Every iteration shrinks the viable search frontier instead of letting either Tensile or the agent wander through the full combinatorial space.
The implemented agentic loop is as follows:
- observe spoof & native performance (Suite & Tensile)
- propose mutations (Agent)
- validate (Tensile)
- benchmark (Suite + Tensile)
- retain winners (Suite)
- mutate the winners (Agent + Suite)
- Repeat
I ran it overnight from 14 Sept to 15th Sept. Here are the highlights from crunching results vs the 13 Sep initial Python prototype:
- Ran for 7.7 hours across five iterations
- 2,400 raw measurements and accumulated 1,652 registry records
- Candidate mutation was functional
- HH family improved from 1127.8 to 1732.0 GFLOPS (+53.6%)
- +89%-650% coverage across families, 185% increase in explored cells overall
- Already at 91.3% of theoretical limit on DD
- Crossed original realistic limit for HH_HPA, 73.4% of theoretical
Ooh also I designed a very pretty auto-updating IBM Carbon design system style dashboard for the tuning suite progress report, you can view the latest results in this image:
I plan to create a separate repository for the new agentic kernel-tuning suite so that its independent from the code I use for plotting graphs etc for my articles.
Completed milestones:
- 14 article series on ROCm, HIP, GPU architecture, GPU execution and memory hierarchy, AMDGPU assembly, LLVM IR and register allocation, ELF relocation and PIC/PIE, etc.
- Study Tensile, experiment with kernel tuning
- Design agentic Python kernel tuning suite
- First autonomous benchmark run.
Ongoing
- Generate
TensileLibrary.dat for gfx90c
Next milestones:
- Build out the agentic search engine: Improve candidate generation, mutation strategies, evidence extraction, scheduling, novelty accounting, convergence detection and persistent experiment memory
- Maximise
gfx90c against its hardware ceilings: Expand beyond the initial search regions, explore the remaining Tensile parameter space systematically, and drive each workload family toward the theoretical ceilings established from the hardware and measurements.
- Validate the methodology on
gfx902: Apply the complete pipeline to an unsupported architecture and determine which parts of the system can be derived from hardware constraints, ISA capabilities and Tensile semantics without requiring an existing upstream target.
- Refactor so that a new AMD GPU can be described through explicit hardware constraints that the suite can natively reason against.
- Package the workflow so any tester can invoke it easily with an API key and generate a tuned
TensileLibrary.dat for their hardware. (Jefferson will be able to help with this)
- Attempt to upstream the resulting
gfx90c configuration and tuning data, and investigate a corresponding upstream path for gfx902.
- Benchmark the resulting custom
TensileLibrary.dat against the available upstream gfx90c support path under reproducible conditions.
- Generalise the tuning engine to additional AMD architectures after
gfx90c convergence.
- Publish the suite under an open license so the broader community can extend the approach to additional AMD architectures.
- Keep publishing informative educational articles about various complex topics I come across during this effort, such as GPU kernels, Tensile, kernel tuning, GPU features like MFMA, neurosymbolic agentic implementations, agentic approaches to kernel tuning, picking winner kernels, geometry of GPU GEMMs, tracing architectural compatibility through Tensile, YAML to TensileLibrary.dat, postmortems for any runs, interesting failure modes, deriving theoretical & realistic peaks for any GPU, etc.
