Verification status (WIP)
This wiki is work in progress. Pages expanded from Discord (
#vllm-rdna,#llamacpp,#general) and fork release notes mix solid facts, fork-source claims, and community benchmarks that have not been independently reproduced in this repo.
Legend
| Status | Meaning |
|---|---|
| Solid | Hardware fact, upstream/ROCm docs, or fork source code confirms the behavior exists. |
| Fork-source | Confirmed in opengfx1030/vllm-rdna rdna_extras commits (and historical personal-fork SHAs where noted); may still need a fresh -extras image build. |
| Community | Reported in Discord or by a fork author; plausible but not wiki-verified on our hardware. |
| Needs verify | Should be re-tested before treating as guidance (image lag, single-host bench, or opinion). |
| Opinion | Subjective quality/speed trade-off from community testing. |
Global gaps (read first)
| Topic | Status | Notes |
|---|---|---|
Docker -extras images vs extras HEAD | Needs verify | Hub v0.31.0-extras (11 Oct 2026) bakes from opengfx1030/vllm-rdna @ rdna_extras (commit pin in docker-bake.hcl). HEAD after that pin still wants clone + host venv. Older v0.27.1-extras* targets named the removed personal BlivionIaG/vllm clone. |
| Performance numbers (tok/s, × speedups) | Community | Unless marked Solid, treat throughput tables as single-host snapshots. |
| Env-var recipes | Community | Common stacks from Discord; A/B on your topology. |
| ROCm 7.2.1+ multi-card RCCL bug | Community | #vllm-rdna (fork author): stay on 7.2.0 or 7.14.0 |
vLLM (vllm/)
Flash-Next narrative lives in flash-next.md, serve commands in flash-next-serve.md, and
Flash-Next failure modes in troubleshooting/vllm-flash-next.md. Rows below still name the page
they were written against (recipes.md, overview.md, configuration.md).
onboarding.md
| Statement | Status | Verify how |
|---|---|---|
Hub v0.31.0-extras is day-to-day Docker | Community | #announcements 11 Oct 2026; Hub tags v0.31.0-extras / v0.31.0-extras-rocm7.14.0 dated 11 Oct |
rdna_extras = upstream vLLM 0.31.0 + RDNA port | Fork-source | #announcements 11 Oct; org default branch rdna_extras |
| Path A 27B AWQ docker run | Community | Same announcement + prior #vllm-rdna day-to-day model pick |
Path B serve_rdna.sh + PLE=int4 Flash-Next | Community | #announcements + #vllm-rdna Oct 11 launcher example |
P2P=0 for hosts without PCIe P2P | Fork-source | tools/rdna/serve_rdna.sh + #vllm-rdna Oct 11 |
running.md
| Statement | Status | Verify how |
|---|---|---|
| Image tag matrix | Needs verify | Docker Hub tags |
Hub v0.31.0-extras | Fork-source | #announcements 11 Oct 2026 + Hub push 11 Oct 15:14 UTC (same digest as v0.31.0-extras-rocm7.14.0). ROCm 7.14 / torch 2.12 / Triton 3.7.1. Flash-Next MTP-2 (BF16 + INT4-disk) served. |
Hub v0.28.0-extras | Historical | Sep 18 test tag. Prefer v0.31.0-extras. Source now rdna_extra/v0.28.0. |
PYTORCH_ROCM_ARCH list | Solid | vllm-rdna-docker build |
Prefer --dtype float16 | Solid | RDNA2 BF16 limitation |
Default smoke model cyankiwi/Qwen3.8-27B-AWQ-INT4 | Community | #vllm-rdna Sep 13 2026 day-to-day 27B pick |
recipes.md
| Statement | Status | Verify how |
|---|---|---|
Three stacks (Hub -extras / recipe GHCR / Flash-Next fork) | Community | #vllm-rdna Sep 2026; public READMEs |
Recipe image 0.27.1-rocm7.2.3-gfx1030 + preset: | Community | Recipe containers/README.md |
| 1× V620: prefer MoE; Flash-Next not 1-card | Community | #vllm-rdna Sep 13 2026 |
| Gemma 4 unfinished on gfx1030 vLLM | Needs verify | #vllm-rdna Sep 13 — single-thread reports |
Flash-Next weights wtdcode + primitive-ai PLE | Community | #vllm-rdna Aug 30 / Sep 13 |
| Sep 18 4× serve deltas (7 GiB KV, leftover workers, V2 blocked on Qwen4Exp) | Community | #vllm-rdna Sep 18 host-venv paste |
MTP draft experts W4A16 (quant_mtp_experts.py) | Community | #vllm-rdna Sep 18 + overlay README; 3× PP3 tok/s Needs verify |
PP3 b33f9b6 graph KeyError / 1.5e9 KV | Needs verify | #vllm-rdna Sep 19 — public overlay patches; not a 7 GiB 4× cap |
PP3 HEAD flashnext-pp3 / flashnext-pp3-mtp2 (#96 / #95) | Fork-source / Community | #96 merged 13 Oct. #general 13 Oct author-host 16k/1k: MTP-0 1848/38 c=1, 771/7.7 c=8; MTP-2 1866/70.9 c=1, 1071/10.1 c=4. Hub v0.31.0-extras lags |
| TP4 cyankiwi AWQ env block | Community | #vllm-rdna Aug 31 bench paste (paths sanitized) |
| Flash-Next 4× PIECEWISE serve (Sep 17) | Community | #vllm-rdna — GDN sanitizer 388a61b6f; ~3331/73 @ 16k/1k c=8 |
Flash-Next 4× #17 FULL_DECODE_ONLY (Sep 23) | Community | Merged opengfx1030/vllm-rdna#17; second host confirmed 16k PP |
| Flash-Next vLLM PP4 ~1950 PP / decode collapse | Needs verify | #vllm-rdna Sep 19 — same host returned to TP4 (~1550 PP; ~30–35 t/s MTP-2) |
QSA live-context #15 ~1.8–2.0k PP / ~54–73 TG (TP4 PP1) | Needs verify | #vllm-rdna Sep 20 author-host table + merged opengfx1030/vllm-rdna#15. Sep 21: other 4× hosts on #15 only got ~1.1–1.45k PP. The unpublished recovery is #17 (merged 23 Sep) |
| Recipe decode ~40–49 vs ~27 without TunableOp | Community | Recipe container README / troubleshooting |
configuration.md
| Statement | Status | Verify how |
|---|---|---|
| Recommended env block | Community | Discord default stack |
| Custom AR disable vs PIX force | Community | #vllm-rdna Aug 2026 — pick by P2P topology |
VLLM_USE_V2_MODEL_RUNNER +17% vs V1 | Community | #vllm-rdna bench comment (-extras) |
VLLM_USE_V2_MODEL_RUNNER=0 for Flash-Next long prompts | Community | #vllm-rdna Sep 2026 — stalls/timeouts fixed; ~68 t/s dense INT8 |
VLLM_USE_AOT_COMPILE=0 / VLLM_DISABLE_COMPILE_CACHE=1 | Fork-source | Multi-GPU TP workaround |
CUDA graphs preferred over --enforce-eager | Fork-source | TP allow_in_graph fix; needs verify on image |
ROCM_ATTN Triton hang | Community | Hours-long compile; use RDNA_ATTN |
| Throughput table (277/93/331 tok/s) | Community | Fork author, TP4, 4× V620 |
| Docker Compose ~24 t/s TP2 | Community | #vllm-rdna report |
| Cache sizes ~3 GB / ~700 MB | Community | Order-of-magnitude |
SAFETENSORS_FAST_GPU=1 | Community | #vllm-rdna Sep 2026 + AMD optimization docs |
Flash-Next TP4 GPU_MAX_HW_QUEUES=4 (#107) | Fork-source | #107 16 Oct; launcher default stays 2. Neutral with fork kernels off |
VLLM_CACHE_ROOT + compile cache on | Community | Flash-Next / recipe startups (~5 vs ~10 min) |
| Flash-Next ~580–700 PP / ~50–53 decode | Community | Early Sep recipe; superseded by prefill campaign below |
| Flash-Next ~1000–1400 PP / ~53–100+ decode | Community | #vllm-rdna Sep 4–7 + Sep 13 suite + Flash-Next RESULTS.md; host-dependent |
| Flash-Next container ~40 t/s vs llama.cpp ~18–19 | Community | #vllm-rdna Sep 2026 same-host comparison |
| Intel AutoRound Flash-Next ~962 PP / ~41–56 decode | Community | #vllm-rdna Sep 10–11 + draft opengfx1030/vllm-rdna#5; 4× V620; dirty PR |
--max-num-batched-tokens 4096 fixes 128k PP cliff | Community | #vllm-rdna Sep 11 — 375 → ~950 PP; A/B on your scheduler |
| Intel AutoRound ~75 GB / BF16 PLE ~95 GiB | Community | Publisher checkpoint + PR validation notes |
| Quantized group-16 INT4 PLE incoherent | Needs verify | Historical #vllm-rdna sidecar path — not #49. Known-good AutoRound benches used BF16 PLE |
| GDN varlen second-seq output bug | Fork-source | gdn_prefill_o_rdna2.cu in draft PR #5; concurrent prefill |
| Recipe ports / Hybrid gfx10 opt-in | Needs verify | Draft opengfx1030/vllm-rdna#6 — RDNA2 stays auto default |
| vLLM 0.29 defaults V2 runner | Community | #vllm-rdna Sep 11 (when Hub day-to-day was still 0.27.1); Sep 18 rdna_extra/v0.29.0 rebase. Day-to-day Hub is now v0.31.0-extras — re-A/B V2 on that line. |
No MTP on Hub v0.28.0-extras | Community | #vllm-rdna Sep 18 — MTP aimed at 0.29 / Qwen4Exp |
| MTP can block int4 PLE fused decode | Community | #vllm-rdna Sep 18 — A/B MTP off on no-P2P hosts |
MTP-2 ~0.73–0.85× vs MTP-0 after 700753d9 | Community | Author-host 4× W4A16 Flash-Next; #vllm-rdna Sep 25 “boots, still a loss”. 30632b2 (27 Sep) lifts c=8 but stays ~0.87× at 16k |
| ROCR 1.21 idle CPU spin on 7.14 | Community | TheRock#7051; patch in Flash-Next fork ROCR-CPU-FIX.md |
| EXL3 / Quark on gfx1030 | Needs verify | #announcements 11 Oct: EXL3 listed on v0.31.0 extras / RECIPE=27b-exl3 — still experimental. Sep 29–1 Oct: mul1 + v0.30 loader on rdna_extras HEAD GPU-run (215/215; 27B TP=4). Oct 2: mul1-tested only; 3inst untested. Quark still not Hub |
| Intel AutoRound W4A16 Flash-Next | Community | #vllm-rdna Sep 10–11 — draft PR #5; not in Hub -extras |
Official extras source is opengfx1030/vllm-rdna rdna_extras | Solid | #vllm-rdna Sep 2026 move; default branch rdna_extras; PRs go to the org. Current Hub extras bakes (v0.28 / v0.31) clone this repo |
Draft PRs #5 / #6 on org extras | Needs verify | #5 AutoRound closed unmerged (14 Sep); #6 recipe ports still open/draft — do not treat as released |
opengfx1030/vllm-rdna ready as published Docker | Fork-source | Bake repo opengfx1030/vllm-rdna-docker; Hub v0.31.0-extras published 11 Oct 2026 from org rdna_extras |
Flash-Next source is rdna_extras | Fork-source | #vllm-rdna Sep 14–15 cherry-picks + #announcements 11 Oct Hub bake served Flash-Next MTP-2. leapdragon tree is legacy container / docs |
| Org 0.28 rebase / Flash-Next cherry-picks | Historical | #vllm-rdna Sep 3–15 era; superseded by v0.31.0 release row |
| Flash-Next KV ~24 GiB ≈ 300k tokens | Community | #vllm-rdna Sep 14 — 4× 32 GB still tight. Sep 14 “128 GB RAM offload not enough” is the pre-#24 path |
| Flash-Next concurrency splits TG / tanks PP | Community | #vllm-rdna Sep 14 — ~70–80 t/s total vs ~70 single; PP ~500 |
| 2× V620 Flash-Next: llama.cpp not vLLM | Community | #vllm-rdna Sep 14 — RAM offload |
| vLLM 3-card → PP=3 (even TP) | Community | #vllm-rdna Sep 14 |
| Flash-Next V2=0: 100% util / ~40 W stall | Community | #vllm-rdna Sep 15 — TP4 leapdragon container; MoE wants V1 before 0.29 |
Flash-Next PP3: leapdragon OK, rdna_extras garbage | Needs verify | #vllm-rdna Sep 15–16 — historical pins. HEAD recipes #96 supersede leapdragon-only advice |
Flash-Next PP3 HEAD recipes #96 / #95 | Fork-source / Community | #96 merged 13 Oct: flashnext-pp3 / flashnext-pp3-mtp2. #general 13 Oct author-host table. Not Hub |
Flash-Next PP3 b33f9b6 small-prefill KeyError | Needs verify | #vllm-rdna Sep 19 — capture sizes [1,2,4,8]; NaN+concurrent gone on that pin only |
| Flash-Next FULL graphs corrupt; PIECEWISE holds | Needs verify | #vllm-rdna Sep 16–17 — TP rdna_extras; --max-num-seqs 4–6 + batched 2048 |
| Flash-Next 4× PIECEWISE serve ~3331 PP / ~73 TG | Community | #vllm-rdna Sep 17 — 16k/1k c=8; GDN sanitizer 388a61b6f; not Hub |
VLLM_RDNA_DENSE_GEMV=1 avoids wvSplitK fault | Community | #vllm-rdna Sep 17 — wvSplitK_hf_sml_* after profile; multi-GB coredumps |
| Flash-Next hybrid KV log ~2.5× high | Needs verify | #vllm-rdna Sep 17 — livelock between real pool and max-model-len |
VLLM_USE_BREAKABLE_CUDAGRAPH=1 disables compile | Community | #vllm-rdna Sep 17 — ~39 vs ~55 t/s A/B |
Prefix cache 0% before e45dd5cb | Fork-source | #vllm-rdna Sep 17 + opengfx1030/vllm-rdna commit; TTFT 12.2s → 0.3s |
vllm#55506 port 741e5bc3 | Fork-source | V2 mamba spec-decode tables; did not fix V1 0% / !!! |
| 3× leapdragon + org MoE HIP overlay | Community | #vllm-rdna Sep 17–18 — alanoo81/flashnext-v620-pp3; ~1078–2493 PP / ~57–63 TG |
| DeepSeek-V4 Flash INT4 + gfx1030 tree | Needs verify | #vllm-rdna Sep 17 — yiminyuan HF + gfx1030/v0.28.0; intended TP=4 PP=2. Oct 10: extras DSV4 expected after mixed-stall / CUDA-graph-range fix |
| No Q3 on vLLM Flash-Next | Community | #vllm-rdna Sep 17 — stay W4A16 / AWQ; EXL3 3bpw experimental |
--no-enable-prefix-caching KV offload non-starter | Community | #vllm-rdna Sep 16 — QSA/RAM weight packs; still prefer prefix cache. Native KV overflow is #24, not this |
| Intel AutoRound 32–128k ~1296–1393 PP / ~56–73 TG | Community | #vllm-rdna Sep 16 — 4× V620; pin 4c67bf6…; int8 decode-shadow quality |
QSA live-context bound merged on rdna_extras | Fork-source | opengfx1030/vllm-rdna#15 merged 20 Sep 2026; Hub -extras lags |
QSA #15 combined-stack ~1830–2040 PP / ~54–73 TG | Needs verify | PR recorded table + #vllm-rdna Sep 20 author-host; not reproduced on two other 4× hosts Sep 21 (~1.07–1.45k PP). The unpublished recovery is #17 |
QSA #15 public-merge prefill ~1.1–1.45k PP | Community | #vllm-rdna Sep 21 — two 4× hosts after #15 only |
V620 fast stack #17 merged | Fork-source | opengfx1030/vllm-rdna#17 merged 23 Sep 2026; resident W4A16 + Mamba #55450 + FULL_DECODE_ONLY |
#17 16k PP ~1958 / ~1983; second host ~1984 / ~1985 | Community | PR-head table + #vllm-rdna Sep 23 confirm after TunableOp regen |
| TunableOp rows locked to rocBLAS hash | Fork-source | rdna_extras/tunableop README; named profiles rocm7.14-rocblas5.5 (783) and rocm10-rocblas5.6 (70, thin); mismatch ~25% PP drop / first GEMM abort |
#20 PIECEWISE capture (FULL→piecewise redirect) | Fork-source / Community | Merged opengfx1030/vllm-rdna#20 (24 Sep); decode ~26–34 t/s while redirect was on |
Keep-FULL FULL_AND_PIECEWISE (6c26c78d) | Fork-source | rocm_full_executes_as_piecewise → False on rdna_extras HEAD; #vllm-rdna Sep 24; docs V620-FULL-AND-PIECEWISE.md |
#19 unquantized MTP drafter load | Fork-source | Merged opengfx1030/vllm-rdna#19 (24 Sep) |
MTP draft unbreak 700753d9 (amdsmi + local-argmax) | Fork-source | rdna_extras HEAD 25 Sep; MTP-2 boots, ~0.73–0.85× vs MTP-0 on author 4× host |
rdna_extra/v0.30.0 Qwen4 rebase | Needs verify | Branch exists (#vllm-rdna Sep 25); tip f87be4dc MoE capture-buffer fix. Not the day-to-day Flash-Next line |
v0.31.0 extras release (#47 closed) | Fork-source | #announcements 11 Oct 2026 + Hub blivioniag/vllm-rdna:v0.31.0-extras (same digest as v0.31.0-extras-rocm7.14.0). rdna_extras = upstream vLLM 0.31.0 + RDNA port. Old 0.28: rdna_extra/v0.28.0. Author 4× V620 TP=4 Flash-Next AWQ ~2000 PP / 45 (MTP-0) / 69 (MTP-2) TG / ~205 agg @ 8 users. PP3 recipes (#96) are HEAD-only — not in the 11 Oct Hub bake |
| Swift 1.5 Flash-Next AWQ / AutoRound | Community | #vllm-rdna Sep 25–27 — public HF packs; 3× ~1300 PP / ~45 TG Needs verify; AutoRound is 50-iter light, ~5% faster / more quantized; Sep 27 looping on one host |
GLM-5.3-Flash org #2 vs community 8× | Needs verify | #2 side branch runtime-unverified; #vllm-rdna Sep 30 local 0.30 fork ~32 t/s / 320–500 PP on 8× |
| Host-venv TP=2 cyankiwi 27B AWQ | Needs verify | #vllm-rdna Sep 26 — one source-build host; 29.5 t/s single / 46.9 t/s ×4 |
| Host-venv TP=2 Qwen3.8-27B fp16 + MTP ~40–48 t/s | Needs verify | #vllm-rdna Oct 6 — 2× V620; Swift 1.5 27B ~48 / ~70 two-instance |
#18 GPTQ BLOCK_KN_SIZE 128→256 | Needs verify | Open PR; hold for M>32…2048 sweep (#vllm-rdna Sep 24) |
#22 two-shot rdna_ar vs force custom AR | Fork-source | Merged 27 Sep; library default still off; A/B topology |
#23 PCIe P2P KV disagg prefill/decode | Needs verify | Open PR; local PLX-switch split — not a recipe |
Flash-Next 6× PP=3+TP=2 / PP=6 | Needs verify | #vllm-rdna Sep 24 — min 3 cards; no published 6× table |
| Qwen3.8-27B fp16 stock 0.29.0 TP4 W6800X Duo | Community | #benchmarks Sep 24 — 1k–255k ctx, no MTP; ~1394→560 PP / ~29→19 TG |
| Qwen3.8-27B fp16 vLLM 0.30.0 TP4 W6800X Duo ALU 0.9.4 MTP-2 | Community | #benchmarks Oct 10 — 512 gen; 1k ~1398 PP / ~66 TG; 255k ~546 / ~32; c=3 1k ~165 agg. Not a V620 recipe |
Flash-Next vision pixel cap (max_pixels=1605632) | Fork-source | rdna_extras tools/rdna/serve_rdna.sh vision-cap (old serve_gfx1030_flashnext.sh is an alias); --limit-mm-per-prompt alone is not enough |
| Flash-Next vision ~15–20 s/image + runtime OOM | Community | #vllm-rdna / #general Sep 21 — works with launcher flags; no reserved vision pool; PP3 especially tight |
Flash-Next FULL_AND_PIECEWISE executes as PIECEWISE on ROCm | Fork-source (historical) | Launcher comment 18 Sep 2026 + #20-era redirect. Superseded on HEAD by 6c26c78d (rocm_full_executes_as_piecewise → False) — prefer FULL_AND_PIECEWISE again |
| QSA fused MTP-3 draft decode no gain | Needs verify | #vllm-rdna Sep 20 — one host; prefill held ~2k |
INT8 decode shadows excluded from #15 | Fork-source | PR body; Discord ~42 → ~55 t/s without MTP is Needs verify / quality risk |
| vllm-rdna-qa playbook | Needs verify | #vllm-rdna Sep 17 — public repo; maintainer QA only |
| Flash-Next PP3 + MTP on 3× V620 | Needs verify | #vllm-rdna Sep 15 — leapdragon-era; local int8 patches. Prefer HEAD #96 recipes |
Upstream MTP under PP (vllm#46994) | Fork-source / Needs verify | Merged Sep 2026; absent from Hub 0.27.1 -extras. Re-check on Hub v0.31.0-extras / HEAD (PP3 still Needs verify) |
RDNA2W4A16MoEExperts raises concurrency | Community | #vllm-rdna Sep 15 — confirm kernel in logs |
| Native RDNA2 FA WIP on Flash-Next HEAD | Community | #vllm-rdna Sep 15 — A/B VLLM_USE_RDNA2_FA=0; Hub -extras still uses 1 |
Flash-Next QSA bypasses FA / --attention-backend | Community / Fork-source | #vllm-rdna 13 Oct — QSA Triton owner; FA not dense-only; full HIP QSA/PLE still open (#64 backlog). Knobs VLLM_FA_RDNA2_GQA_MODE / VLLM_FA_RDNA2_SPLIT_DECODE in rdna_attn.py |
rdna_ar not a recommended default | Community | #vllm-rdna Sep 15–24 — prefer simpler RDNA_AR over force custom AR; #22 merged 27 Sep (still default-off); one host crash on force custom AR |
| hippihx kernel zoo | Needs verify | #hippihx Sep 16 — public repo; not in published images |
| Upstream vLLM KV CPU/SSD offload ~1 t/s | Community | #vllm-rdna Sep 15 broken baseline. Superseded as the gfx1030 path by merged #24 (next row) |
RAM KV offload #24 (~0.4 s / ~190k; ~2.1 s reload) | Fork-source / Needs verify | Merged opengfx1030/vllm-rdna#24 (27 Sep); #vllm-rdna Sep 26–27. SSD tier untested. 8 GiB ≈ ~100k Needs verify. In Hub v0.31.0-extras |
--kv-offloading-size RAM parking | Community / Needs verify | #vllm-rdna Oct 7 — community --kv-offloading-size 64 (GiB total across TP). Storage tier still untested |
| KV offload restore token-repeat loops | Community / Needs verify | #vllm-rdna 25–26 Oct — loops survived MTP off; 4 h+ clean after KV offload disabled |
MTP draft_sample_method=probabilistic when temp>0 | Community | #vllm-rdna Oct 8 — required on current rdna_extras MTP-2; greedy/local-argmax is the default |
PLE_OFFLOAD_PREFAULT=0 low-RAM | Community / Needs verify | #vllm-rdna Oct 8 — ~22–28% slower vs prefault=1 (prose). Unprefixed overlay name; ~1:2 RAM:VRAM OK; 32 GiB + 2× V620 tight |
Skinny HIP MoE 9–16 rows #26 | Fork-source / Community | Merged 27 Sep; VLLM_ROCM_MOE_SKINNY_MAX_M=16; +11% concurrent regular-text. No single-chat claim |
MTP draft graph 30632b2 | Fork-source | 27 Sep; +18% / +5% at c=8 1k / 16k; still ~0.87× vs MTP-0 at 16k |
| Mixed prefill+decode ~1–6 t/s | Community | #vllm-rdna Sep 27–29 — decode-only 2 streams OK; mixed cliff. Static --prefill-schedule-interval / --long-prefill-token-threshold trade PP for mixed decode. Sep 29: unset those flags for max single-stream PP |
Adaptive prefill #36 (VLLM_RDNA_DYNAMIC_PREFILL=1) | Fork-source / Needs verify | Merged opengfx1030/vllm-rdna#36 (5 Oct). Default off. #vllm-rdna Oct 4–5: ~25 t/s burst at 3 decode + 1 prefill; author Oct 5 recovered below 500 → ~1100 mixed PP. Synthetic benches called a poor measure. In Hub v0.31.0-extras (opt-in) |
| wvSplitK vs stacked LLMM1 concurrency | Community / Needs verify | #vllm-rdna Oct 6 — 4× W6800X Qwen3.8-27B fp16 decode-only; C2 ~40 → ~56 t/s. Keep VLLM_RDNA_DENSE_GEMV=1 if the Sep 17 fault still hits |
| Qwen3.8-27B fp16 2× V620 TP=2 MTP ~40–48 t/s | Community / Needs verify | #vllm-rdna Oct 6 — no quant; Swift 1.5 27B tune ~48 single / ~70 two-instance. Not Flash-Next |
VLLM_RDNA_MOE_RESIDENT=1 missing tanks Flash-Next PP | Community | #vllm-rdna Sep 29 — latest extras ~2k → ~1450 PP with resident MoE off; 1 recovered ~2k |
EXL3 mul1 port #32 + HEAD loader | Fork-source / Needs verify | #vllm-rdna Sep 29–30 + rdna_extras 30 Sep–1 Oct — mul1 / K=1..8 GPU-run (215/215 tests; 27B 3.00 bpw TP=4 graphs). Author-host 74315f4 1k/512 ~18.9 / ~81.8 t/s; 16k/1k ~13.7 t/s / ~830 PP. Not Hub. Oct 2: HEAD mul1-tested only; 3inst untested |
| GLM-5.3-Flash AWQ 8× TP=8 ~32 t/s / 320–500 PP | Community / Needs verify | #vllm-rdna Sep 30 — wtdcode/GLM-5.3-Flash-AWQ-W4A16 on a local 0.30 fork; 262k + vision; org #2 still unverified side branch |
| W4A8 vs W4A16 | Needs verify | #vllm-rdna Sep 29 explore #9; Oct 1: in HEAD (W4A8=1) not much faster (context room). Oct 9: 0.31-line kernel called somewhat competitive vs W4A16 — not a recipe |
FA-RDNA2 #28 / split-decode #29 | Needs verify | #28 merged 28 Sep without gfx1030 GPU runs; #29 draft, stacked, also untested |
Two-shot rdna_ar #22 | Fork-source | Merged 27 Sep; library default still off; A/B topology |
vllm#57160 private pinned CPU KV tensors | Needs verify | #vllm-rdna Sep 20 — mainline ROCm; gfx1030 follow-through is merged #24, not this PR alone |
| LMCache RDNA Docker integration | Needs verify | #lmcache Sep 23–24: 0.5.6.dev wheel built; official kernels CDNA-only; hybrid Qwen HMA + LMCacheConnectorV1 not HMA-capable. Still no working compose |
| Upstream vLLM 0.28 / 0.30 gfx1030 support | Needs verify | Official 0.28–0.30 still omit Navi 21 (#general Sep 22: 0.30 + ROCm 10.0). Keep extras |
quantization.md
| Statement | Status | Verify how |
|---|---|---|
GPTQ → RDNA2W4A16LinearKernel | Fork-source | Startup logs |
AWQ dense → RDNA2W4A16LinearKernel | Fork-source | Commits 73eb04a/5ac31e4; needs verify on image |
| Older AWQ ~4–5 t/s (Triton) | Community | True on old images only |
| KV cache prefer float16 | Opinion | Community quality / agents |
int8_per_token_head vs fp8 | Community | Faster TG than fp8 in limited older #vllm-rdna testing; not the QSA path |
QSA / Flash-Next --kv-cache-dtype fp8 startup fail | Community | #vllm-rdna Sep 22 — QSA backends unquantized-only + AITER/CDNA gate |
| KVarN | Community | Skip for tool calling |
| MTP acceptance ~0.25 | Community | Model-dependent |
| MTP-2 hurts high concurrency (35B-A3B) | Community | TP4 #vllm-rdna 4-cell matrix |
| INT4 vdot2 fp16 dequant | Solid | ISA + fork code |
Qwen3.8-27B AWQ needs head_size=256 | Fork-source | Same as fa_rdna2 commit; confirm on image |
| GDN hybrid ~93/331 tok/s | Community | Fork author bench |
| EXL3 9B / Quark W4A16 | Needs verify | #vllm-rdna Sep 2026 — experimental; Sep 19 3inst-only; Sep 24 v0.29.0 mul1; Sep 29–1 Oct HEAD mul1 GPU-run (27B 3.00 bpw TP=4). Oct 2: mul1-tested only; public quark75 27B mul1; calibrated 3inst Flash-Next untested. Not Hub |
| INT8 KV (not decode shadows) | Needs verify | #vllm-rdna Sep 23–24 — community testing; QSA backends still unquantized-only |
| HIP MoE non-deterministic vs Triton | Needs verify | #vllm-rdna Sep 19 — ~3.5% top-token drift; Sep 30 fp32 deterministic path described as opt-in (perf cost); flag name not posted |
| Intel AutoRound W4A16 Flash-Next | Community | #vllm-rdna Sep 10–11 + draft opengfx1030/vllm-rdna#5; Sep 20 QSA #15 used this pack + BF16 PLE (not wtdcode + PLE-quant) |
| Swift 1.5 Flash-Next AWQ first-look ~1300/45 on 3× | Needs verify | #vllm-rdna Sep 25–27 — public ukisai packs; AutoRound 50-iter vs Intel 200; Sep 27 AutoRound looping |
INT8 decode shadows not in QSA #15 | Fork-source | #vllm-rdna Sep 20 — excluded from merge; quality loss already noted Sep 16 |
Asymmetric W4A16 MoE #98 | Fork-source | #98 merged 14 Oct — native experts for asymmetric compressed-tensors (e.g. cyankiwi/*-AWQ-INT4). 8-bit still Triton. Not Hub |
fork.md
| Statement | Status | Verify how |
|---|---|---|
Fork landscape: official opengfx1030/vllm-rdna rdna_extras; Hub v0.31.0-extras current | Solid | Org repo + default branch; #announcements 11 Oct 2026 Hub bake. leapdragon is legacy container/docs |
| Consolidation / 0.28 gap audit | Community | #vllm-rdna Sep 3–7 channel notes; treat as WIP |
| No WMMA on RDNA2 | Solid | Architecture |
| Kernel file list | Solid | Fork tree (rdna_extras) |
fa_rdna2 head_size=256 | Fork-source | Commit 03b2d91 |
Public extras fa_rdna2 dropped FP16 pieces | Community | #vllm-rdna Sep 22 — maintainer focused on W4A16 validation |
2kiss/flash-attention-rdna2 | Needs verify | #vllm-rdna Sep 22–24 — independent kernel; stale; org fa_rdna2 currently disabled after #17 |
| GDN decode ~9.3× vs Triton | Community | Fork microbench |
| GDN full HIP prefill chain | Fork-source | Commits 69d2efe, b53a7a2c |
TP allow_in_graph fix | Fork-source | Commit b583d64 |
| HIP MoE non-deterministic (~3.5% top-token drift) | Needs verify | #vllm-rdna Sep 19 — Triton MoE bit-identical; Sep 30 fp32 deterministic opt-in (perf cost); no public flag name |
| QSA live-context prefill bound | Fork-source | Merged opengfx1030/vllm-rdna#15; Python-only scoring width change |
Upstream vllm#57160 CPU KV offload pin | Needs verify | #vllm-rdna Sep 20 — mainline, not 0.29; #24 is the gfx1030-facing follow-through |
Adaptive prefill #36 | Fork-source / Needs verify | Merged 5 Oct; #vllm-rdna Oct 4 community mixed-load notes. In Hub v0.31.0-extras (opt-in) |
MoE expert DRAM cache #37 | Needs verify | Draft opengfx1030/vllm-rdna#37; #vllm-rdna Oct 4; no gfx1030 tok/s |
| v0.31.0 extras release | Fork-source | #announcements 11 Oct + Hub v0.31.0-extras. Three PLE modes tested. Author Flash-Next benches above. |
PP3 Flash-Next #96 / #95 | Fork-source / Community | #96 merged 13 Oct; #95 int8 row-chunk shadows. Author-host 16k/1k Community |
Prefix-cache off #97 | Fork-source | #97 merged 14 Oct — ENABLE_PREFIX_CACHING=0 now --no-enable-prefix-caching |
Asymmetric W4A16 MoE #98 | Fork-source | #98 merged 14 Oct |
Flash-Next TP4 HW queues #107 | Fork-source | #107 merged 16 Oct — GPU_MAX_HW_QUEUES=4; launcher default 2 |
rocprofiler register #108 | Fork-source / Community | #108 merged 16 Oct; #vllm-rdna idle ≥1 core/GPU. Distinct from ROCR TheRock#7051 |
Flash-Next MonoKernel #94 | Fork-source | #94 merged 16 Oct; --monokernel / VLLM_ROCM_MONO_DECODE=1; off by default; not Hub; recipes do not enable yet |
#94 same-session ITL table (4× TP4) | Fork-source | PR body 16 Oct — ITL −6% to −13%; MTP-2 decode tok/s follows acceptance. Use ITL |
| MonoKernel second-host PP cliff / 100% VRAM | Community / Needs verify | #vllm-rdna 16 Oct — TunableOp suspected; author-host PP stayed >2.1k |
| Flash-Next vLLM 2-card still does not fit | Community | #vllm-rdna 16 Oct — MonoKernel does not change that |
| Triton 3.9+ drops RDNA1/RDNA2 | Community | #vllm-rdna 17 Oct — stay on bake 3.7.1 / 7.2 pin 3.5.1 |
HSA_KERNARG_POOL_SIZE=262144 ~14 GB host RAM | Community / Needs verify | #vllm-rdna 17 Oct — TP=8; c=8 16k under 1%. Prefer HIP_FORCE_DEV_KERNARG=1 |
| PP3 13 Oct table not re-benched after later HEAD | Community | #vllm-rdna 17 Oct — snapshot, not a current-HEAD guarantee |
MonoKernel QSA HIP #126 | Fork-source | #126 merged 18 Oct; --monokernel indexer in HIP; default path unchanged |
Dynamic MTP opt-in #127 | Fork-source / Community | #127 merged 18 Oct; #vllm-rdna 22 Oct leave num_speculative_tokens_per_batch_size unset (repeat loops) |
rdna_ar sticky abort #128 | Fork-source | #128 merged 19 Oct — WEDGED instead of silent RPC timeout |
HIP set default-on #130 (closes #99) | Fork-source | #130 merged 19 Oct; Flash-Next recipes HIP on; ITL −6 to −8% without --monokernel. Not Hub |
Non-mono graph hang #131 | Fork-source | #131 merged 19 Oct — no aux-stream SharedExperts in CUDA graphs on gfx10x |
Flash-Next DFlash2 #132 | Fork-source | #132 merged 21 Oct; RECIPE=flashnext-dflash2; MTP-2 faster in PR A/B. Not Hub |
| DFlash2 vs MTP-2 agentic | Community / Needs verify | #vllm-rdna 22 Oct — maintainer MTP-2 still ahead; DFlash2 can win short / single-user |
EXL3 Flash-Next #117 | Fork-source | #117 merged 17 Oct; RECIPE=flashnext-exl3 / PLE=exl3; MTP=0. Not Hub |
MTP-2 MonoKernel #137 | Fork-source | #137 merged 24 Oct — sticky-fault containment; still opt-in --monokernel. Not Hub |
INT4 KV int4_fp16_block #138 | Fork-source | #138 merged 25 Oct — opt-in; native QSA path; other backends refuse. Not Hub |
| INT4 KV + MonoKernel | Needs verify | #138 documents non-mono QSA; mono combo not claimed |
INT8 KV restore #71 | Needs verify | Draft; #vllm-rdna 21 Oct maintainer priority |
fp16 SSM + W4A8 default #140 | Fork-source | #140 merged 26 Oct; 27b-awq W4A8=1; author-host 860→1075 PP / 42.5→52.3 agg. Not Hub |
| HP Z6 G4 slot-5 PSP / fan PWM | Community / Needs verify | #motherboard 18–23 Oct — BIOS factory reset unstuck blowers; slot 5 PSP load kdb failed! unresolved |
| DeepSeek-V4 correctness write-up | Community / Needs verify | #vllm-rdna 21 Oct — community fork of rdna_extras; in-tree dsv4-flash still Needs verify |
fork-progress.md
| Statement | Status | Verify how |
|---|---|---|
| Snapshot dated 26 Oct 2026 | Fork-source / Community | Public merges on rdna_extras + #announcements / #vllm-rdna |
Hub v0.31.0-extras feature set | Fork-source | #announcements 11 Oct |
HEAD table (#94–#140) | Fork-source | Merged PR dates on opengfx1030/vllm-rdna |
| Open backlog rows | Needs verify | Draft/open PRs — not recipes |
host-venv.md
| Statement | Status | Verify how |
|---|---|---|
7.2.0 index download.pytorch.org/whl/rocm7.2, torch 2.12.0, triton 3.5.1 | Solid | vllm-rdna-docker base-rocm720 |
7.2.0 host paste also had triton-rocm==3.7.1 | Community | #general 1 Aug 2026 |
7.14.0 index repo.amd.com/rocm/whl-multi-arch/, torch 2.12.0+rocm7.14.0, triton 3.7.1 | Solid | docker-bake.hcl base-rocm714 |
Published 7.14.0 tag described as PyTorch 2.13.0 | Solid (bake) / stale README | docker-bake.hcl base-rocm714 pins 2.12.0+rocm7.14.0; wiki Running matches bake. Docker-repo README 2.13.0 cell is stale |
pytorch.org/whl/rocm7.14 has no cp312 x86_64 torchaudio | Community | #vllm-rdna 21 Sep 2026 |
Device wheels rocm-sdk-device-gfx1030 + amd-torch-device-gfx1030 | Solid | Dockerfile.base INSTALL_ROCM_SDK_DEVICE=1 |
Missing device wheels → hipErrorInvalidImage on index torch | Fork-source | Dockerfile.base comment |
[device-gfx1030] is only an extra on multi-arch index wheels | Solid | TheRock device extras; source wheels have no such extra |
From-source gfx1030 wheels: install the .whl, no device extra | Community | uv warning on ~/wheels/rdna2/ (torch==2.12.0+git6bbd260) |
| FA allowlist drops gfx1030; Triton FA at runtime | Solid | Dockerfile.base |
causal-conv1d pin 4f6ae4e + HIP_ARCHITECTURES | Solid | Dockerfile.vllm |
Do not pass --torch-backend=rocm7.14 | Community | That flag selects the PyTorch index that failed on 21 Sep |
_rocm_sdk_* LD_LIBRARY_PATH | Community | #vllm-rdna serve blocks; same dirs as the image ldconfig |
| Triton 3.9+ drops gfx1030 | Community | #vllm-rdna 17 Oct — do not bump past 3.7.1 on 7.14 / 3.5.1 on 7.2 |
serve-scripts.md
| Statement | Status | Verify how |
|---|---|---|
tools/rdna/serve_rdna.sh is the single entry point; old serve_gfx1030_*.sh names are aliases | Fork-source | tools/rdna/ on rdna_extras and rdna_extra/v0.31.0 (moved from scripts/) |
Recipe deltas (flashnext-mtp2 / mtp0 / flashnext / flashnext-dflash2 / 27b-awq / 27b-exl3 / dsv4-flash / full) | Fork-source | tools/rdna/recipes/*.env. flashnext-dflash2 is #132 (21 Oct); Hub lags |
| MTP=2 wins at 8×16k/1k c=8; MTP=0 wins at 8×1k/512 c=8 | Fork-source | Recipe file comments dated 2026-09-27. Tok/s tables stay on Flash-Next serve lines |
Flash-Next PLE bf16 / int4 / int4-disk | Fork-source | #announcements 11 Oct + serve_rdna.sh PLE=. INT4 table ~30 GB; #vllm-rdna Oct 11 corrected inflated ~90 GB host-RAM figure (~35 GB working set). Disk mmap via VLLM_PLE_CPU_OFFLOAD=1. Speed at parity on author 4× V620. |
Drop VLLM_USE_BREAKABLE_CUDAGRAPH=1 / do not force VLLM_RDNA_FUSED_HC=0 on Flash-Next | Community | #vllm-rdna Oct 10 (agent pastes). dsv4-flash still sets breakable graphs. |
PRINT=1 prints env, command, and TunableOp profile and does not launch | Fork-source | serve_rdna.sh header; RECIPE is an argument, MODEL and VENV stay required |
TunableOp auto-select by librocblas.so.5 hash; TUNABLEOP_PROFILE mismatch aborts | Fork-source | tunableop/README.md + profiles.json + tools/rdna2_028/tunableop_env.sh: rocm7.14-rocblas5.5 (783 rows, f30bb442e9b5), rocm10-rocblas5.6 (70, c27e2252cc7a) |
| ROCm 10 / rocBLAS 5.6 needs no launcher change; 70-row profile is thin | Fork-source | Same helper. Untuned GEMMs use the rocBLAS heuristic. Validated serve numbers are gfx1030 + ROCm 7.14. Extend rows with tunableop_rows_pipeline.sh + verify_tunableop_lookup.py |
27B EXL3 recipe defaults EAGER=1 and HIP_VISIBLE_DEVICES=4,5,6,7 | Fork-source | tools/rdna/recipes/27b-exl3.env (bring-up; second PLX on an 8-GPU chassis) |
Launcher is foreground (exec); stale kill is scoped to VLLM_CACHE_ROOT | Fork-source | serve_rdna.sh + rdna_kill_stale in tools/rdna/rdna_launcher_common.sh |
flashnext-pp3 / flashnext-pp3-mtp2 | Fork-source | #96 13 Oct; partition 16,16,16 / 17,17,14. Not Hub |
ENABLE_PREFIX_CACHING=0 → --no-enable-prefix-caching | Fork-source | #97 14 Oct |
Flash-Next TP4 GPU_MAX_HW_QUEUES=4 | Fork-source | #107 16 Oct; launcher default stays 2 |
ROCPROFILER_REGISTER_ENABLED=0 | Fork-source | #108 16 Oct; rdna_launcher_common.sh |
EXTRA_ARGS="--monokernel" / VLLM_ROCM_MONO_DECODE | Fork-source | #94 16 Oct; off by default |
llama.cpp (llama-cpp/)
building.md
| Statement | Status | Verify how |
|---|---|---|
| Fedora + ROCm 7.2.0 build | Solid | Standard recipe |
Ubuntu + ROCm 7.14 stock HIP cmake (GGML_HIP_RCCL / GRAPHS / NO_VMM) | Community | #llamacpp / #general Oct 13 — 2× V620 Precision 5820; ~3–6% vs default mainline |
--spec-draft-device on TP2+ | Community | Discord pattern |
rdna2-benchmarks.md
| Statement | Status | Verify how |
|---|---|---|
| All benchmark tables | Community | Author-reported; run your own before/after |
| Stock vs fork +56.8% (Qwen3.8-27B Q6_K MTP) | Community | #llamacpp matched A/B, byte-identical outputs |
| Q4_0 / Q8_0 fastest on fork | Community | #harnesses / #llamacpp |
| Long-context quant sweep (Q8 wins) | Community | #benchmarks Aug 30 — single host |
| ROCm 7.1 vs 10.0 Ice Lake sweep | Community | Close numbers; prefer 7.14 for RCCL day-to-day |
| Flash-Next ~28 t/s llama.cpp vs ~60 t/s vLLM | Community | #llamacpp 4× V620 comparison |
| Flash-Next APEX GGUF ~370 PP / ~26 t/s (2× V620) | Community | #llamacpp Sep 11 — mudler/Qwen3.8-Flash-Next-APEX-GGUF |
| Flash-Next Q4 ~6 t/s on 2× V620 (LocalAI) | Community | #llamacpp Sep 10 — n-gram on NVMe |
| Flash-Next IQ4_XS ~150–200 PP / ~25 t/s (2× V620) | Community | #llamacpp / #vllm-rdna Sep 14–15 — layer split + --n-cpu-moe |
| Flash-Next llama.cpp ~300 PP / ~25 t/s “normal” (2×) | Community | #llamacpp Sep 16 — drops ~200/15 on long ctx |
| Flash-Next MTP needs PR / Unsloth desktop | Community | #llamacpp Sep 17 — stock / Studio bundled llama.cpp refused MTP |
| Unsloth Studio bundled llama.cpp loads Flash-Next MTP | Needs verify | #llamacpp Sep 20 — one host; called a same-day bundle fix |
| Flash-Next 4× patched MTP ~400 PP / ~27–45 t/s | Needs verify | #llamacpp Sep 17 — others call 400 PP low |
| Flash-Next 3× + 32 GB RAM does not load | Community | #llamacpp Sep 17 — n-gram / host RAM bound |
| Flash-Next 3× + MTP needs ~64 GB RAM | Community | #llamacpp Sep 17 — later report; SSD offload untested |
| Flash-Next MTP: stock llama.cpp can load when forks refuse | Needs verify | #llamacpp Sep 18 — single-thread |
Flash-Next MTP on markldn / okigan trees, no TP | Needs verify | #llamacpp Sep 19 — MTP loaded; tensor parallel did not |
| DeepSeek-V4 llama.cpp TP4 ~22 t/s | Needs verify | #llamacpp Sep 17 — kernel unpublished |
| Flash-Next UD-IQ3_XXS ~530 PP / ~30 t/s (2× V620, stock ROCm) | Community | #general / #forum Sep 15 — official llama.cpp ROCm; ~10 t/s at 80k+; Vulkan ~25% slower |
| Flash-Next AD-4.27 GGUF ~550 PP / ~29–30 t/s (2× V620, no MTP) | Community | #llamacpp Sep 28–29 — Navin-Models/Qwen3.8-Flash-Next-Uncensored-AD-4.27-GGUF; 262k q8 KV; MTP headers failed; PCIe 3.0 host. Sep 30 LocalAI YAML + ISTA-DASLab GSQ-RCO-Coder used on another 2× host |
| Swift 1.5 Flash-Next GSQ-RCO GGUF ~400 PP / ~40 TG (2×, MTP) | Community | #llamacpp Sep 30 — ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF; IQ3_xs + q4_0 ngrams; ~48 GB RAM; batch/ubatch 4k/1k |
| Swift GSQ-RCO IQ3_XXS ~800→650 PP / ~35→20 TG (2×, MTP) | Community | #llamacpp Oct 2 — same pack IQ3_XXS; -ot per_layer_token_embd=CPU; -c 71680; 2k/1k; VRAM-bound |
| Flash-Next n-gram table ~50 GB RAM | Community | #llamacpp Sep 10 — 4× V620 to avoid storage offload |
| PR #10 / #12 status | Solid | Both merged on edwinbrowwn/llama.cpp-rdna2 (#10 13 Aug 2026; #12 17 Aug 2026) |
rdna2-speculative.md
| Statement | Status | Verify how |
|---|---|---|
| DFlash2 draft Q4_K_M not Q8_0 | Community | Discord consensus |
| DFlash2 vs MTP bench (~30 vs ~40) | Community | Synthetic bench caveat documented |
| DFlash hurts PP more than MTP | Community | #llamacpp Aug 28 |
| MTP n=3 often beats n=4 (27B Q8 TP4) | Community | Real-prompt A/B; acceptance dropped at n=4 |
| Flash-Next TP experimental / deferred | Community | Fork update + forum benches; layer-split only |
| Flash-Next llama.cpp << vLLM | Community | #llamacpp / forum Sep 2026 — except 2-card / RAM-offload |
| 2× V620 IQ4_XS layer-split + ngram-mod | Community | #llamacpp / #vllm-rdna Sep 14–15 — host-specific -t |
| 2× V620 AD-4.27 GGUF ~550 PP / ~29–30 t/s | Community | #llamacpp Sep 28–29 — no MTP; 262k q8; ISTA-DASLab GSQ pack mentioned, no tok/s |
SPEC_SIDECAR=1 MTP path | Community | Fork maintainer tip; pull latest |
| Sidecar GGUF identity / Unsloth vs other Q8 | Community | #llamacpp Sep 2026 — match publisher families |
| DFlash2 aperture violation crash | Community | #llamacpp Sep 2026 — pull latest / A/B MTP |
--spec-draft-p-min ≠ 0 disarms MTP | Community | #benchmarks Sep 2026 tip |
| Full DFlash2 TP4 command | Community | Author production recipe |
| MTP + LCP prompt-cache position desync | Community | #llamacpp Sep 10 — HTTP 200 / no tokens; --ctx-checkpoints 0 does not fix |
markldn / okigan MTP trees, no tensor parallel | Needs verify | #llamacpp Sep 19 — one host |
| Unsloth Studio bundled llama.cpp loads Flash-Next MTP | Needs verify | #llamacpp Sep 20 — one host; re-try bundled binary first |
rdna2-serving.md
| Statement | Status | Verify how |
|---|---|---|
| Short env stack preferred | Community | #llamacpp — long GFX1030_* lists can clash with RCCL |
GGML_HIP_SAFE_STATE_IO=1 FA workaround | Community | Fork maintainer note |
| ubatch ~1024 per GPU | Community | #llamacpp PP tuning tip |
| Docker compose TP2 MTP4 | Community | #llamacpp example |
| KV checkpoint workaround | Community | --ctx-checkpoints 0 |
| Multi-socket hurts TP | Community | Topology advice |
| 160 W / 140 W PSU workaround | Community | Transients on TP prefill; miner PSU / P620 cables |
| TP3 driver crash | Community | Prefer 2 or 4 GPUs |
hipcub-devel for GPU sampling | Community | Forum / #llamacpp build note |
| Single V620 Q4_0 ~39–49 t/s (27B) | Community | #llamacpp Sep 2026 |
| Embedder idle +~40 W/GPU | Community | #llamacpp — nomic/etc. alongside chat |
| Fork day-to-day on ROCm 7.14 | Community | #llamacpp Sep 2026 — not mid-7.2.x |
Concurrent --parallel crush (~5–9 t/s) | Community | #llamacpp Sep 2026 V620 multi-agent |
| Upstream PR #22466 fast tensor loads | Needs verify | Community <35 s on 122B-Q4; watch merge |
| Long-ctx prefer Q6+/Q8 | Opinion | #llamacpp Sep 2026 quality reports |
Reference & troubleshooting
| Area | Status | Notes |
|---|---|---|
reference/env-vars.md tables | Community | Cheat-sheet; not exhaustive upstream API |
VLLM_PLE_QUANT_DIR INT4 (#49 / older mmap) | Fork-source / Community | #49 (11 Oct) on rdna_extras: INT4 into upstream pinned storage. Older 700753d9 mmaps shards. #vllm-rdna Oct 11: on-disk load still WIP |
troubleshooting/vllm.md graph fix | Fork-source | Same image-lag caveat |
troubleshooting/vllm.md AMDSMI / missing render | Community | #vllm-rdna Docker compose |
tuning/power.md Fedora path | Solid | From v620_toolbox powertuning |
tuning/power.md Ubuntu 26.04 path | Community | ubuntu_powertuning/ — validated kernel 7.0.0-30-generic; re-verify after kernel upgrades |
tuning/power.md Fedora Server 44 / kernel 6.19 | Community | Ice Lake 4× V620 host — power floor + 180 W + ~7 W idle |
tuning/power.md V620 slot-power / setperflevel | Community | #llamacpp — TDP from slot, not 8-pin |
tuning/power.md soft unlock (amd-v620-soft-unlock) | Community | Upstream README + #general passthrough reports; wiki-unverified TFLOPS / 232–275 W range |
setup/hardware.md prefer soft unlock over W6800 flash | Community | Soft unlock keeps 72 CUs; W6800 flash → 54 CU |
tuning/ecc.md two-reboot ras_enable=0 | Community | lunnova on W6800; Discord reports V620 ECC-on (~30 GB). Confirm rocm-smi after two reboots |
tuning/ecc.md Vulkan memtest before ECC-off | Community | #general Sep 23 — quiet dmesg ≠ clean card; leave ECC on if errors |
tuning/ecc.md Flash-Next can trip ECC that 27B misses | Community | #general Sep 26 — dmesg only on Flash-Next; 5–8 min Vulkan memtest; replacement OK |
tuning/p2p.md validation | Solid | From v620_toolbox on Fedora + AMD CPU |
tuning/p2p.md Ubuntu 26 / 7.0 kernel gap | Community | #vllm-rdna Sep 24 — validated Pre-7 ≤6.19 or Post-7 ≥7.1; 7.0 not the Fedora recipe |
tuning/p2p.md ~25 GB/s bandwidth | Community | Bandwidth ≠ inference speed |
tuning/p2p.md host topology table | Community | Dual-socket, gen3 x4, PLX, TP3 — #general / #llamacpp |
tuning/p2p.md Ice Lake P2P no-op / ~4% regression | Community | Ice Lake 4× V620 host — llama.cpp + vLLM |
tuning/p2p.md Intel IOMMU-off breaks P2P | Community | Ice Lake host; opposite of some generic docs |
tuning/p2p.md ACS bits / ACSCtl=0x0019 | Community / Needs verify | #general 25 Oct — all-off vs leave bits 0/3/4; toolbox “ACS off” can still show 0x0019. Clearing remaining bits broke V620↔V620 on one host |
tuning/p2p.md PLX daisy-chain / heatsink fan | Community | #general PLX 88096 |
tuning/p2p.md PEX88096 8-riser / H12D-8D x8x8 | Community | #general Sep 19 — 6 GPUs still awkward; 2/4/8 vs any even count |
tuning/p2p.md X570 x8x8 gen4 + single x16 PEX88096 uplink | Community | #general Sep 21 — cards talk on the switch bus |
troubleshooting/llama-cpp.md RADV crash / AMDVLK slow | Community | #llamacpp — prefer ROCm for TP |
troubleshooting/llama-cpp.md Windows Unsloth Vulkan | Needs verify | #general Oct 10 — 2× V620 Q8 ~12–13 t/s |
troubleshooting/llama-cpp.md FA max_blocks_per_sm abort | Community | head-256 occupancy 0 on gfx1030; q8 KV needs FA |
troubleshooting/llama-cpp.md DAX mmap SVM oops | Community | --no-mmap mandatory on dax=always |
troubleshooting/general.md CPU governor / unsupported AMDGPU punt | Community | Flash-Next PP; Polaris/WX4100-in-box ROCm skip; unbind > ROCR_VISIBLE alone |
troubleshooting/vllm.md leftover EngineCore / PleOffloadWorker | Community | #vllm-rdna Sep 18 |
troubleshooting/vllm.md TunableOp rocBLAS hash lock | Fork-source | #vllm-rdna Sep 23–24 + tunableop/ README (named profiles; canonical f30bb442e9b5) |
troubleshooting/vllm.md Flash-Next vision pixel-cap / runtime OOM | Fork-source / Community | Launcher max_pixels; #vllm-rdna Sep 21 ~15–20 s/image |
troubleshooting/vllm.md FP8 KV rejected on QSA | Community | #vllm-rdna Sep 22 — unquantized-only backends + AITER/CDNA |
troubleshooting/vllm.md no-P2P PYNCCL / MTP vs PLE | Community | #vllm-rdna Sep 18 |
troubleshooting/vllm.md MTP concurrency / PLE stall | Community | #vllm-rdna — recipe PRs + P2P A/B |
troubleshooting/vllm.md ROCR idle CPU spin | Community | TheRock 7.14 / ROCR 1.21 — Flash-Next fork patch |
troubleshooting/vllm.md rocprofiler register idle CPU #108 | Fork-source / Community | #vllm-rdna 16 Oct — distinct from ROCR busy-spin; launchers default ROCPROFILER_REGISTER_ENABLED=0. Not Hub |
troubleshooting/vllm-flash-next.md PP3 HEAD recipes #96 | Fork-source | #96 13 Oct supersedes leapdragon-only “what to do” |
reference/env-vars.md GPU_MAX_HW_QUEUES=4 on Flash-Next TP4 | Fork-source | #107 16 Oct |
reference/env-vars.md ROCPROFILER_REGISTER_ENABLED | Fork-source / Community | #108 16 Oct |
reference/env-vars.md VLLM_ROCM_MONO_DECODE | Fork-source | #94 16 Oct |
reference/env-vars.md HSA_KERNARG_POOL_SIZE | Community / Needs verify | #vllm-rdna 17 Oct — TP=8 ~14 GB; not a 4× recipe |
vllm/flash-next-serve.md MonoKernel #94 ITL table | Fork-source | PR #94 16 Oct; Hub lags |
vllm/flash-next-serve.md MonoKernel second-host PP | Community / Needs verify | #vllm-rdna 16 Oct |
vllm/host-venv.md / images.md Triton 3.9 drop | Community | #vllm-rdna 17 Oct |
reference/env-vars.md VLLM_FA_RDNA2_GQA_DECODE | Fork-source | launcher ATTN=fa sets 1; not a Flash-Next QSA lever |
vllm/quantization.md asymmetric W4A16 MoE #98 | Fork-source | #98 14 Oct; not Hub |
vllm/flash-next-serve.md PP3 HEAD 16k/1k table | Community | #general 13 Oct author-host; not wiki-tested |
troubleshooting/vllm.md Flash-Next V2=0 long prompts | Community | #vllm-rdna Sep 2026; Sep 15 100%/~40 W stall |
troubleshooting/vllm.md Flash-Next PP3 rdna_extras corruption | Needs verify | #vllm-rdna Sep 15–16 — leapdragon OK; fresh pull still bad |
troubleshooting/vllm.md PP3 b33f9b6 graph KeyError | Needs verify | #vllm-rdna Sep 19 — [1,2,4,8] capture; 1.5e9 KV cap; 0% hit-rate log unreliable |
troubleshooting/vllm.md Flash-Next FULL-graph corruption | Needs verify | #vllm-rdna Sep 16–17 — PIECEWISE workaround |
troubleshooting/vllm.md wvSplitK / VLLM_RDNA_DENSE_GEMV | Community | #vllm-rdna Sep 17 fault + Oct 6 4× W6800X concurrency contrast |
troubleshooting/vllm.md hybrid KV overstated / livelock | Needs verify | #vllm-rdna Sep 17 |
troubleshooting/vllm.md prefix cache 0% / e45dd5cb | Fork-source | #vllm-rdna Sep 17 + org extras commit |
troubleshooting/vllm-flash-next.md --enable-prompt-tokens-details | Community | #vllm-rdna Oct 7 — DeepSeek / Pi harness 0% is missing metadata, not a dead cache |
troubleshooting/vllm-flash-next.md AutoRound INCConfig PLE reject | Needs verify | #vllm-rdna Oct 12 — latest rdna_extras; upstream vLLM #59990 merged; fork cherry-pick queued |
troubleshooting/vllm-flash-next.md FA-RDNA2 partial on QSA | Community / Fork-source | #vllm-rdna Oct 13 — QSA Triton owner ignores --attention-backend |
troubleshooting/vllm.md --kv-offloading-size | Community / Needs verify | #vllm-rdna Oct 7 — 64 GiB total RAM parking; SSD untested |
troubleshooting/vllm.md / Flash-Next KV-restore loops | Community / Needs verify | #vllm-rdna 25–26 Oct — token-repeat after MTP off; 26 Oct: CPU-RAM KV + prefix cache OK when disk KV offload is off |
troubleshooting/vllm.md repeating !!!! tokens | Community / Needs verify | #vllm-rdna 27 Oct — UV >~20 mV or core >~2479 MHz; memtest_vulkan + dmesg ECC |
vllm/quantization.md EXL3 vs AWQ / hi2 | Community | #vllm-rdna / #hi2 27 Oct — EXL3 size/quality, AWQ speed; hi2 rdna_hint_flex public, no recipe. 30 Oct + public README: Hadamard + QAD, quality target, 3 bpw compact maybe aggressive |
vllm/host-venv.md / setup/installing-rocm.md ROCm 10 on Ubuntu 26.04 | Community / Needs verify | #vllm-rdna 29 Oct — core-10 / core-10.1 (no core-7.14); Triton pin unconfirmed. 30 Oct: 10 ≈ 7.14 for serving; one 10.1 host-venv fail Needs verify |
setup/host-firmware.md HP Z6 G4 slot map | Community / Needs verify | #motherboard 26–29 Oct — slots 2+5 x16, slot 4 x8; hidden BIOS guess for 2nd V620 |
tuning/power.md V620 dual 8-pin enumeration | Community | #general 29 Oct — one aux cable can light LEDs without lspci |
vllm/fork.md v0.31.0 release | Fork-source | #announcements 11 Oct + Hub v0.31.0-extras + closed #47. Author 4× V620 Flash-Next benches. PP3 recipes (#96) are HEAD-only |
vllm/fork.md MonoKernel #94 | Fork-source | #94 merged 16 Oct; not Hub; recipes do not enable |
vllm/fork.md #126 / #127 / #128 / #130 / #131 / #132 | Fork-source | Merged 18–21 Oct; Hub v0.31.0-extras lags |
vllm/fork.md #117 / #137 / #138 / #140 | Fork-source | Merged 17–26 Oct; EXL3 Flash-Next, MTP mono, INT4 KV, W4A8 default. Not Hub |
vllm/fork.md / flash-next-serve.md #146 TP2 MoE offload | Fork-source | #146 merged 28 Oct; serve_offload.sh; author-host 2× V620 16k/1024 ~19.8 / ~26.7 agg. Not Hub. Needs verify on a second host |
vllm/fork-progress.md snapshot | Fork-source / Community | 31 Oct Hub vs HEAD vs backlog |
vllm/fork-progress.md draft #148 | Fork-source | #vllm-rdna 30 Oct + public draft — tiered KV paging; live paging stalls; not a recipe |
troubleshooting/vllm.md disk KV write-amp / Optane | Community / Needs verify | #vllm-rdna 30 Oct — terabytes/day NVMe at huge context; prefer U.2/PCIe/DIMM Optane over M10; LMCache NVMe still unused |
troubleshooting/general.md / tuning/power.md IOMMU + PSU-group drop | Community / Needs verify | #general 31 Oct — 8× V620 TP8/load; 6 cards on 1500 W breakout vanished; 2 on workstation PSU stayed |
vllm/flash-next-serve.md HIP defaults #130 | Fork-source | 19 Oct — Flash-Next HEAD recipes HIP on |
vllm/flash-next-serve.md DFlash2 #132 | Fork-source | 21 Oct — MTP-2 still faster in PR A/B |
troubleshooting/vllm.md rdna_ar silent hang #128 | Fork-source | 19 Oct |
troubleshooting/vllm.md non-mono graph hang #131 | Fork-source | 19 Oct |
troubleshooting/vllm-flash-next.md dynamic MTP #127 | Fork-source / Community | #vllm-rdna 22 Oct — leave schedule off. 25–26 Oct: loops can continue with MTP off if KV offload restore is on |
setup/host-firmware.md HP Z6 G4 PSP / fans | Community / Needs verify | #motherboard 18–23 Oct |
vllm/fork.md PLE three modes | Fork-source | #announcements 11 Oct + launcher PLE= + #vllm-rdna Oct 11 RAM correction |
troubleshooting/vllm.md W6800X Duo ALU 0.9.4 MTP-2 | Community | #benchmarks Oct 10 — 4× W6800X Duo, vLLM 0.30.0 TP4, Qwen3.8-27B fp16, 512 gen |
vllm/quantization.md W4A8 somewhat competitive | Needs verify | #vllm-rdna Oct 9 — 0.31 test line |
vllm/quantization.md W4A8 #140 author-host table | Fork-source | #140 26 Oct — 27B 860→1075 PP / 42.5→52.3 agg; Hub may lag |
vllm/quantization.md INT4 KV #138 | Fork-source | #138 25 Oct — int4_fp16_block; native QSA; not Hub |
troubleshooting/vllm.md mixed stall / CUDA-graph range | Needs verify | #vllm-rdna Oct 9–10 — no public flag; DSV4 expected after |
vllm/flash-next-serve.md DSV4 on extras WIP | Needs verify | #vllm-rdna Oct 10 — will work after stall fix; stay on yiminyuan tree |
llama-cpp/rdna2-serving.md froggeric tool calls | Needs verify | #general Oct 9 — OG Qwen3.8 + Open WebUI |
troubleshooting/llama-cpp.md Windows Unsloth Vulkan | Needs verify | #general Oct 10 — 2× V620 Q8 ~12–13 t/s; check both GPUs / Q4_0 |
troubleshooting/llama-cpp.md V620 not in WSL / Windows P2P CPU fallback | Needs verify | #general Oct 11 — silent CPU path for GPU↔GPU; install Linux |
vllm/serve-scripts.md launcher P2P=0 | Fork-source | serve_rdna.sh on rdna_extras + #vllm-rdna Oct 11 |
vllm/flash-next-serve.md cyankiwi Flash-Next AWQ incoherent | Community | #vllm-rdna Oct 11 — prefer wtdcode AWQ-W4A16 + primitive-ai PLE |
vllm/recipes.md TP vs PP llama.cpp mapping | Community | #vllm-rdna Oct 8 — PP≈layer split, TP≈tensor split; can combine |
troubleshooting/general.md unused V620 fan pads | Community | #general Oct 7 — 4-pad footprint; do not use as PWM (100% / no curve) |
vllm/quantization.md MTP draft_sample_method | Community | #vllm-rdna Oct 8 — probabilistic when temperature > 0 |
reference/env-vars.md PLE_OFFLOAD_PREFAULT | Community / Needs verify | #vllm-rdna Oct 8 — overlay/sidecar name; ~25% slower off; not org extras HEAD |
choose-a-stack.md / meta/resources.md Strata HIP | Community / Needs verify | #general Oct 7–8 + public Niko1221/Strata. 2× V620 0.1.39+#835/#849 PP jump; 3-card behind vLLM. No serve recipe |
choose-a-stack.md GLM-5.3-Flash not 1-card | Community / Needs verify | #general Oct 25 — 1× V620 + RAM offload poor fit; Maya ~200/13 stale; public 320B/18B active. 8× recipe already on recipes.md |
choose-a-stack.md / meta/resources.md Strata 0.1.49.2 DeepSeek engine | Community | #strata Oct 25 — signed feed has no gfx1030 HIP DeepSeek engine (gfx1200/gfx1201 + CUDA) |
llama-cpp/rdna2-serving.md froggeric Qwen Jinja | Needs verify | #vllm-rdna Oct 8 + public froggeric/Qwen-Fixed-Chat-Templates; #general Oct 9 tool calls |
vllm/recipes.md 3× MoE HIP overlay | Community | #vllm-rdna Sep 17–18 — public temp repo |
vllm/recipes.md DeepSeek-V4 Flash | Needs verify | #vllm-rdna Sep 17 — not Hub |
setup/hardware.md no NVIDIA+V620 TP | Community | #general Sep 17 — mixed prefill collapsed |
vllm/recipes.md 4× PIECEWISE Flash-Next serve | Community | #vllm-rdna Sep 17 — host venv, not Hub |
troubleshooting/vllm.md native KV RAM offload #24 | Fork-source / Needs verify | #vllm-rdna Sep 26–27 + merged vllm-rdna#24 (27 Sep); Sep 15 ~1 t/s is the broken baseline |
troubleshooting/vllm.md mixed prefill+decode / #26 / 30632b2 / #36 | Community / Fork-source | #vllm-rdna Sep 27–29 static flags; #36 merged 5 Oct (opt-in dynamic). Sep 29 unset flags for max PP + resident MoE PP drop |
tuning/p2p.md 5-slot PCIe switch ATX Printables | Community | #other Sep 30 — public model 1860633; mechanical mount only |
vllm/recipes.md GLM-5.3-Flash 8× local 0.30 fork | Community / Needs verify | #vllm-rdna Sep 30 — wtdcode AWQ; org #2 unverified |
vllm/recipes.md 2× fp16 27B MTP / Swift 1.5 27B | Community / Needs verify | #vllm-rdna Oct 6 — not Flash-Next Swift |
vllm/quantization.md EXL3 mul1 GPU-run on HEAD | Fork-source / Needs verify | rdna_extras 30 Sep–1 Oct + #vllm-rdna Sep 30 |
troubleshooting/vllm.md TORCHINDUCTOR_CACHE_DIR | Fork-source | EXL3 74315f4 — VLLM_DISABLE_COMPILE_CACHE does not clear it |
troubleshooting/vllm.md GLM vision MIOpen per-size compile | Community | #vllm-rdna Sep 30 — conv-as-matmul workaround |
llama-cpp/rdna2-serving.md LocalAI AD-4.27 YAML | Community | #llamacpp Sep 30 — 2×; context 229376 |
llama-cpp/rdna2-speculative.md Swift GSQ-RCO GGUF | Community | #llamacpp Sep 30 — ~400/40 on 2× MTP; Oct 2 IQ3_XXS ~800→650 / ~35→20 |
llama-cpp/rdna2-serving.md Swift IQ3_XXS llama-server | Community | #llamacpp Oct 2 — 2×; paths generalized |
vllm/quantization.md EXL3 mul1-only + W4A8 in HEAD | Community / Needs verify | #vllm-rdna Oct 1–2 — 3inst untested; Oct 1 W4A8 not a speed win; Oct 9 0.31-line kernel more competitive (Needs verify) |
vllm/serve-scripts.md TunableOp 4-card / 8× local rows | Community | #vllm-rdna Oct 1 — 783-row set is 4-card 7.14; other rocBLAS needs local tune |
setup/hardware.md ComfyUI / image-video bf16 | Community | #general Sep 28 — no native BF16; HIP emulates; prefer fp16/int8 |
troubleshooting/vllm.md vllm#57160 pinned CPU KV | Needs verify | #vllm-rdna Sep 20 — mainline; #24 is the fork follow-through |
troubleshooting/general.md 120 mm Makerworld ducts / 80 mm 7k | Community | #general / #other Sep 27–28 |
vllm/recipes.md 3× PP3 + MTP VRAM squeeze | Needs verify | #vllm-rdna Sep 15 — local patches |
setup/hardware.md V620 vs 32 GB MI50 more PP less TG | Community | #general Sep 15 — first-pass A/B |
setup/host-firmware.md GpuMMIOFix every-boot remap | Needs verify | #motherboard Sep 15–16 — suggested for WS-Z270 ReBAR |
troubleshooting/vllm.md 128k PP cliff / batched-tokens 4096 | Community | #vllm-rdna Sep 11 — Intel AutoRound draft |
troubleshooting/vllm.md shm_broadcast / Triton vs RCCL | Community | #vllm-rdna Sep 7 — wait + cache; stay on 7.14 |
troubleshooting/llama-cpp.md DFlash2 / sidecar / concurrent | Community | #llamacpp Sep 2026 |
troubleshooting/llama-cpp.md MTP LCP position desync | Community | #llamacpp Sep 10 — ctx_dft / M-RoPE X < Y |
troubleshooting/general.md V620 thermals / graphene pads | Community | #general Sep 2026 — mixed repaste reports |
vllm/overview.md MoE sweet spot / Flash-Next needs 4 cards | Opinion | #general Sep 2026 workload consensus |
tuning/power.md 8× @ 180 W ≈ 1440 W + HELA 2050 | Community | #forum Sep 2026 build notes |
tuning/power.md 1600 W for 5× stock 250 W; 1200 W tight for 4× | Community | #general Sep 22–23 — 0.7 A blowers OK at 180 W; ~86 °C at 250 W |
vllm/recipes.md 3× overlay unmaintained / KV dump loops | Community | #general Sep 23 — cap ~3 agents on long ctx; ~45 t/s / ~1100 PP on one 3-card host |
tuning/power.md 180 W token-cost economics | Community | #llamacpp vs stock 250 W |
tuning/power.md 6.1 power1_cap write without toolbox | Community / Needs verify | #general 17 Oct — Debian 12.12 / kernel 6.1 accepted 180 W; power1_cap_min stayed 0; temps down. Not a 120 W floor unlock |
tuning/power.md 6.7+ sysfs power-limit ignore | Community | #general 17 Oct + r/linux_gaming share; maintainer toolbox still works on 7.x |
tuning/p2p.md SlimSAS / passive riser notes | Community | #general cabling |
tuning/p2p.md external Xpander / narrow uplink | Community | #general Sep 2026 — long-ctx collapse |
tuning/p2p.md layer vs tensor split explainer | Community | #benchmarks Sep 2026 |
setup/installing-rocm.md avoid mid-7.2.x (e.g. 7.2.4) | Community | Same RCCL pin as 7.2.1+; prefer 7.2.0 or 7.14.0 |
setup/hardware.md W6800 BIOS on V620 → 54 CU | Community | #general Sep 2026 PSA — stay on stock V620 VBIOS |
setup/host-firmware.md Dell Precision MMIO High / mmiohsize | Community | #motherboard / #general Sep 12–14 2026 — dump your own UEFI; 0x03 vs 0x04 |
setup/host-firmware.md T7820 4× V620 + PEX880xx POSTed after SR-IOV off | Needs verify | Single-host #motherboard Sep 14 2026 — board-specific hidden vars |
setup/host-firmware.md CMOS loss re-enables SR-IOV / disables ReBAR | Community | #general Sep 23 — MMIO / missing GPUs until BIOS restored |
setup/host-firmware.md 256 GB MMIO window for 4× V620 | Community | #motherboard — 64 GB too small; no known 128 GB option |
tuning/p2p.md 4-slot vs 5-slot PLX card width | Community | #general Sep 12 2026 |
tuning/p2p.md 5 GPUs → TP=4 + 1 standalone | Community | #general Sep 11 2026 |
troubleshooting/llama-cpp.md lemonade-sdk M-RoPE A/B | Needs verify | #llamacpp Sep 13 — one host; not the RDNA2 fork |
troubleshooting/general.md blower vs unducted 120 mm | Community | #general Sep 13 2026 |
troubleshooting/general.md 9733 / Thingiverse 7296707 + PWM hub | Community | #general Oct 4–6 — 9733 loud with pulse PWM or Molex-at-100%; EFH-08E12W-JP01 + Arctic ACFAN00351A + CoolerControl; SUNON PF97331B1 + SilverStone CPF04 quieter path; Molex-for-four Needs verify |
setup/host-firmware.md H12D POST loop / SR-IOV + ReBAR + CSM | Needs verify | #general Oct 6 — codes 95, DE, AD, DF, 36, 05, 37; then POSTed. Single-host |
meta/resources.md TensorFold ROCm #389 | Needs verify | #tensorfold Oct 4–6 + public ashhart/TensorFold#389 (draft --backend rocm). RDNA2 is FP16 v_dot2, not WMMA. Community: not faster than vLLM. Oct 6: Python engine frozen; ROCm path moving to Zig. No wiki serve recipe |
setup/host-firmware.md
| Statement | Status | Verify how |
|---|---|---|
Dell Precision MMIO High / mmiohsize 0x03 or 0x04 | Community | #motherboard / #general Sep 12–14 — dump your own UEFI |
| T7820 4× V620 + PEX880xx POSTed after SR-IOV off | Needs verify | Single-host #motherboard Sep 14 — board-specific |
| CMOS / BIOS loss re-enables SR-IOV / disables ReBAR | Community | #general Sep 23 — fewer GPUs / MMIO errors until settings restored |
| 256 GB MMIO window for 4× V620 | Community | #motherboard — 64 GB too small |
| GpuMMIOFix OS BAR remap | Needs verify | #motherboard Sep 15–16 — typically every boot |
HP Z4 G4 32-bit MIMO + ReBAR + pci=realloc=off | Needs verify | #motherboard Sep 16 — Xeon W-2133; Linux reallocates 64 GB VRAM |
| HP Z6 G4 blower PWM + slot-5 PSP kdb | Needs verify | #motherboard 18–23 Oct — factory reset unstuck fans; second V620 in slot 5 unresolved. 29 Oct: slots 2+5 x16 / slot 4 x8; hidden BIOS guess |
| H12D V620 POST loop; SR-IOV off + Above 4G + CSM off + ReBAR on | Needs verify | #general Oct 6 — one host; menu-level, not Dell hidden vars |
Checklist before production
docker pulllatest-extrasimage; confirm fork commit in build metadata.- Grep logs for
Using RDNA2W4A16LinearKernel. - Try CUDA graphs before
--enforce-eager. - One matched A/B on your hardware.
- Update this page when you confirm or refute a claim.
See Contributing and Wiki structure.