llama.cpp on gfx1030 — Overview
llama.cpp is a fast, low-dependency way to run GGUF LLMs on gfx1030. This section covers stock builds and the community RDNA2-optimized fork.
Where to start
| Goal | Page |
|---|---|
| Build stock llama.cpp (ROCm or Vulkan) | Building & running |
| Multi-GPU tensor parallel, MMQ tuning, DFlash2 | RDNA2 fork overview |
| Fork benchmarks and PR status | RDNA2 benchmarks |
| DFlash2, MTP, ngram speculative decoding | RDNA2 speculative decoding |
| Launch commands, Docker, limits | RDNA2 serving |
| Something broke | llama.cpp troubleshooting |
Which path?
- Stock llama.cpp — good starting point, single-GPU. Vulkan without ROCm is possible but
#llamacppprefers HIP for--split-mode tensor(RADV has crashed V620 hosts). edwinbrowwn/llama.cpp-rdna2— multi-GPU V620 rigs, tensor parallel, RCCL all-reduce, DFlash2. Most#llamacppperformance work happens here. A matched A/B on Qwen3.8-27B Q6_K + MTP reported +57% vs stock with identical outputs — see Benchmarks.
llama.cpp vs vLLM on V620 (#llamacpp / #vllm-rdna)
Community rule of thumb (Aug 2026):
| Workload | Prefer |
|---|---|
| Getting ROCm + multi-GPU working; GGUF; single-stream / low concurrency | llama.cpp (RDNA2 fork) — more battle-tested on V620 |
| Multi-stream / agentic loads with prefix caching | vLLM -extras — caching + concurrency usually win |
| Qwen3.8 Flash-Next on 4× V620 | Prefer vLLM Flash-Next recipe (~60–100+ t/s decode class after Sep 2026 prefill work; earlier ~50–60) over llama.cpp (~15–35 t/s typical; 2-card APEX GGUF ~26 t/s) — see vLLM overview and Flash-Next on llama.cpp |
| MoE / lighter agentic | Either — community sweet spot for these cards; see What fits well |
Neither stack is “finished” for every model. New to the cards? Start with RDNA2 serving, then try vLLM when you need concurrency.
Multi-GPU tensor parallel benefits greatly from PCIe P2P.