llama.cpp on gfx1030 — Overview
llama.cpp is a fast, low-dependency way to run GGUF LLMs on gfx1030. This section covers stock builds and the community RDNA2-optimized fork.
Where to start
| Goal | Page |
|---|---|
| Build stock llama.cpp (ROCm or Vulkan) | Building & running |
| Multi-GPU tensor parallel, MMQ tuning, DFlash2 | RDNA2 fork overview |
| Fork benchmarks and PR status | RDNA2 benchmarks |
| DFlash2, MTP, ngram speculative decoding | RDNA2 speculative decoding |
| Launch commands, Docker, limits | RDNA2 serving |
| Something broke | llama.cpp troubleshooting |
Which path?
- Stock llama.cpp — good starting point, single-GPU. Vulkan without ROCm is possible but
#llamacppprefers HIP for--split-mode tensor(RADV has crashed V620 hosts). On Ubuntu + ROCm 7.14, use the extra HIP cmake flags in Building — default llama.cpp HIP flags were called not enough (#general/#llamacpp, Oct 2026). edwinbrowwn/llama.cpp-rdna2— multi-GPU V620 rigs, tensor parallel, RCCL all-reduce, DFlash2. Most#llamacppperformance work happens here. A matched A/B on Qwen3.8-27B Q6_K + MTP reported +57% vs stock with identical outputs — see Benchmarks.
llama.cpp vs vLLM
The comparison table and the “what fits on a V620” notes live on llama.cpp or vLLM. Short version: GGUF and single-stream start here; concurrency and 4× Flash-Next go to vLLM.
Multi-GPU tensor parallel benefits greatly from PCIe P2P.