llama.cpp or vLLM
Pick a stack before you follow a recipe. Both run on gfx1030; they are good at different jobs. Numbers below are community snapshots — see Verification status.
Which stack
| Workload | Prefer |
|---|---|
| Getting ROCm + multi-GPU working; GGUF; single-stream / low concurrency | llama.cpp (RDNA2 fork) — more battle-tested on V620 |
| Multi-stream / agentic loads with prefix caching | vLLM -extras — caching + concurrency usually win |
| Qwen3.8 Flash-Next on 4× V620 (weights in VRAM) | vLLM Flash-Next / rdna_extras — see Flash-Next. llama.cpp on the same job is typically much slower |
| Flash-Next on 2× V620, or any host that must RAM/CPU-offload | llama.cpp (--n-cpu-moe, -ot …=CPU) is the non-HEAD path — community 2-card IQ4_XS ~25 t/s / 150–200 PP. HEAD vLLM after #146 (28 Oct) has serve_offload.sh — 2× offload. Needs verify on a second host. |
| MoE / lighter agentic (Qwen3.6 35B-A3B, Ornith-class) | Either — the sweet spot on these cards |
| Curious about Strata (HIP, not llama.cpp) | Experimental. #general Oct 7–8: 2× V620 prompt jumped after gfx103x PRs #835 / #849 + STRATA_HIP_PROMPT_F16=1; 3-card hosts still could not match vLLM prefill and decode sagged under concurrency. #strata (25 Oct): Strata 0.1.49.2 signed DeepSeek engine feed has no gfx1030 HIP package (HIP build lists gfx1200/gfx1201). #vllm-rdna (28 Oct): easier onboarding / dashboard than vLLM; more interesting on fewer than 4 cards or when experts must spill. Not a wiki default — resources |
New to the cards? Start with RDNA2 serving, then vLLM onboard when you need concurrency. Multi-GPU tensor parallel benefits from PCIe P2P.
What fits well on V620
#general (Sep 2026) consensus — RDNA2 has no matrix / tensor cores, so prefill on dense
models is the weak spot (agentic “read a pile of files” workloads frustrate people even when decode
looks fine):
| Workload | Community take |
|---|---|
| MoE (e.g. Qwen3.6 35B-A3B, Ornith-class) | Sweet spot on 1–4× V620 |
| Dense 27B | Usable; expect mediocre PP vs newer silicon |
| Flash-Next | Promising on 4× V620 via vLLM Flash-Next; not a 1-card path |
| GLM-5.3-Flash / DeepSeek-class large MoE | Not 1–2 cards. Public zai-org/GLM-5.3-Flash is 320B / 18B active. #general (25 Oct): 1× V620 + host-RAM offload is a poor fit (community Maya snapshot ~200 PP / ~13 TG — Needs verify). Community vLLM path is 8× — recipes. |
| Dense agentic on 1–2 cards | Often disappointing TTFT / PP — prefer MoE or more cards |
| ComfyUI image / video | Poor fit — many workflows default to bf16 (emulated on V620). See hardware |
Next
- Getting started
- vLLM recipes — Hub
-extrasvs recipe container - llama.cpp overview