Q: What quantization of Yi 1.5 34B should I use on a AMD Radeon RX 6800 XT?

For 16 GB VRAM on the AMD Radeon RX 6800 XT, the Q8_0 variant is the best fit. Estimated ~4 tokens/sec on the Q8_0 quantization.

Q: How fast does Yi 1.5 34B run on AMD Radeon RX 6800 XT?

Roughly 4 tokens/sec for Q8_0. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Q: What if AMD Radeon RX 6800 XT is not enough for Yi 1.5 34B?

Consider upgrading to Apple M4 Pro (48 GB VRAM) which fits the recommended 44 GB target. Or pick a smaller quantization to stay on your current card.

Question 1

Can I run Yi 1.5 34B on a AMD Radeon RX 6800 XT?

Accepted Answer

Sort of — AMD Radeon RX 6800 XT can run Yi 1.5 34B (Q8_0) only by spilling layers to RAM. Generation will be slow. CPU + GPU hybrid — not enough VRAM (16 GB < 38 GB min), but 64 GB RAM is sufficient. Expect significantly slower inference.

Question 2

What quantization of Yi 1.5 34B should I use on a AMD Radeon RX 6800 XT?

Accepted Answer

For 16 GB VRAM on the AMD Radeon RX 6800 XT, the Q8_0 variant is the best fit. Estimated ~4 tokens/sec on the Q8_0 quantization.

Question 3

How fast does Yi 1.5 34B run on AMD Radeon RX 6800 XT?

Accepted Answer

Roughly 4 tokens/sec for Q8_0. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Question 4

What if AMD Radeon RX 6800 XT is not enough for Yi 1.5 34B?

Accepted Answer

Consider upgrading to Apple M4 Pro (48 GB VRAM) which fits the recommended 44 GB target. Or pick a smaller quantization to stay on your current card.

Quantization	File Size	Min VRAM	Rec VRAM	Context	Verdict	Estimated tok/s
Q4_K_M	20 GB	22 GB	24 GB	4K / 4K	Hybrid CPU+GPU	~7
Q8_0Best fit	36 GB	38 GB	44 GB	4K / 4K	Hybrid CPU+GPU	~4

Can I Run Yi 1.5 34B on AMD Radeon RX 6800 XT?

Share this matchup

Every Yi 1.5 34B quantization on AMD Radeon RX 6800 XT

Upgrade options that fit Yi 1.5 34B better