Q: What quantization of Mistral Small 3.1 24B should I use on a NVIDIA GeForce GTX 1080?

For 8 GB VRAM on the NVIDIA GeForce GTX 1080, the FP16 variant is the best fit. Estimated ~2 tokens/sec on the FP16 quantization.

Q: How fast does Mistral Small 3.1 24B run on NVIDIA GeForce GTX 1080?

Roughly 2 tokens/sec for FP16. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Q: What if NVIDIA GeForce GTX 1080 is not enough for Mistral Small 3.1 24B?

Consider upgrading to Apple M1 Max (64 GB VRAM) which fits the recommended 54 GB target. Or pick a smaller quantization to stay on your current card.

Question 1

Can I run Mistral Small 3.1 24B on a NVIDIA GeForce GTX 1080?

Accepted Answer

Sort of — NVIDIA GeForce GTX 1080 can run Mistral Small 3.1 24B (FP16) only by spilling layers to RAM. Generation will be slow. CPU + GPU hybrid — not enough VRAM (8 GB < 48 GB min), but 64 GB RAM is sufficient. Expect significantly slower inference.

Question 2

What quantization of Mistral Small 3.1 24B should I use on a NVIDIA GeForce GTX 1080?

Accepted Answer

For 8 GB VRAM on the NVIDIA GeForce GTX 1080, the FP16 variant is the best fit. Estimated ~2 tokens/sec on the FP16 quantization.

Question 3

How fast does Mistral Small 3.1 24B run on NVIDIA GeForce GTX 1080?

Accepted Answer

Roughly 2 tokens/sec for FP16. Real speed depends on context length, backend (Ollama, llama.cpp, LM Studio), and KV cache size.

Question 4

What if NVIDIA GeForce GTX 1080 is not enough for Mistral Small 3.1 24B?

Accepted Answer

Consider upgrading to Apple M1 Max (64 GB VRAM) which fits the recommended 54 GB target. Or pick a smaller quantization to stay on your current card.

Quantization	File Size	Min VRAM	Rec VRAM	Context	Verdict	Estimated tok/s
Q4_K_M	12.6 GB	14 GB	16 GB	8K / 128K	Hybrid CPU+GPU	~7
FP16Best fit	45.6 GB	48 GB	54 GB	8K / 128K	Hybrid CPU+GPU	~2

Can I Run Mistral Small 3.1 24B on NVIDIA GeForce GTX 1080?

Share this matchup

Every Mistral Small 3.1 24B quantization on NVIDIA GeForce GTX 1080

Upgrade options that fit Mistral Small 3.1 24B better