Q4_K_M
12 GBMin VRAM: 13.8 GB
Recommended VRAM: 15.6 GB
Min RAM: 18 GB
Context: 8K / 8K
Loading model details...
Fetching variants, compatibility details, and metadata.
Loading model details...
Fetching variants, compatibility details, and metadata.
Share Devstral Small 2 24B with someone who is deciding what to run locally.
Social proof
56% of 982 scanned PCs run Devstral Small 2 24B fully on GPU.
683 keep at least some work on GPU. Based on anonymous compatibility checks.
General-purpose local model brief
Best for
Consider alternatives if
Quantization tip: Benchmark at least two quantizations and validate with a task-specific eval set before production use.
New to local models? Smaller quantization variants are easier to run, while larger ones can improve quality at the cost of more memory.
Q4_K_M
12 GBMin VRAM: 13.8 GB
Recommended VRAM: 15.6 GB
Min RAM: 18 GB
Context: 8K / 8K
Q5_K_M
15 GBMin VRAM: 17.3 GB
Recommended VRAM: 19.5 GB
Min RAM: 23 GB
Context: 8K / 8K
Q8_0
24 GBMin VRAM: 27.6 GB
Recommended VRAM: 31.2 GB
Min RAM: 36 GB
Context: 8K / 8K
FP16
48 GBMin VRAM: 55.2 GB
Recommended VRAM: 62.4 GB
Min RAM: 72 GB
Context: 8K / 8K
| Quantization | File Size | Min VRAM | Recommended VRAM | Min RAM | Context |
|---|---|---|---|---|---|
| Q4_K_M | 12 GB | 13.8 GB | 15.6 GB | 18 GB | 8K / 8K |
| Q5_K_M | 15 GB | 17.3 GB | 19.5 GB | 23 GB | 8K / 8K |
| Q8_0 | 24 GB | 27.6 GB | 31.2 GB | 36 GB | 8K / 8K |
| FP16 | 48 GB | 55.2 GB | 62.4 GB | 72 GB | 8K / 8K |
These GPUs meet the recommended 15.6 GB VRAM for the Q4_K_M quantization. Estimated speeds are approximate and assume full GPU offloading.
Budget Pick
NVIDIA GeForce RTX 508016 GB VRAM · ~64 tok/s
Lowest cost that meets recommended VRAM
Check price on AmazonFastest Pick
NVIDIA GeForce RTX 509032 GB VRAM · ~119.5 tok/s
Highest estimated throughput
Check price on AmazonBest Value
NVIDIA GeForce RTX 5070 Ti16 GB VRAM · ~59.7 tok/s
Best speed per dollar of VRAM
Check price on AmazonNeed a detailed comparison? See all GPU rankings for Devstral Small 2 24B.