Q3_K_M
13 GBMin VRAM: 15 GB
Recommended VRAM: 18 GB
Min RAM: 18 GB
Context: 8K / 8K
Loading model details...
Fetching variants, compatibility details, and metadata.
Loading model details...
Fetching variants, compatibility details, and metadata.
New to local models? Smaller quantization variants are easier to run, while larger ones can improve quality at the cost of more memory.
Q3_K_M
13 GBMin VRAM: 15 GB
Recommended VRAM: 18 GB
Min RAM: 18 GB
Context: 8K / 8K
Q4_K_M
16 GBMin VRAM: 18 GB
Recommended VRAM: 24 GB
Min RAM: 20 GB
Context: 8K / 8K
Q8_0
28.7 GBMin VRAM: 30 GB
Recommended VRAM: 36 GB
Min RAM: 36 GB
Context: 8K / 8K
| Quantization | File Size | Min VRAM | Recommended VRAM | Min RAM | Context |
|---|---|---|---|---|---|
| Q3_K_M | 13 GB | 15 GB | 18 GB | 18 GB | 8K / 8K |
| Q4_K_M | 16 GB | 18 GB | 24 GB | 20 GB | 8K / 8K |
| Q8_0 | 28.7 GB | 30 GB | 36 GB | 36 GB | 8K / 8K |