quantizationsub4 bit quantized m

quantization - sub-4 bit quantized model on nvidia gpu - Stack Overflow

I was trying to run deepseek-r1-distill-llama70b-bf16.gguf (131gb on disk) on two A6000 gpus (48gb vram

quantizationsub4 bit quantized model on nvidia gpuStack Overflow

admin 23天前

19 0