DeepSeek-R1-Distill-Qwen-7B

This repository contains quantized versions of the model from the original repository: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B.

Name	Quantization Method	Size (GB)
deepseek-r1-distill-qwen-7b.Q8_0.gguf	q8_0	7.54
deepseek-r1-distill-qwen-7b.Q4_0.gguf	q4_0	4.13

GGUF

Model size

7.62B params

Architecture

qwen2

4-bit

8-bit

Inference Examples

Inference API (serverless) has been turned off for this model.

Model tree for pbatra/DeepSeek-R1-Distill-Qwen-7B-GGUF

Base model

Quantized

(46)

this model