GGUF models of the following model : https://huggingface.co/mridul3301/BioMistral-7B-finetuned

3 format of quantization:

  1. fp8
  2. fp16
  3. fp32

Converted the safetensors to GGUF for inference in CPU using llama_cpp

Downloads last month
19
GGUF
Model size
7.24B params
Architecture
llama
Inference Providers NEW
This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.