Quantization
Collection
A collection of quantized models. All the models can be fine-tuned by adding a LoRA Adapter.
•
83 items
•
Updated
•
3
The Quantized E5-mistral-7b-instruct Model
Original Base Embedding Model: intfloat/e5-mistral-7b-instruct.
Link: https://huggingface.co/intfloat/e5-mistral-7b-instruct
Quantization Configurations
"quantization_config": {
"bits": 4,
"checkpoint_format": "gptq",
"damp_percent": 0.01,
"desc_act": true,
"group_size": 128,
"model_file_base_name": null,
"model_name_or_path": null,
"quant_method": "gptq",
"static_groups": false,
"sym": true,
"true_sequential": true
},
Source Codes
Source Codes: https://github.com/vkola-lab/medpodgpt/tree/main/quantization.
- Downloads last month
- 101
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.