README.md

---
license: apache-2.0
language:
- en
---
# Model Card for Model ID

This is Meta's Llama 2 7B quantized in 2-bit using AutoGPTQ from Hugging Face Transformers. 
## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** [The Kaitchup](https://kaitchup.substack.com/)
- **Model type:** Causal (Llama 2)
- **Language(s) (NLP):** English
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), [Llama 2 license agreement](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)

### Model Sources

The method and code used to quantize the model are explained here:
[Quantize and Fine-tune LLMs with GPTQ Using Transformers and TRL](https://kaitchup.substack.com/p/quantize-and-fine-tune-llms-with)

## Uses

This model is pre-trained and not fine-tuned. You may fine-tune it with PEFT using adapters.
Note that the 2-bit quantization significantly decreases the performance of Llama 2.


## Other versions

- [kaitchup/Llama-2-7b-gptq-4bit](https://huggingface.co/kaitchup/Llama-2-7b-gptq-4bit)
- [kaitchup/Llama-2-7b-gptq-3bit](https://huggingface.co/kaitchup/Llama-2-7b-gptq-3bit)




## Model Card Contact

[The Kaitchup](https://kaitchup.substack.com/)