Hugging Face's logo

Ranjanunicode
/
unicode-llama-2-chat-Hf-q4-gguf

GGUF
English
Model card Files Community
unicode-llama-2-chat-Hf-q4-gguf / README.md
Ranjanunicode's picture
Ranjanunicode
Update README.md
576c997 verified
preview code
|
raw
history blame
No virus
2.47 kB
metadata
license: llama2
language:
  - en
metrics:
  - perplexity

Model Card for Model ID

Model Developers Ranjanunicode

A quantized version of Llama 2 chat hugging face that can be implemented with minimal hardware requirements.

Input Models input text only.

Output Models generate text only.

Model Details

Model Description

  • Developed by: Ranjan Pandit
  • Model type: Quantized version of "meta-llama/Llama-2-7b-chat-hf"
  • Finetuned from model [optional]: "meta-llama/Llama-2-7b-chat-hf"

Model Sources [optional]

Uses

  • Intended Use Cases unicode-llama-2-chat-Hf-q4-2 is intended for commercial and research use in English.

  • Tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.

  • To get the expected features and performance for the chat versions, a specific formatting needs to be followed,including the INST and <> tags, BOS and EOS tokens, and the whitespaces and breaklines in between (we recommend calling strip() on inputs to avoid double-spaces). See our reference code in github for details: chat_completion.

  • Just Install ctransformers:

!pip install ctransformers>=0.2.24

  • Use the following to get started.

  •

from ctransformers import AutoModelForCausalLM

#Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system. llm = AutoModelForCausalLM.from_pretrained("Ranjanunicode/unicode-llama-2-chat-Hf-q4-2", model_file="unicode-llama-2-chat-Hf-q4-2.gguf", model_type="llama", gpu_layers=40)

print(llm("AI is going to"))


### Out-of-Scope Use

- Out-of-scope Uses Use in any manner that violates applicable laws or regulations (including trade compliance laws).Use in languages other than English. Use in any other way that is prohibited by the Acceptable Use Policy and Licensing Agreement for Llama 2.


### Compute Infrastructure
- Google collab Tesla T4 Gpu.

[More Information Needed]


## Citation [optional]

- Meta 
- Meta LLama
- "https://arxiv.org/abs/2310.19102"

## Model Card Authors [optional]

- Ranjan

## Model Card Contact
- "[email protected]"