TheBloke

/

Llama-2-7B-Chat-GGML

like 851

PR is empty?

sorry my bad . Creating one . Some issue with checking out .
Cloning into 'Llama-2-7B-Chat-GGML'...
remote: Enumerating objects: 72, done.
remote: Counting objects: 100% (72/72), done.
remote: Compressing objects: 100% (70/70), done.
remote: Total 72 (delta 27), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (72/72), done.
^Cwarning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

Oh I never recommend you git clone a GGML repo! You'll get every single file, and you only need one. And using Git for large files results in storing twice as much data as needed, because Git LFS stores all the data twice - once in the filename, and again in the blobs directory. So if you git clone one of my GGML repos you'll end up with hundreds of gigabytes of data you don't ever use :)

Use wget to grab the file you want - or any download client.

Or if you want to do it from Python code, you can use hf_hub_download from huggingface_hub, like this:

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id='TheBloke/Llama-2-7B-Chat-GGML',  filename='llama-2-7b-chat.ggmlv3.q4_K_M.bin')

I should start adding that info to my GGML repos.

If you do plan to do inference from Python, you don't need to download the file manually at all - you can use the ctransformers Python library for GGML inference, and that automatically downloads the GGML file you specify. There's a link to ctransformers in the README.

Thanks for the update . Will do as intended .