Hugging Face's logo

Spaces:
limcheekin
/
orca_mini_v3_13B-GGML
Paused

App Files Community
limcheekin commited on
Commit
a5da896
·
1 Parent(s): 73cc25e

feat: updated to 13B ggmlv3.q6_K model

Browse files
Files changed (2) hide show
  1. Dockerfile +1 -1
  2. start_server.sh +1 -1
Dockerfile CHANGED
@@ -14,7 +14,7 @@ RUN pip install -U pip setuptools wheel && \
14
 
15
  # Download model
16
  RUN mkdir model && \
17
- curl -L https://huggingface.co/TheBloke/orca_mini_v2_7B-GGML/resolve/main/orca-mini-v2_7b.ggmlv3.q4_0.bin -o model/ggml-model-q4_0.bin
18
 
19
  COPY ./start_server.sh ./start_server.sh
20
 
 
14
 
15
  # Download model
16
  RUN mkdir model && \
17
+ curl -L https://huggingface.co/TheBloke/h2ogpt-4096-llama2-13B-chat-GGML/blob/main/h2ogpt-4096-llama2-13b-chat.ggmlv3.q6_K.bin -o model/ggmlv3-model.bin
18
 
19
  COPY ./start_server.sh ./start_server.sh
20
 
start_server.sh CHANGED
@@ -3,4 +3,4 @@
3
  # For mlock support
4
  ulimit -l unlimited
5
 
6
- python3 -B -m llama_cpp.server --model model/ggml-model-q4_0.bin
 
3
  # For mlock support
4
  ulimit -l unlimited
5
 
6
+ python3 -B -m llama_cpp.server --model model/ggmlv3-model.bin --n_threads 2 --embedding False