Luke Stanley commited on
Commit
83e4d57
1 Parent(s): 56e785c

Rename serverless test file, set default model to Phi 2 for test, removed jq install, and env vars that are set the same in utils.py already, ignore .cache in git

Browse files
.gitignore CHANGED
@@ -1 +1,2 @@
1
  .aider*
 
 
1
  .aider*
2
+ .cache
runpod.dockerfile CHANGED
@@ -19,16 +19,13 @@ RUN python3.11 -m pip install pytest cmake \
19
  # Install llama-cpp-python (build with cuda)
20
  ENV CMAKE_ARGS="-DLLAMA_CUBLAS=on"
21
  RUN python3.11 -m pip install git+https://github.com/lukestanley/llama-cpp-python.git@expose_json_grammar_convert_function --upgrade --no-cache-dir --force-reinstall
22
- RUN apt-get update; apt-get install jq -y
23
- ADD runpod_handler.py .
24
 
 
25
  ADD chill.py .
26
  ADD utils.py .
27
  ADD promptObjects.py .
28
 
29
- #ENV REPO_ID="TheBloke/phi-2-GGUF"
30
- #ENV MODEL_FILE="phi-2.Q2_K.gguf"
31
- ENV N_GPU_LAYERS=-1
32
- ENV CONTEXT_SIZE=2048
33
  CMD python3.11 -u /runpod_handler.py
34
 
 
19
  # Install llama-cpp-python (build with cuda)
20
  ENV CMAKE_ARGS="-DLLAMA_CUBLAS=on"
21
  RUN python3.11 -m pip install git+https://github.com/lukestanley/llama-cpp-python.git@expose_json_grammar_convert_function --upgrade --no-cache-dir --force-reinstall
 
 
22
 
23
+ ADD runpod_handler.py .
24
  ADD chill.py .
25
  ADD utils.py .
26
  ADD promptObjects.py .
27
 
28
+ ENV REPO_ID="TheBloke/phi-2-GGUF"
29
+ ENV MODEL_FILE="phi-2.Q2_K.gguf"
 
 
30
  CMD python3.11 -u /runpod_handler.py
31
 
test.sh → serverless_local_test.py RENAMED
File without changes