Spaces:
Sleeping
Sleeping
Luke Stanley
commited on
Commit
•
83e4d57
1
Parent(s):
56e785c
Rename serverless test file, set default model to Phi 2 for test, removed jq install, and env vars that are set the same in utils.py already, ignore .cache in git
Browse files- .gitignore +1 -0
- runpod.dockerfile +3 -6
- test.sh → serverless_local_test.py +0 -0
.gitignore
CHANGED
@@ -1 +1,2 @@
|
|
1 |
.aider*
|
|
|
|
1 |
.aider*
|
2 |
+
.cache
|
runpod.dockerfile
CHANGED
@@ -19,16 +19,13 @@ RUN python3.11 -m pip install pytest cmake \
|
|
19 |
# Install llama-cpp-python (build with cuda)
|
20 |
ENV CMAKE_ARGS="-DLLAMA_CUBLAS=on"
|
21 |
RUN python3.11 -m pip install git+https://github.com/lukestanley/llama-cpp-python.git@expose_json_grammar_convert_function --upgrade --no-cache-dir --force-reinstall
|
22 |
-
RUN apt-get update; apt-get install jq -y
|
23 |
-
ADD runpod_handler.py .
|
24 |
|
|
|
25 |
ADD chill.py .
|
26 |
ADD utils.py .
|
27 |
ADD promptObjects.py .
|
28 |
|
29 |
-
|
30 |
-
|
31 |
-
ENV N_GPU_LAYERS=-1
|
32 |
-
ENV CONTEXT_SIZE=2048
|
33 |
CMD python3.11 -u /runpod_handler.py
|
34 |
|
|
|
19 |
# Install llama-cpp-python (build with cuda)
|
20 |
ENV CMAKE_ARGS="-DLLAMA_CUBLAS=on"
|
21 |
RUN python3.11 -m pip install git+https://github.com/lukestanley/llama-cpp-python.git@expose_json_grammar_convert_function --upgrade --no-cache-dir --force-reinstall
|
|
|
|
|
22 |
|
23 |
+
ADD runpod_handler.py .
|
24 |
ADD chill.py .
|
25 |
ADD utils.py .
|
26 |
ADD promptObjects.py .
|
27 |
|
28 |
+
ENV REPO_ID="TheBloke/phi-2-GGUF"
|
29 |
+
ENV MODEL_FILE="phi-2.Q2_K.gguf"
|
|
|
|
|
30 |
CMD python3.11 -u /runpod_handler.py
|
31 |
|
test.sh → serverless_local_test.py
RENAMED
File without changes
|