Commit
·
3c8477d
1
Parent(s):
dc1a22b
Update docker run command
Browse files
README.md
CHANGED
@@ -40,7 +40,6 @@ This model is compatible with **[Friendli Container](https://friendli.ai/product
|
|
40 |
- Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
|
41 |
- Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
|
42 |
- Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
|
43 |
-
- Install Hugging Face CLI with `pip install -U "huggingface_hub[cli]"`
|
44 |
|
45 |
### Preparing Personal Access Token
|
46 |
|
@@ -79,24 +78,14 @@ You should pass the container secret as an environment variable to run the conta
|
|
79 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
80 |
|
81 |
```sh
|
82 |
-
export MODEL_DIR=$PWD/FriendliAI--Mistral-7B-Instruct-v0.2-fp8
|
83 |
-
export FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET"
|
84 |
-
export FRIENDLI_CONTAINER_IMAGE="registry.friendli.ai/trial"
|
85 |
-
export GPU_ENUMERATION='"device=0"'
|
86 |
-
|
87 |
-
huggingface-cli download FriendliAI/Mistral-7B-Instruct-v0.2-fp8 \
|
88 |
-
--local-dir $MODEL_DIR \
|
89 |
-
--local-dir-use-symlinks False
|
90 |
-
|
91 |
docker run \
|
92 |
-
--gpus
|
93 |
-
-
|
94 |
-
-
|
95 |
-
|
96 |
-
|
97 |
-
--web-server-port
|
98 |
-
--
|
99 |
-
--ckpt-type hf_safetensors"
|
100 |
```
|
101 |
|
102 |
---
|
|
|
40 |
- Before you begin, make sure you have signed up for [Friendli Suite](https://suite.friendli.ai/). **You can use Friendli Containers free of charge for four weeks.**
|
41 |
- Prepare a Personal Access Token following [this guide](#preparing-personal-access-token).
|
42 |
- Prepare a Friendli Container Secret following [this guide](#preparing-container-secret).
|
|
|
43 |
|
44 |
### Preparing Personal Access Token
|
45 |
|
|
|
78 |
Once you've prepared the image of Friendli Container, you can launch it to create a serving endpoint.
|
79 |
|
80 |
```sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
docker run \
|
82 |
+
--gpus '"device=0"' \
|
83 |
+
-p 8000:8000 \
|
84 |
+
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
85 |
+
-e FRIENDLI_CONTAINER_SECRET="YOUR CONTAINER SECRET" \
|
86 |
+
registry.friendli.ai/trial \
|
87 |
+
--web-server-port 8000 \
|
88 |
+
--hf-model-name FriendliAI/Mistral-7B-Instruct-v0.2-fp8
|
|
|
89 |
```
|
90 |
|
91 |
---
|