Hugging Face's logo

Moses25
/
Llama-3-8B-chat-32K

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
Moses25 commited on
Commit
df632e1
·
verified ·
1 Parent(s): ff06b08

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -77,6 +77,7 @@ model_path = Llama-3-8B-chat-32K
77
  python -m vllm.entrypoints.openai.api_server --model=$model_path \
78
  --trust-remote-code --host 0.0.0.0 --port 7777 \
79
  --gpu-memory-utilization 0.8 \
 
80
  --max-model-len 8192 --chat-template llama3-chat-template.jinja \
81
  --tensor-parallel-size 1 --served-model-name chatbot
82
  ```
 
77
  python -m vllm.entrypoints.openai.api_server --model=$model_path \
78
  --trust-remote-code --host 0.0.0.0 --port 7777 \
79
  --gpu-memory-utilization 0.8 \
80
+ --enforce_eager \
81
  --max-model-len 8192 --chat-template llama3-chat-template.jinja \
82
  --tensor-parallel-size 1 --served-model-name chatbot
83
  ```