Commit
·
6099b48
1
Parent(s):
d9b2258
Update README.md
Browse files
README.md
CHANGED
|
@@ -81,9 +81,10 @@ There are many factors to consider when choosing a university. Here are some tip
|
|
| 81 |
```bash
|
| 82 |
git clone https://github.com/neuralmagic/sparseml
|
| 83 |
pip install -e "sparseml[transformers]"
|
|
|
|
| 84 |
python sparseml/src/sparseml/transformers/sparsification/obcq/obcq.py TinyLlama/TinyLlama-1.1B-Chat-v0.4 open_platypus --recipe recipe.yaml --save True
|
| 85 |
python sparseml/src/sparseml/transformers/sparsification/obcq/export.py --task text-generation --model_path obcq_deployment
|
| 86 |
cp deployment/model.onnx deployment/model-orig.onnx
|
| 87 |
-
wget https://huggingface.co/nm-testing/TinyLlama-1.1B-Chat-v0.4-pruned50-quant/raw/main/
|
| 88 |
python onnx_kv_inject.py --input-file deployment/model-orig.onnx --output-file deployment/model.onnx
|
| 89 |
```
|
|
|
|
| 81 |
```bash
|
| 82 |
git clone https://github.com/neuralmagic/sparseml
|
| 83 |
pip install -e "sparseml[transformers]"
|
| 84 |
+
wget https://huggingface.co/nm-testing/TinyLlama-1.1B-Chat-v0.4-pruned50-quant/raw/main/recipe.yaml # download recipe
|
| 85 |
python sparseml/src/sparseml/transformers/sparsification/obcq/obcq.py TinyLlama/TinyLlama-1.1B-Chat-v0.4 open_platypus --recipe recipe.yaml --save True
|
| 86 |
python sparseml/src/sparseml/transformers/sparsification/obcq/export.py --task text-generation --model_path obcq_deployment
|
| 87 |
cp deployment/model.onnx deployment/model-orig.onnx
|
| 88 |
+
wget https://huggingface.co/nm-testing/TinyLlama-1.1B-Chat-v0.4-pruned50-quant/raw/main/onnx_kv_inject.py # kv_cache file
|
| 89 |
python onnx_kv_inject.py --input-file deployment/model-orig.onnx --output-file deployment/model.onnx
|
| 90 |
```
|