Intel
/

gemma-2b-int4-inc

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

weiweiz1 commited on 25 days ago

Commit

d2f5d11

•

1 Parent(s): fdbb37d

Update README.md

Files changed (1) hide show

README.md +1 -3

README.md CHANGED Viewed

@@ -98,10 +98,8 @@ python3 main.py \
 --group_size 128 \
 --bits 4 \
 --iters 400 \
---use_quant_input \
---model_dtype "float16"
 --deployment_device 'gpu' \
---disable_low_gpu_mem_usage \
 --output_dir "./tmp_autoround"
 ```

 --group_size 128 \
 --bits 4 \
 --iters 400 \
+--model_dtype "float16" \
 --deployment_device 'gpu' \
 --output_dir "./tmp_autoround"
 ```