michaelfeil commited on
Commit
2cf07a5
1 Parent(s): 2ac4b1b

Add infinity / docker example to readme

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -3053,6 +3053,15 @@ const similarities = document_embeddings.map(x => dot(source_embeddings, x));
3053
  console.log(similarities); // [0.25145517380846977, 0.3865060421197194]
3054
  ```
3055
 
 
 
 
 
 
 
 
 
 
3056
 
3057
  ## FAQ
3058
 
 
3053
  console.log(similarities); // [0.25145517380846977, 0.3865060421197194]
3054
  ```
3055
 
3056
+ ## Using Infinity
3057
+
3058
+ OpenAI compatible API deployment with [Infinity](https://github.com/michaelfeil/infinity) and Docker.
3059
+
3060
+ ```bash
3061
+ docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
3062
+ michaelf34/infinity:0.0.70 \
3063
+ v2 --model-id Snowflake/snowflake-arctic-embed-l --dtype float16 --batch-size 32 --engine torch --port 7997
3064
+ ```
3065
 
3066
  ## FAQ
3067