Attempt to clarify how hosted API ≠ local endpoint (#373)
Browse files
README.md
CHANGED
@@ -55,11 +55,11 @@ docker run -d -p 27017:27017 --name mongo-chatui mongo:latest
|
|
55 |
|
56 |
In which case the url of your DB will be `MONGODB_URL=mongodb://localhost:27017`.
|
57 |
|
58 |
-
Alternatively, you can use a [free MongoDB Atlas](https://www.mongodb.com/pricing) instance for this, Chat UI should fit comfortably within
|
59 |
|
60 |
### Hugging Face Access Token
|
61 |
|
62 |
-
You will need a Hugging Face access token to run Chat UI locally,
|
63 |
|
64 |
## Launch
|
65 |
|
@@ -152,7 +152,11 @@ You can change things like the parameters, or customize the preprompt to better
|
|
152 |
|
153 |
#### Running your own models using a custom endpoint
|
154 |
|
155 |
-
If you want to,
|
|
|
|
|
|
|
|
|
156 |
|
157 |
```
|
158 |
|
|
|
55 |
|
56 |
In which case the url of your DB will be `MONGODB_URL=mongodb://localhost:27017`.
|
57 |
|
58 |
+
Alternatively, you can use a [free MongoDB Atlas](https://www.mongodb.com/pricing) instance for this, Chat UI should fit comfortably within their free tier. After which you can set the `MONGODB_URL` variable in `.env.local` to match your instance.
|
59 |
|
60 |
### Hugging Face Access Token
|
61 |
|
62 |
+
You will need a Hugging Face access token to run Chat UI locally, if you use a remote inference endpoint. You can get one from [your Hugging Face profile](https://huggingface.co/settings/tokens).
|
63 |
|
64 |
## Launch
|
65 |
|
|
|
152 |
|
153 |
#### Running your own models using a custom endpoint
|
154 |
|
155 |
+
If you want to, instead of hitting models on the Hugging Face Inference API, you can run your own models locally.
|
156 |
+
|
157 |
+
A good option is to hit a [text-generation-inference](https://github.com/huggingface/text-generation-inference) endpoint. This is what is done in the official [Chat UI Spaces Docker template](https://huggingface.co/new-space?template=huggingchat/chat-ui-template) for instance: both this app and a text-generation-inference server run inside the same container.
|
158 |
+
|
159 |
+
To do this, you can add your own endpoints to the `MODELS` variable in `.env.local`, by adding an `"endpoints"` key for each model in `MODELS`.
|
160 |
|
161 |
```
|
162 |
|