Update README.md
Browse files
README.md
CHANGED
@@ -15,7 +15,7 @@ This updated model includes training for Pythagora GPT Pilot version 0.1.12 prom
|
|
15 |
[LoupGarou/deepseek-coder-6.7b-instruct-pythagora-v3-gguf](https://huggingface.co/LoupGarou/deepseek-coder-6.7b-instruct-pythagora-v3-gguf), is compatible with the following versions:
|
16 |
|
17 |
[GPT-Pilot (version: 0.1.12)](https://github.com/Pythagora-io/gpt-pilot/tree/4c1ffa957742c47419ab1aba7f5bf62b3f00bb90) and
|
18 |
-
[LM Studio (version: 0.2.21)](https://releases.lmstudio.ai/windows/0.2.
|
19 |
|
20 |
Please ensure you are using one of the above versions when working with this model to ensure proper functionality and compatibility.
|
21 |
|
@@ -23,11 +23,10 @@ Please ensure you are using one of the above versions when working with this mod
|
|
23 |
|
24 |
Many issues related to empty plans, tasks, circular questions, and poor model performance are related to the following parameters:
|
25 |
|
26 |
-
1. **Prompt eval batch size (n_batch)**: Impacts how the instruction is divided and sent to the LLM. To prevent empty tasks, plans, and circular questions, set this to match your Context Length (n_ctx). For example, if your **n_ctx = 8192** then set your prompt eval bacth size to match **n_batch = 8192**. Warning: If the n_batch < n_ctx then your model will give bad results.
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
3. **MAX_TOKENS (GPT Pilot .env)**: Sets the maximum tokens the OpenAI API generate in the output. Warning: Setting this value too low will result in truncated messages.
|
31 |
|
32 |
## Model Details
|
33 |
|
|
|
15 |
[LoupGarou/deepseek-coder-6.7b-instruct-pythagora-v3-gguf](https://huggingface.co/LoupGarou/deepseek-coder-6.7b-instruct-pythagora-v3-gguf), is compatible with the following versions:
|
16 |
|
17 |
[GPT-Pilot (version: 0.1.12)](https://github.com/Pythagora-io/gpt-pilot/tree/4c1ffa957742c47419ab1aba7f5bf62b3f00bb90) and
|
18 |
+
[LM Studio (version: 0.2.21)](https://releases.lmstudio.ai/windows/0.2.22/c/latest/LM-Studio-0.2.22-Setup.exe)
|
19 |
|
20 |
Please ensure you are using one of the above versions when working with this model to ensure proper functionality and compatibility.
|
21 |
|
|
|
23 |
|
24 |
Many issues related to empty plans, tasks, circular questions, and poor model performance are related to the following parameters:
|
25 |
|
26 |
+
1. **Prompt eval batch size (n_batch)**: LM Studio - Impacts how the instruction is divided and sent to the LLM. To prevent empty tasks, plans, and circular questions, set this to match your Context Length (n_ctx). For example, if your **n_ctx = 8192** then set your prompt eval bacth size to match **n_batch = 8192**. Warning: If the n_batch < n_ctx then your model will give bad results.
|
27 |
+
2. **Context Length (n_ctx)**: LM Studio - Sets the maximum length of the instruction and truncates the instruction once the limit is exceeded. Set this value to the maximum your hardware can handle and the maximum for the model. For example, DeepSeek Coder has a maximum token length of 16,384. Warning: GPT Pilot will often create instruction prompts 10,000 to 20,000 tokens in length which is why Pythagora-LLM-Proxy was created to permit toggling to higher capacity APIs such as OpenAI.
|
28 |
+
3. **System Prompt**: LM Studio - System Prompt must be set to DeepSeek Coder prompt: "You are an AI programming assistant, utilizing the DeepSeek Coder model, developed by DeepSeek Company, and you only answer questions related to computer science. For politically sensitive questions, security and privacy issues, and other non-computer science questions, you will refuse to answer."
|
29 |
+
4. **MAX_TOKENS (GPT Pilot .env)**: GPT Pilot - Sets the maximum tokens the OpenAI API generate in the output. Warning: Setting this value too low will result in truncated messages.
|
|
|
30 |
|
31 |
## Model Details
|
32 |
|