Nikolai1902
commited on
Commit
•
af49ad5
1
Parent(s):
3b83ab9
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,17 @@
|
|
1 |
---
|
2 |
license: wtfpl
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: wtfpl
|
3 |
+
pipeline_tag: text-generation
|
4 |
---
|
5 |
+
|
6 |
+
This model was a QLoRA of LLaMA 2-13B base finetuned on a desuarchive dump of the 4channel /mlp/ board and then merged with the base model (as most GGML loading apps don't support LoRAs), and quantized for llama.cpp-based frontends. Was trained with 1024 context length
|
7 |
+
|
8 |
+
|
9 |
+
There are two options, depending on the resources you have:
|
10 |
+
- Q5_K_M: Low quality loss K-quantized 5 bits model. Max RAM consumption is 11.73 GB, recommended if you have 12GB of VRAM to load 40 layers
|
11 |
+
- Q4_K_S: Compact K-quantized 4 bits. Max RAM consumption is 9.87 GB
|
12 |
+
|
13 |
+
This not an instruction tuned model, it was trained on raw text, so treat it like an autocomplete.
|
14 |
+
|
15 |
+
Specifically, the dataset was a dump of all the board's posts, from the time of its creation to about late 2019. Prompting it appropriately will cause it to write greentext.
|
16 |
+
|
17 |
+
|