Nikolai1902 commited on
Commit
af49ad5
1 Parent(s): 3b83ab9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -1,3 +1,17 @@
1
  ---
2
  license: wtfpl
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: wtfpl
3
+ pipeline_tag: text-generation
4
  ---
5
+
6
+ This model was a QLoRA of LLaMA 2-13B base finetuned on a desuarchive dump of the 4channel /mlp/ board and then merged with the base model (as most GGML loading apps don't support LoRAs), and quantized for llama.cpp-based frontends. Was trained with 1024 context length
7
+
8
+
9
+ There are two options, depending on the resources you have:
10
+ - Q5_K_M: Low quality loss K-quantized 5 bits model. Max RAM consumption is 11.73 GB, recommended if you have 12GB of VRAM to load 40 layers
11
+ - Q4_K_S: Compact K-quantized 4 bits. Max RAM consumption is 9.87 GB
12
+
13
+ This not an instruction tuned model, it was trained on raw text, so treat it like an autocomplete.
14
+
15
+ Specifically, the dataset was a dump of all the board's posts, from the time of its creation to about late 2019. Prompting it appropriately will cause it to write greentext.
16
+
17
+