Peeepy commited on
Commit
b68311b
·
1 Parent(s): 284ecd1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -1
README.md CHANGED
@@ -2,4 +2,12 @@
2
 
3
  The code to merge these can be found [here](https://files.catbox.moe/mg5v4g.py). Change information as needed.
4
 
5
- NOTE: This requires a monkey patch to work. FlashVenom has, along with kindly quantising this model to 4bit, added the monkeypatch file to their repo. You can access this [here](https://huggingface.co/flashvenom/Airoboros-13B-SuperHOT-8K-4bit-GPTQ).
 
 
 
 
 
 
 
 
 
2
 
3
  The code to merge these can be found [here](https://files.catbox.moe/mg5v4g.py). Change information as needed.
4
 
5
+ NOTE: This requires a monkey patch to work. FlashVenom has, along with kindly quantising this model to 4bit, added the monkeypatch file to their repo. You can access this [here](https://huggingface.co/flashvenom/Airoboros-13B-SuperHOT-8K-4bit-GPTQ).
6
+
7
+ FROM THE ORIGINAL LORA MODEL CARD:
8
+ This is a second prototype of SuperHOT, this time with 4K context and no RLHF. In my testing, it can go all the way to 6K without breaking down and I made the change with intention to reach 8K, so I'll assume it will go to 8K although I only trained on 4K sequences.
9
+
10
+ In order to use the 8K context, you will need to apply the monkeypatch I have added in this repo -- without it, it will not work. The patch is very simple, and you can make the changes yourself:
11
+
12
+ Increase the max_position_embeddings to 8192 to stretch the sinusoidal
13
+ Stretch the frequency steps by a scale of 0.25