Create README.md

README.md

@@ -0,0 +1,71 @@
+Merge of [SuperHOT-LoRA-prototype](https://huggingface.co/kaiokendev/SuperHOT-LoRA-prototype) and [llama-30b](https://huggingface.co/huggyllama/llama-30b)
+
+
+Quantization:
+CUDA_VISIBLE_DEVICES=0 python llama.py ausboss/Llama30B-SuperHOT c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors Llama30B-SuperHOT-4bit-128g.safetensors
+
+Make sure to run with 128g and 4bit arguments in ooba or use the kobold fork that allows for 4bit.
+
+
+# From the SuperHot Page:
+
+## Prototypes for SuperHOT
+
+No guarantees for output quality, simply uploading what I have so others can play around with it. Not even sure if the rank in cutoff-8192 is correct (think it should be 10 maybe.. can't remember)
+
+All prototypes are extremely early epochs (sub 0.5)
+
+## Model/Training
+All trained with Flash Attention with conversation sequence lengths ranging from 8K to 16K tokens (No Alibi unless otherwise mentioned)
+
+All trained on LLaMa 13B 4-bit (no groupsize)
+
+(*Personally, I like the 8K cutoff version better, so I would say start with that one*)
+
+## Data
+A combination of various datasets and cleaned logs converted into datasets including but not limited to:
+- Bluemoon Fanbased
+- Roleplaying Guild
+- Community-sourced outputs
+- [Dan's PocketDoc/RUCAIBox-Story-Generation-Alpaca](https://huggingface.co/datasets/PocketDoc/RUCAIBox-Story-Generation-Alpaca)
+- [IlyaGusev/gpt_roleplay_realm](https://huggingface.co/datasets/IlyaGusev/gpt_roleplay_realm)
+- others
+
+## Bias
+SuperHOT is a fiction-focused model. No alignment has been performed on the training data. Be mindful that this model may output harmful, violent, or otherwise problematic content
+
+## Format
+Any format should work with such early checkpoints. However the training data is entirely in the following format:
+```
+---
+mode: chat
+characters:
+    <char1 name>: <descriptive tags for char1>
+    <char2 name>: <descriptive tags for char2>
+summary: <summary of the story thus far or the purpose of the chat> (optional)
+<any other miscellaneous data>
+---
+<chat history>
+```
+
+By "any other miscellaneous data", it means you should be able to put any additional metadata for the story or characters. I.e.,
+```
+...
+locations:
+    location1: <tags for location1>
+inventory:
+    item1: <tags for item1>
+```
+
+Again, format does not hold such a large weight on these early checkpoints. I have found success with the following setup for an RPG-like experience. Just play around with the format and see what works:
+```
+---
+mode: rpg
+characters:
+     You: a new player
+system: The system controls the RPG, handles character creation, world narration, and quest management. Also controls any NPCs and inventory tracking. Their first message provides a lengthy introduction for the player into the RPG world they are about to play in. After completing the character creation, the system will give a lengthy introduction into the world of ___. The first quest will follow right after
+rpg setting: The world of ___
+rpg rules: Any rules typical of RPG games, including typical items, battle stats, etc
+---
+```
+