zattio770/120-Days-of-LORA-v2-13B

Model Description

A LORA trained on the book 120 Days of Sodom. It Is intended for heavy NSFW content. Many users might find it disturbing.

• Developed by: Zattio770
• Model type: large language model LORA
• License:
• Finetuned from model [optional]: MythoMax-L2-13b, Llama-13B

Uses

Direct Use

This model is finetuned for heavy NSFW roleplay, while can still an assistant. (Might not be very helpfull)

Out-of-Scope Use

Be safe, I am not responsible for what this lora outputs. Do anything you want. I don't care

Bias, Risks, and Limitations

Model might have bias to NSFW due to the large % of NSFW data in the training set.

Training Details

Training Data

raw text The 120 Days of Sodom by Marquis de Sade(1904)

Training Procedure

4bit MythoMaxL2 Rank: 128 Alpha: 256 Batch size: 128 Micro Batch: 1 Cutoff: 256 Epoch: 3 Learning Rate: 3e-4

Overlap Length: 64 Prefer Newline: 128

Training Hyperparameters

• Training regime: BF16, QLoRA, constant LR 5e-5

Compute Infrastructure

Hardware Type: 3080Ti Hours used: 10 OogaBooga