metadata

license: apache-2.0
base_model:
  - mistralai/Mistral-Nemo-Instruct-2407
tags:
  - roleplay
  - conversational
language:
  - en

Teleut 7b RP

A roleplay-focused LoRA finetune of Mistral Nemo Instruct. Methodology and hyperparams inspired by SorcererLM and Slush.

Dataset

The worst mix of data you've ever seen. Like, seriously, you do not want to see the things that went into this model. It's bad.

Recommended Settings

Chat template: Mistral v3-Tekken
Recommended samplers (not the be-all-end-all, try some on your own!):

Temp 1.25 / MinP 0.1
Temp 1.03 / TopK 200 / MinP 0.05 / TopA 0.2

Hyperparams

General

Epochs = 2
LR = 6e-5
LR Scheduler = Cosine
Optimizer = Paged AdamW 8bit
Effective batch size = 12

LoRA

Rank = 16
Alpha = 32
Dropout = 0.25 (Inspiration: Slush)

Credits

Humongous thanks to the people who created the data. I would credit you all, but that would be cheating ;)
Big thanks to all Allura members, especially Toasty, for testing and emotional support ilya /platonic
NO thanks to Infermatic. They suck at hosting models