|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
base_model: |
|
- mistralai/Mistral-7B-Instruct-v0.3 |
|
pipeline_tag: text2text-generation |
|
library_name: unsloth |
|
--- |
|
|
|
# ModelCard for UnGPT-v1 |
|
|
|
## Model Details |
|
- **Name:** UnGPT-v1 |
|
- **Foundation Model:** Mistral v0.3 (7B parameters) |
|
- **Recommended Context Length:** 16k tokens |
|
- **Fine-tuning Methodology:** LoRA-based training with Odds Ratio Preference Optimization method, using a combination of ebooks and synthetic data. |
|
|
|
## Usage Instructions |
|
Use the Alpaca format for prompts: |
|
``` |
|
### Instruction: |
|
{instruction} |
|
|
|
### Input: |
|
{input} |
|
|
|
### Response: |
|
``` |
|
**Example prompts** |
|
|
|
For instructions, it is not recommended to deviate from the provided examples. For the input, a minimum is 10 sentences, but more can work as the model can handle longer context sizes (Thanks to the Mistral 7B v0.3 base model). |
|
|
|
1. **Completion Prompt:** |
|
``` |
|
### Instruction: |
|
Continue writing the story while retaining writing style. Write about 10 sentences. |
|
|
|
### Input: |
|
It was a dark and stormy night... |
|
|
|
### Response: |
|
``` |
|
|
|
2. **Fill-in-the-middle Prompt:** |
|
``` |
|
### Instruction: |
|
Fill in the missing part of the story ({{FILL_ME}}) with about 10 sentences while retaining the writing style. |
|
|
|
### Input: |
|
The bus was speeding down the road, cops chasing after it. |
|
{{FILL_ME}} |
|
She woke up to find herself in an unfamiliar room... |
|
|
|
### Response: |
|
``` |
|
|
|
## Dataset Preparation |
|
|
|
For dataset acquisition and cleanup please refer steps 1 and 2 of my text-completion example, [molbal/llm-text-completion-finetune](https://github.com/molbal/llm-text-completion-finetune/). |
|
|
|
Chunking: Split texts into chunks based on sentence boundaries, aiming for 100 sentences per example. |
|
- For completion examples, 90 sentences were used as input, 10 sentences as response. |
|
- For fill-in-the-middle examples, 80 + 10 sentences as input (before and after the {{FILL_ME}} placeholder, respectively), and 10 sentences as response. |
|
|
|
The beauty of the ORPO method is that for a single prompt we can set both a positive and a negative example. I wanted the model to avoid 'GPTisms' so I had gpt4o-mini generate answers both for completion and FOM tasks and added them as a neative example. |
|
|
|
The dataset used is ~15k examples, each approximately 9000 characters long including input, accepted and refused response. (Note these are characters not tokens) |
|
|
|
## Training setup |
|
|
|
- Fine-tuned the Mistral v0.3 foundation model using Unsloth and ORPO trainer. |
|
|
|
- Training configuration: |
|
- Batch size: 1 |
|
- Gradient accumulation steps: 4 |
|
- Learning rate scheduler type: Linear |
|
- Optimizer: AdamW (8-bit) |
|
- Number of training epochs: 1 |
|
|
|
- Hardware |
|
- I used GPU accelerated containers from the provider vast.ai (My referral link: https://cloud.vast.ai/?ref_id=123492 ) and executed training for ~8 hours on a single RTX 4090. |
|
|
|
- Training costs |
|
- ~5€ for renting a GPU pod (+15€ in unsuccessful attempts) |
|
- ~5€ in OpenAI API costs for generating refusals |
|
|
|
**Licensing and Citation** |
|
|
|
- **License:** This model is licensed under the Apache License 2.0. |
|
- **Citation:** |
|
```bibtex |
|
@misc{ungpt-v1, |
|
author = Bálint Molnár-Kaló, |
|
title = {UnGPT-v1: A Fine-tuned Mistral Model for Story Continuation}, |
|
howpublished = {\url{https://huggingface.co/models/molbal/UnGPT-v1}}, |
|
year = 2024 |
|
} |
|
``` |
