- MOTH is a general chat AI.
- MOTH is finetuned on high quality synthetic data.
- MOTH is trained on a variety of skills and specialties.
- This version of MOTH is trained on the Llama 3.1 Instruct format.
- MOTH is also available for Gemma 2; more MOTH finetunes for other models to follow.
- MOTH has not been manually tested and uses automatically generated datasets.
- Do as you will.
(uses llama 3.1 license available at https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 20.37 |
| IFEval (0-Shot) | 52.08 |
| BBH (3-Shot) | 26.45 |
| MATH Lvl 5 (4-Shot) | 11.86 |
| GPQA (0-shot) | 2.57 |
| MuSR (0-shot) | 3.79 |
| MMLU-PRO (5-shot) | 25.48 |
- Downloads last month
- 24
Inference API (serverless) is not available, repository is disabled.
Model tree for sequelbox/Llama3.1-8B-MOTH
Base model
meta-llama/Meta-Llama-3.1-8B
Finetuned
meta-llama/Meta-Llama-3.1-8B-Instruct
Finetuned
this model
Dataset used to train sequelbox/Llama3.1-8B-MOTH
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard52.080
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard26.450
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard11.860
- acc_norm on GPQA (0-shot)Open LLM Leaderboard2.570
- acc_norm on MuSR (0-shot)Open LLM Leaderboard3.790
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard25.480