Huge thanks to Unsloth and the Huggingface TRL library.
This model is Qwen 2.5 14B fine tuned for a full epoch on the high quality garage-bAInd/Open-Platypus dataset for STEM reasoning.
| Training Detail | Value |
|---|---|
| Epochs | 1 |
| Steps | 2077 |
| Loss | 0.4218 |
| Batch size | 4 |
| Gradient Acc. Steps | 3 |
| Learning Rate | 2e-4 |
| LR Scheduler | cosine |
| Rank | 32 |
| Rank-Stabilized LoRA | Yes |
| Warm up steps | 5 |
| Weight Decay | 0.01 |
| Seed | 3407 |
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 35.46 |
| IFEval (0-Shot) | 59.81 |
| BBH (3-Shot) | 47.75 |
| MATH Lvl 5 (4-Shot) | 23.11 |
| GPQA (0-shot) | 16.00 |
| MuSR (0-shot) | 17.95 |
| MMLU-PRO (5-shot) | 48.12 |
