llemma_7b_muinstruct_camelmath is an instruction-following finetune of Llemma 7B, trained on the μInstruct and camel-ai/math datasets.
Input Formatting
Format input queries as follows:
input_text = f"Input:{input}\n\nResponse:"
Note that due to an error during training, this model's end-of-sequence token ID is 0 instead of the 2 which is standard for Llama-2 based models. Inference APIs should handle this automatically by reading this repo's config.json, but be aware of this difference if you are doing token surgery.
Evals
llemma_7b_muinstruct_camelmath compares favorably to other 7B parameter models on the Hungarian Math Exam. It surpasses the few-shot performance of Llemma 7B whilst being the strongest Llama-2 7B based model.
| Model | Exam Score |
|---|---|
| Code Llama 7B (few-shot) | 8% |
| MetaMath 7B | 20% |
| MAmmoTH 7B | 17% |
| MAmmoTH Coder 7B | 11% |
| Llemma 7B (few-shot) | 23% |
| Llemma_7B_muinstruct_camelmath | 25% |
| - | - |
| Mistral 7B (few-shot) | 22% |
| MetaMath Mistral 7B | 29% |
| OpenChat 3.5 | 37% |
- Downloads last month
- 462
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.