| license: apache-2.0 | |
| language: | |
| - en | |
| - ru | |
| ## Description | |
| This is an instruction following model (based on Mistral v0.1 Base) optimized for Russian language. | |
| It was trained using [kolibrify](https://github.com/oKatanaaa/kolibrify) on a multitude of instruction datasets. | |
| The model uses ChatML template. Adding a system prompt will likely improve the model's performance on your task (experiment with it). | |
| I recommend using the model with LMStudio. | |
| Currently in pre-alpha, later releases will include more details regarding training procedure and data mix. | |
| ## Instruction following evals | |
| The model was tested using the following benchmarks: | |
| - [ruIFEval](https://github.com/NLP-Core-Team/ruIFEval) | |
| - [ifeval](https://github.com/google-research/google-research/tree/master/instruction_following_eval) | |
| | Eval name |Strict Value| Loose Value | |
| |---------------------------------|----|----| | |
| |Avg. |*49.92*|*52.52*| | |
| |ifeval-prompt-level |49.72|52.31| | |
| |ifeval-instruction-level |60.91|63.18| | |
| |ru-ifeval-prompt-level |39.55|42.32| | |
| |ru-ifeval-instruction-level |49.52|52.27| | |