File size: 1,168 Bytes
3f47d0b 5990049 3f47d0b 5990049 3f47d0b 5990049 3f47d0b 5990049 3f47d0b 5990049 3f47d0b 5990049 3f47d0b 5990049 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
---
license: apache-2.0
language:
- en
- ru
---
## Description
This is an instruction following model (based on Mistral v0.1 Base) optimized for Russian language.
It was trained using [kolibrify](https://github.com/oKatanaaa/kolibrify) on a multitude of instruction datasets.
The model uses ChatML template. It was trained to be sensitive to the system prompt, experiment with it.
Currently in pre-alpha, later releases will include more details regarding training procedure and data mix.
> [!NOTE]
> This model is an improved version of older kolibri-mistral-0427.
## Instruction following evals
The model was tested using the following benchmarks:
- [ruIFEval](https://github.com/NLP-Core-Team/ruIFEval)
- [ifeval](https://github.com/google-research/google-research/tree/master/instruction_following_eval)
| Eval name |Strict Value| Loose Value
|---------------------------------|----|----|
|Avg. |*53.81*|*56.57*|
|ifeval-prompt-level |52.68|56.19|
|ifeval-instruction-level |62.82|66.18|
|ru-ifeval-prompt-level |44.36|46.39|
|ru-ifeval-instruction-level |55.39|57.55|
|