Update README.md
Browse files
README.md
CHANGED
@@ -15,15 +15,27 @@ tags:
|
|
15 |
- mergekit
|
16 |
- merge
|
17 |
|
|
|
|
|
18 |
---
|
19 |
# Moza-7B-v1.0
|
20 |
|
21 |
-
|
|
|
|
|
|
|
|
|
22 |
|
23 |
-
##
|
24 |
### Merge Method
|
25 |
|
26 |
-
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
|
28 |
### Models Merged
|
29 |
|
@@ -38,6 +50,17 @@ The following models were included in the merge:
|
|
38 |
* [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)
|
39 |
* [Intel/neural-chat-7b-v3-3](https://huggingface.co/Intel/neural-chat-7b-v3-3)
|
40 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
41 |
### Configuration
|
42 |
|
43 |
The following YAML configuration was used to produce this model:
|
|
|
15 |
- mergekit
|
16 |
- merge
|
17 |
|
18 |
+
|
19 |
+
|
20 |
---
|
21 |
# Moza-7B-v1.0
|
22 |
|
23 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/63474d73511cd17d2c790ed7/e7hw2xIzfpUseCFEOINg7.png)
|
24 |
+
|
25 |
+
This is a [meme-merge](https://en.wikipedia.org/wiki/Joke) of pre-trained language models,
|
26 |
+
created using [mergekit](https://github.com/cg123/mergekit).
|
27 |
+
Use at your own risk.
|
28 |
|
29 |
+
## Details
|
30 |
### Merge Method
|
31 |
|
32 |
+
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method,
|
33 |
+
using [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) as a base.
|
34 |
+
|
35 |
+
The value for `density` are from some blog I found,
|
36 |
+
and the weight was randomly generated and then assigned to the models,
|
37 |
+
with priority (of using the bigger weight) to `NeuralHermes`, `OpenOrca`, and `neural-chat`.
|
38 |
+
The models themselves are chosen by "vibes".
|
39 |
|
40 |
### Models Merged
|
41 |
|
|
|
50 |
* [meta-math/MetaMath-Mistral-7B](https://huggingface.co/meta-math/MetaMath-Mistral-7B)
|
51 |
* [Intel/neural-chat-7b-v3-3](https://huggingface.co/Intel/neural-chat-7b-v3-3)
|
52 |
|
53 |
+
### Prompt Format
|
54 |
+
|
55 |
+
You (probably) can use `Alpaca` formatting for inference
|
56 |
+
(I mean it's not like I actually tested it in the first place :joy:)
|
57 |
+
|
58 |
+
```
|
59 |
+
### Instruction:
|
60 |
+
|
61 |
+
### Response:
|
62 |
+
```
|
63 |
+
|
64 |
### Configuration
|
65 |
|
66 |
The following YAML configuration was used to produce this model:
|