Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,7 @@ base_model:
|
|
18 |
|
19 |
## Code is work in progress
|
20 |
|
21 |
-
<img src="https://
|
22 |
|
23 |
PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
|
24 |
* [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
|
@@ -29,7 +29,7 @@ PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using me
|
|
29 |
* [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
|
30 |
* [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
|
31 |
by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
|
32 |
-
|
33 |
|
34 |
|
35 |
## 🧩 Configuration
|
|
|
18 |
|
19 |
## Code is work in progress
|
20 |
|
21 |
+
<img src="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11201acc-4089-416d-921b-cbd71fbf8ddb_1024x1024.jpeg" width="500" class="center"/>
|
22 |
|
23 |
PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
|
24 |
* [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
|
|
|
29 |
* [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
|
30 |
* [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
|
31 |
by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
|
32 |
+
* artwork by
|
33 |
|
34 |
|
35 |
## 🧩 Configuration
|