paulilioaica commited on
Commit
18b67aa
·
verified ·
1 Parent(s): d7568e1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -18,7 +18,7 @@ base_model:
18
 
19
  ## Code is work in progress
20
 
21
- <img src="https://static.wixstatic.com/media/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg/v1/fill/w_600,h_400,al_c,q_80,usm_0.66_1.00_0.01,enc_auto/3eee0b_4d2513facc0f4a1cb21b2b8f4e1df179~mv2.jpg" />
22
 
23
  PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
24
  * [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
@@ -29,7 +29,7 @@ PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using me
29
  * [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
30
  * [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
31
  by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
32
-
33
 
34
 
35
  ## 🧩 Configuration
 
18
 
19
  ## Code is work in progress
20
 
21
+ <img src="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11201acc-4089-416d-921b-cbd71fbf8ddb_1024x1024.jpeg" width="500" class="center"/>
22
 
23
  PhiMiX-2x2B is a Mixure of Experts (MoE) made with the following models using mergekit:
24
  * [cognitivecomputations/dolphin-2_6-phi-2](https://huggingface.co/cognitivecomputations/dolphin-2_6-phi-2)
 
29
  * [mlabonne's phixtral](https://huggingface.co/mlabonne/phixtral-4x2_8) for the PhiConfig and inference code.
30
  * [mergekit](https://github.com/cg123/mergekit) code which I tweaked (you can find the PhiConfig [here](https://github.com/cg123/mergekit/blob/508348ae34be17ea0a95d0a288a6e34491a2558a/mergekit/architecture.py#L289))
31
  by mainly adding the config in the `moe_mixtral.py` script from `mixtral` branch.
32
+ * artwork by
33
 
34
 
35
  ## 🧩 Configuration