xzuyn commited on
Commit
444dadd
·
verified ·
1 Parent(s): 77d6df1

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +59 -0
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - not-for-all-audiences
4
+ ---
5
+ Refer to the original models for best usage.
6
+
7
+ - [Sao10K/Fimbulvetr-11B-v2](https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)
8
+
9
+ - [KoboldAI/Mistral-7B-Holodeck-1](https://huggingface.co/KoboldAI/Mistral-7B-Holodeck-1)
10
+
11
+ - [KoboldAI/Mistral-7B-Erebus-v3](https://huggingface.co/KoboldAI/Mistral-7B-Erebus-v3)
12
+
13
+ - [senseable/WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
14
+
15
+ ---
16
+
17
+ # Mergekit Recipe
18
+ ```yaml
19
+ # Mistral Merging
20
+ merge_method: linear
21
+ dtype: float16
22
+ models:
23
+ - model: KoboldAI/Mistral-7B-Holodeck-1
24
+ parameters:
25
+ weight: 0.45
26
+ - model: KoboldAI/Mistral-7B-Erebus-v3
27
+ parameters:
28
+ weight: 0.45
29
+ - model: senseable/WestLake-7B-v2
30
+ parameters:
31
+ weight: 0.1
32
+ name: MV01-Holodeck-Erebus-Westlake-7B
33
+ ---
34
+
35
+ # Mistral Stacking
36
+ merge_method: passthrough
37
+ dtype: float16
38
+ slices:
39
+ - sources:
40
+ - model: MV01-Holodeck-Erebus-Westlake-7B
41
+ layer_range: [0, 24]
42
+ - sources:
43
+ - model: MV01-Holodeck-Erebus-Westlake-7B
44
+ layer_range: [8, 32]
45
+ name: MV01-Stacked-Holodeck-Erebus-Westlake-Unhealed-10.7B
46
+ ---
47
+
48
+ # Mistral Stack Healing
49
+ merge_method: task_arithmetic
50
+ dtype: float16
51
+ base_model: Undi95/Mistral-11B-v0.1
52
+ models:
53
+ - model: Sao10K/Fimbulvetr-11B-v2
54
+ parameters:
55
+ weight: 1.0
56
+ - model: MV01-Stacked-Holodeck-Erebus-Westlake-Unhealed-10.7B
57
+ parameters:
58
+ weight: 1.0
59
+ ```