shadowml commited on
Commit
f0ae750
1 Parent(s): f4a5993

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -47
README.md CHANGED
@@ -1,62 +1,59 @@
1
  ---
2
- {}
 
 
 
 
 
 
3
  ---
4
- ---
5
- license: apache-2.0
6
- tags:
7
- - moe
8
- - mergekit
9
- - merge
10
- - mlabonne/Marcoro14-7B-slerp
11
- - beowolx/CodeNinja-1.0-OpenChat-7B
12
- ---
13
 
14
- # Beyonder-2x7B-v2
15
 
16
- This model is a Mixure of Experts (MoE) made with [mergekit](https://github.com/cg123/mergekit) (mixtral branch). It uses the following base models:
17
- * [mlabonne/Marcoro14-7B-slerp](https://huggingface.co/mlabonne/Marcoro14-7B-slerp)
18
- * [beowolx/CodeNinja-1.0-OpenChat-7B](https://huggingface.co/beowolx/CodeNinja-1.0-OpenChat-7B)
19
 
20
- ## 🧩 Configuration
21
 
22
- ```yaml
23
  base_model: mlabonne/Marcoro14-7B-slerp
24
  experts:
25
- - source_model: mlabonne/Marcoro14-7B-slerp
26
- positive_prompts:
27
- - "chat"
28
- - "assistant"
29
- - "tell me"
30
- - "explain"
31
- - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
32
- positive_prompts:
33
- - "code"
34
- - "python"
35
- - "javascript"
36
- - "programming"
37
- - "algorithm"
38
  ```
39
 
40
- ## 💻 Usage
41
 
42
- ```python
43
- !pip install -qU transformers bitsandbytes accelerate
44
 
45
- from transformers import AutoTokenizer
46
- import transformers
47
- import torch
48
 
49
- model = "shadowml/Beyonder-2x7B-v2"
50
 
51
- tokenizer = AutoTokenizer.from_pretrained(model)
52
- pipeline = transformers.pipeline(
53
- "text-generation",
54
- model=model,
55
- model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
56
- )
57
 
58
- messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
59
- prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
60
- outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
61
- print(outputs[0]["generated_text"])
62
- ```
 
1
  ---
2
+ license: apache-2.0
3
+ tags:
4
+ - moe
5
+ - mergekit
6
+ - merge
7
+ - mlabonne/Marcoro14-7B-slerp
8
+ - beowolx/CodeNinja-1.0-OpenChat-7B
9
  ---
 
 
 
 
 
 
 
 
 
10
 
11
+ # Beyonder-2x7B-v2
12
 
13
+ This model is a Mixure of Experts (MoE) made with [mergekit](https://github.com/cg123/mergekit) (mixtral branch). It uses the following base models:
14
+ * [mlabonne/Marcoro14-7B-slerp](https://huggingface.co/mlabonne/Marcoro14-7B-slerp)
15
+ * [beowolx/CodeNinja-1.0-OpenChat-7B](https://huggingface.co/beowolx/CodeNinja-1.0-OpenChat-7B)
16
 
17
+ ## 🧩 Configuration
18
 
19
+ ```yaml
20
  base_model: mlabonne/Marcoro14-7B-slerp
21
  experts:
22
+ - source_model: mlabonne/Marcoro14-7B-slerp
23
+ positive_prompts:
24
+ - "chat"
25
+ - "assistant"
26
+ - "tell me"
27
+ - "explain"
28
+ - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
29
+ positive_prompts:
30
+ - "code"
31
+ - "python"
32
+ - "javascript"
33
+ - "programming"
34
+ - "algorithm"
35
  ```
36
 
37
+ ## 💻 Usage
38
 
39
+ ```python
40
+ !pip install -qU transformers bitsandbytes accelerate
41
 
42
+ from transformers import AutoTokenizer
43
+ import transformers
44
+ import torch
45
 
46
+ model = "shadowml/Beyonder-2x7B-v2"
47
 
48
+ tokenizer = AutoTokenizer.from_pretrained(model)
49
+ pipeline = transformers.pipeline(
50
+ "text-generation",
51
+ model=model,
52
+ model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
53
+ )
54
 
55
+ messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
56
+ prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
57
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
58
+ print(outputs[0]["generated_text"])
59
+ ```