bloc97 commited on
Commit
e72b36c
·
1 Parent(s): 0ad472f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -2
README.md CHANGED
@@ -1,10 +1,20 @@
 
 
 
 
 
 
 
1
  # Model Card: Nous-Yarn-Llama-2-13b-128k
2
 
3
 
4
 
5
  ## Model Description
6
 
7
- Nous-Yarn-Llama-2-13b-128k is a state-of-the-art language model for long context, further pretrained on long context data for 600 steps.
 
 
 
8
 
9
  ## Model Training
10
 
@@ -20,7 +30,6 @@ Starting from the base Llama 2 models, this model was further pretrained on a su
20
  The authors would like to thank Stability AI, Carper AI, and Eleuther AI for their generous support of significant computing resources that enabled the training of these models and the completion of this research. We would also like to thank Jonathan Tow and Dakota Mahan directly for their help in advising on the use of the Stability AI compute cluster. Additionally, we would like to thank a16z, and PygmalionAI, for providing resources to run evaluations and experiments on the models.
21
 
22
  ## Usage and Prompt Format
23
- This model requires the [Flash Attention library](https://pypi.org/project/flash-attn/) in order to function correctly.
24
 
25
  Install FA2 and Rotary Extensions:
26
  ```
 
1
+ ---
2
+ datasets:
3
+ - pg19
4
+ metrics:
5
+ - perplexity
6
+ library_name: transformers
7
+ ---
8
  # Model Card: Nous-Yarn-Llama-2-13b-128k
9
 
10
 
11
 
12
  ## Model Description
13
 
14
+ Nous-Yarn-Llama-2-13b-128k is a state-of-the-art language model for long context, further pretrained on long context data for 600 steps.
15
+ This model is the Flash Attention 2 patched version of the original model: https://huggingface.co/conceptofmind/Yarn-Llama-2-13b-128k
16
+
17
+ Note that this model **requires** the [Flash Attention library](https://pypi.org/project/flash-attn/) in order to function correctly, see the Model Usage section for installation instructions.
18
 
19
  ## Model Training
20
 
 
30
  The authors would like to thank Stability AI, Carper AI, and Eleuther AI for their generous support of significant computing resources that enabled the training of these models and the completion of this research. We would also like to thank Jonathan Tow and Dakota Mahan directly for their help in advising on the use of the Stability AI compute cluster. Additionally, we would like to thank a16z, and PygmalionAI, for providing resources to run evaluations and experiments on the models.
31
 
32
  ## Usage and Prompt Format
 
33
 
34
  Install FA2 and Rotary Extensions:
35
  ```