Commit
•
b84a594
1
Parent(s):
4a02fc6
Update README.md
Browse files
README.md
CHANGED
|
@@ -83,6 +83,23 @@ output = model.generate(
|
|
| 83 |
print(tokenizer.decode(output[0]))
|
| 84 |
```
|
| 85 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
## Acknowledgements
|
| 87 |
|
| 88 |
The MPT model was originally finetuned by Sam Havens and the MosaicML NLP team. Please cite this model using the following format:
|
|
|
|
| 83 |
print(tokenizer.decode(output[0]))
|
| 84 |
```
|
| 85 |
|
| 86 |
+
## Evaluation
|
| 87 |
+
[MPT-7B-Instruct](https://huggingface.co/mosaicml/mpt-7b-instruct)
|
| 88 |
+
|
| 89 |
+
| Task |Version| Metric | Value | |Stderr|
|
| 90 |
+
|--------|------:|---------------|------:|---|------|
|
| 91 |
+
|wikitext| 1|word_perplexity|10.8864| | |
|
| 92 |
+
| | |byte_perplexity| 1.5628| | |
|
| 93 |
+
| | |bits_per_byte | 0.6441| | |
|
| 94 |
+
|
| 95 |
+
[MPT-7B-Instruct (4-bit 128-group AWQ)](https://huggingface.co/abhinavkulkarni/mpt-7b-instruct-w4-g128-awq)
|
| 96 |
+
|
| 97 |
+
| Task |Version| Metric | Value | |Stderr|
|
| 98 |
+
|--------|------:|---------------|------:|---|------|
|
| 99 |
+
|wikitext| 1|word_perplexity|11.2696| | |
|
| 100 |
+
| | |byte_perplexity| 1.5729| | |
|
| 101 |
+
| | |bits_per_byte | 0.6535| | |
|
| 102 |
+
|
| 103 |
## Acknowledgements
|
| 104 |
|
| 105 |
The MPT model was originally finetuned by Sam Havens and the MosaicML NLP team. Please cite this model using the following format:
|