Update README
Browse files
README.md
CHANGED
@@ -33,14 +33,19 @@ There are the following branches:
|
|
33 |
|
34 |
* Calibration dataset used for conversion: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)
|
35 |
* Evaluation dataset used to calculate perplexity: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/validation/0000.parquet)
|
|
|
|
|
36 |
* PPL max seq. length used: 1792 (2048 with 5.0-bpw-h8 causes OOM on RTX 4090 when evaluating ppl, so had to go down a bit)
|
37 |
|
38 |
|
39 |
-
| BPW |
|
40 |
-
| ----------- | ----------- | -------------- |
|
41 |
-
| 2.55-h6 | 15.0901 |
|
42 |
-
| 2.75-h6 | 13.6153 |
|
43 |
-
| 3.8-h6 | 6.8803 |
|
44 |
-
| 4.125-h6 | 6.8095 |
|
45 |
-
| 4.625-h6 | 6.7992 |
|
46 |
-
| 5.0-h8 | 6.7785 |
|
|
|
|
|
|
|
|
33 |
|
34 |
* Calibration dataset used for conversion: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)
|
35 |
* Evaluation dataset used to calculate perplexity: [wikitext-v2](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/validation/0000.parquet)
|
36 |
+
* Calibration dataset used for conversion of `5_0-bpw-h8-ev`: [wizardLM-evol-instruct_70k](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k/blob/refs%2Fconvert%2Fparquet/default/train/0000.parquet)
|
37 |
+
* Evaluation dataset used to calculate ppl for `Evol-Ins`: : [nikrosh-evol-instruct](https://huggingface.co/datasets/nickrosh/Evol-Instruct-Code-80k-v1/blob/refs%2Fconvert%2Fparquet/default/train/0000.parquet)
|
38 |
* PPL max seq. length used: 1792 (2048 with 5.0-bpw-h8 causes OOM on RTX 4090 when evaluating ppl, so had to go down a bit)
|
39 |
|
40 |
|
41 |
+
| BPW | PPL on Wiki | PPL on Evol-Ins | File Size (Gb) |
|
42 |
+
| ----------- | ----------- | --------------- | -------------- |
|
43 |
+
| 2.55-h6 | 15.0901 | | 10.56 |
|
44 |
+
| 2.75-h6 | 13.6153 | | 11.33 |
|
45 |
+
| 3.8-h6 | 6.8803 | | 15.37 |
|
46 |
+
| 4.125-h6 | 6.8095 | | 16.65 |
|
47 |
+
| 4.625-h6 | 6.7992 | 2.0499 | 18.58 |
|
48 |
+
| 5.0-h8 | 6.7785 | 2.0448 | 20.09 |
|
49 |
+
| 5.0-h8-ev | 6.9376 | 2.0430 | 20.09 |
|
50 |
+
|
51 |
+
|