leaderboard-pr-bot commited on
Commit
7ae87ff
1 Parent(s): 7b7a2dc

Adding Evaluation Results

Browse files

This is an automated PR created with https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr

The purpose of this PR is to add evaluation results from the Open LLM Leaderboard to your model card.

If you encounter any issues, please report them to https://huggingface.co/spaces/Weyaxi/open-llm-leaderboard-results-pr/discussions

Files changed (1) hide show
  1. README.md +117 -0
README.md CHANGED
@@ -32,6 +32,109 @@ prompt_template: '### System:
32
 
33
  '
34
  quantized_by: TheBloke
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  ---
36
 
37
  <!-- header start -->
@@ -440,3 +543,17 @@ Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edun
440
  year={2023}
441
  }
442
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  '
34
  quantized_by: TheBloke
35
+ model-index:
36
+ - name: orca_mini_v3_13B-GPTQ
37
+ results:
38
+ - task:
39
+ type: text-generation
40
+ name: Text Generation
41
+ dataset:
42
+ name: AI2 Reasoning Challenge (25-Shot)
43
+ type: ai2_arc
44
+ config: ARC-Challenge
45
+ split: test
46
+ args:
47
+ num_few_shot: 25
48
+ metrics:
49
+ - type: acc_norm
50
+ value: 61.95
51
+ name: normalized accuracy
52
+ source:
53
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
54
+ name: Open LLM Leaderboard
55
+ - task:
56
+ type: text-generation
57
+ name: Text Generation
58
+ dataset:
59
+ name: HellaSwag (10-Shot)
60
+ type: hellaswag
61
+ split: validation
62
+ args:
63
+ num_few_shot: 10
64
+ metrics:
65
+ - type: acc_norm
66
+ value: 81.56
67
+ name: normalized accuracy
68
+ source:
69
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
70
+ name: Open LLM Leaderboard
71
+ - task:
72
+ type: text-generation
73
+ name: Text Generation
74
+ dataset:
75
+ name: MMLU (5-Shot)
76
+ type: cais/mmlu
77
+ config: all
78
+ split: test
79
+ args:
80
+ num_few_shot: 5
81
+ metrics:
82
+ - type: acc
83
+ value: 56.1
84
+ name: accuracy
85
+ source:
86
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: TruthfulQA (0-shot)
93
+ type: truthful_qa
94
+ config: multiple_choice
95
+ split: validation
96
+ args:
97
+ num_few_shot: 0
98
+ metrics:
99
+ - type: mc2
100
+ value: 49.22
101
+ source:
102
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
103
+ name: Open LLM Leaderboard
104
+ - task:
105
+ type: text-generation
106
+ name: Text Generation
107
+ dataset:
108
+ name: Winogrande (5-shot)
109
+ type: winogrande
110
+ config: winogrande_xl
111
+ split: validation
112
+ args:
113
+ num_few_shot: 5
114
+ metrics:
115
+ - type: acc
116
+ value: 75.77
117
+ name: accuracy
118
+ source:
119
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
120
+ name: Open LLM Leaderboard
121
+ - task:
122
+ type: text-generation
123
+ name: Text Generation
124
+ dataset:
125
+ name: GSM8k (5-shot)
126
+ type: gsm8k
127
+ config: main
128
+ split: test
129
+ args:
130
+ num_few_shot: 5
131
+ metrics:
132
+ - type: acc
133
+ value: 29.49
134
+ name: accuracy
135
+ source:
136
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/orca_mini_v3_13B-GPTQ
137
+ name: Open LLM Leaderboard
138
  ---
139
 
140
  <!-- header start -->
 
543
  year={2023}
544
  }
545
  ```
546
+
547
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
548
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TheBloke__orca_mini_v3_13B-GPTQ)
549
+
550
+ | Metric |Value|
551
+ |---------------------------------|----:|
552
+ |Avg. |59.01|
553
+ |AI2 Reasoning Challenge (25-Shot)|61.95|
554
+ |HellaSwag (10-Shot) |81.56|
555
+ |MMLU (5-Shot) |56.10|
556
+ |TruthfulQA (0-shot) |49.22|
557
+ |Winogrande (5-shot) |75.77|
558
+ |GSM8k (5-shot) |29.49|
559
+