Adding the Open Portuguese LLM Leaderboard Evaluation Results
#1
by
leaderboard-pt-pr-bot
- opened
README.md
CHANGED
|
@@ -73,4 +73,23 @@ The following hyperparameters were used during training:
|
|
| 73 |
- Transformers 4.36.0.dev0
|
| 74 |
- Pytorch 2.0.1+cu117
|
| 75 |
- Datasets 2.14.7
|
| 76 |
-
- Tokenizers 0.15.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
- Transformers 4.36.0.dev0
|
| 74 |
- Pytorch 2.0.1+cu117
|
| 75 |
- Datasets 2.14.7
|
| 76 |
+
- Tokenizers 0.15.0
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
# Open Portuguese LLM Leaderboard Evaluation Results
|
| 80 |
+
|
| 81 |
+
Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/heloisy/zephyr-7b-b-cosmosqa-ptbr-text-v3) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
|
| 82 |
+
|
| 83 |
+
| Metric | Value |
|
| 84 |
+
|--------------------------|---------|
|
| 85 |
+
|Average |**62.46**|
|
| 86 |
+
|ENEM Challenge (No Images)| 56.89|
|
| 87 |
+
|BLUEX (No Images) | 45.76|
|
| 88 |
+
|OAB Exams | 38.22|
|
| 89 |
+
|Assin2 RTE | 89.05|
|
| 90 |
+
|Assin2 STS | 67.49|
|
| 91 |
+
|FaQuAD NLI | 64.55|
|
| 92 |
+
|HateBR Binary | 77.14|
|
| 93 |
+
|PT Hate Speech Binary | 67.20|
|
| 94 |
+
|tweetSentBR | 55.87|
|
| 95 |
+
|