readme: add leaderboard section
Browse files
README.md
CHANGED
|
@@ -21,6 +21,36 @@ Historic Multilingual and Monolingual ByT5 Models. Following languages are curre
|
|
| 21 |
|
| 22 |
More details can be found in [our GitHub repository](https://github.com/stefan-it/hmByT5).
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
# Acknowledgements
|
| 25 |
|
| 26 |
We thank [Luisa März](https://github.com/LuisaMaerz), [Katharina Schmid](https://github.com/schmika) and
|
|
|
|
| 21 |
|
| 22 |
More details can be found in [our GitHub repository](https://github.com/stefan-it/hmByT5).
|
| 23 |
|
| 24 |
+
# Leaderboard
|
| 25 |
+
|
| 26 |
+
We test our pretrained language models on various datasets from HIPE-2020, HIPE-2022 and Europeana. The following table
|
| 27 |
+
shows an overview of used datasets.
|
| 28 |
+
|
| 29 |
+
| Language | Dataset | Additional Dataset |
|
| 30 |
+
|----------|--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
|
| 31 |
+
| English | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
|
| 32 |
+
| German | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | - |
|
| 33 |
+
| French | [AjMC](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-ajmc.md) | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) |
|
| 34 |
+
| Finnish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
|
| 35 |
+
| Swedish | [NewsEye](https://github.com/hipe-eval/HIPE-2022-data/blob/main/documentation/README-newseye.md) | - |
|
| 36 |
+
| Dutch | [ICDAR-Europeana](https://github.com/stefan-it/historic-domain-adaptation-icdar) | - |
|
| 37 |
+
|
| 38 |
+
Current best models:
|
| 39 |
+
|
| 40 |
+
| Model | English AjMC | German AjMC | French AjMC | Finnish NewsEye | Swedish NewsEye | Dutch ICDAR | French ICDAR | Avg. |
|
| 41 |
+
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|--------------|--------------|-----------------|-----------------|--------------|--------------|------|
|
| 42 |
+
| [`hmbyt5/byt5-small-english`](https://huggingface.co/hmbyt5/byt5-small-english) | 85.65 ± 1.21 | 87.27 ± 0.50 | 84.44 ± 0.79 | | | | | |
|
| 43 |
+
| [`hmbyt5-preliminary/byt5-small-english-german`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german) | 85.74 ± 0.72 | 87.45 ± 0.67 | 84.23 ± 0.65 | | | | | |
|
| 44 |
+
| [`hmbyt5-preliminary/byt5-small-english-german-french`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french) | 85.61 ± 0.96 | 87.24 ± 0.76 | 84.39 ± 0.68 | | | | | |
|
| 45 |
+
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish) | 85.30 ± 1.14 | 87.37 ± 0.53 | 84.12 ± 0.42 | | | | | |
|
| 46 |
+
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish) | 85.40 ± 0.78 | 87.12 ± 0.19 | 84.41 ± 0.34 | | | | | |
|
| 47 |
+
| [`hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch`](https://huggingface.co/hmbyt5-preliminary/byt5-small-english-german-french-finnish-swedish-dutch) | 85.51 ± 0.68 | 87.58 ± 0.39 | 84.39 ± 0.83 | 55.46 ± 1.99 | 73.38 ± 2.45 | 84.80 ± 0.44 | 75.97 ± 0.55 | |
|
| 48 |
+
| [`hmbyt5-preliminary/byt5-small-multilingual-4g`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g) | 83.49 ± 0.96 | 87.65 ± 0.63 | 84.16 ± 0.90 | | | | | |
|
| 49 |
+
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-2e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-2e) | 83.86 ± 0.61 | 87.54 ± 0.19 | 84.29 ± 0.41 | | | | | |
|
| 50 |
+
| [`hmbyt5-preliminary/byt5-small-multilingual-4g-3e`](https://huggingface.co/hmbyt5-preliminary/byt5-small-multilingual-4g-3e) | 83.49 ± 0.99 | 87.38 ± 0.53 | 84.30 ± 0.51 | | | | | |
|
| 51 |
+
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-flax) | 83.28 ± 1.67 | 86.98 ± 0.71 | 83.49 ± 1.06 | 76.96 ± 1.58 | 78.80 ± 1.89 | 86.47 ± 0.79 | 77.43 ± 0.51 | |
|
| 52 |
+
| [`hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax`](https://huggingface.co/hmbyt5-preliminary/byt5-small-historic-multilingual-span20-flax) | 84.91 ± 0.86 | 88.02 ± 0.35 | 84.78 ± 0.75 | 77.77 ± 1.83 | 79.94 ± 0.60 | 86.85 ± 0.91 | 77.45 ± 0.54 | |
|
| 53 |
+
|
| 54 |
# Acknowledgements
|
| 55 |
|
| 56 |
We thank [Luisa März](https://github.com/LuisaMaerz), [Katharina Schmid](https://github.com/schmika) and
|