Post
2919
She arrived 😍
[Expect more models soon...]
[Expect more models soon...]
- 1 reply
hmBERT 64k
non-profit
AI & ML interests
Pretraining Historical Multilingual Language Models
hmbert-64k's activity
Post
1539
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
stefan-it
updated
13
models
over 1 year ago
hmbert-64k/flair-hipe-2022-topres19th-en
Token Classification
•
Updated
•
2
hmbert-64k/flair-hipe-2022-newseye-sv
Token Classification
•
Updated
•
6
hmbert-64k/flair-hipe-2022-newseye-fr
Token Classification
•
Updated
•
7
hmbert-64k/flair-hipe-2022-newseye-fi
Token Classification
•
Updated
•
5
hmbert-64k/flair-hipe-2022-newseye-de
Token Classification
•
Updated
•
9
hmbert-64k/flair-hipe-2022-letemps-fr
Token Classification
•
Updated
•
6
hmbert-64k/flair-icdar-nl
Token Classification
•
Updated
•
3
hmbert-64k/flair-icdar-fr
Token Classification
•
Updated
•
6
hmbert-64k/flair-hipe-2022-hipe2020-fr
Token Classification
•
Updated
•
10
hmbert-64k/flair-hipe-2022-hipe2020-de
Token Classification
•
Updated
•
4
hmbert-64k/flair-hipe-2022-ajmc-fr
Token Classification
•
Updated
•
10
hmbert-64k/flair-hipe-2022-ajmc-en
Token Classification
•
Updated
•
6
hmbert-64k/flair-hipe-2022-ajmc-de
Token Classification
•
Updated
•
9
stefan-it
authored
2
papers
over 1 year ago
stefan-it
authored
2
papers
almost 2 years ago