Hugging Face's logo

hmByT5 Preliminary

community
https://github.com/stefan-it/hmByT5
stefan-it
Activity Feed Request to join this org

AI & ML interests

ByT5, historic language models

hmbyt5-preliminary's activity

stefan-it 
posted an update 1 day ago
view post
Post
2919
She arrived 😍

[Expect more models soon...]
  • 1 reply
·
stefan-it 
posted an update 3 months ago
view post
Post
1539
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.

👉 Link: https://github.com/stefan-it/model-garden-lms

An overview of some features:

- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS

I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!

👉 Model Hub Link: https://huggingface.co/model-garden-lms

If you find these resources useful, please give them a like!

Made from Bavarian Oberland with ❤️ and 🥨.
stefan-it 
in hmbyt5-preliminary/byt5-small-multilingual-4g-2e 3 months ago

Adding `safetensors` variant of this model
#1 opened 4 months ago by
SFconvertbot
stefan-it 
updated 14 models over 1 year ago
stefan-it 
updated a Space over 1 year ago
stefan-it 
updated 2 models over 1 year ago