
HIPE
non-profit
AI & ML interests
The HIPE HF organisation contains models and datasets related to the HIPE evaluation campaigns on the processing of named entity in historical documents. HIPE-eval is gradually evolving towards hosting models and datasets related the evaluation of information extraction systems on historical documents.
hipe's activity
Post
1539
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.
๐ Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
๐ Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with โค๏ธ and ๐ฅจ.
๐ Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
๐ Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with โค๏ธ and ๐ฅจ.

stefan-itย
authored
2
papers
over 1 year ago

stefan-itย
authored
4
papers
almost 2 years ago
Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0
Paper
โข
2204.05211
โข
Published
German's Next Language Model
Paper
โข
2010.10906
โข
Published
FLERT: Document-Level Features for Named Entity Recognition
Paper
โข
2011.06993
โข
Published
hmBERT: Historical Multilingual Language Models for Named Entity Recognition
Paper
โข
2205.15575
โข
Published