---
title: README
emoji: ⚡
colorFrom: pink
colorTo: red
sdk: static
pinned: false
---

# 🇩🇪 Welcome to GERTuraX

* GERTuraX is a series of pretrained encoder-only language models for German.

* The models are ELECTRA-based and pretrained with the [TEAMS](https://aclanthology.org/2021.findings-acl.219/) approach on the [CulturaX](https://huggingface.co/datasets/uonlp/CulturaX) corpus. 

* In total, three different models were trained and released with pretraining corpus sizes ranging from 147GB to 1.1TB.

# 📊 Models

The following models are available:

* [GERTuraX-1](https://huggingface.co/gerturax/gerturax-1)
* [GERTuraX-2](https://huggingface.co/gerturax/gerturax-2)
* [GERTuraX-3](https://huggingface.co/gerturax/gerturax-3)


# ❤️ Acknowledgements

GERTuraX is the outcome of the last 12 months of working with TPUs from the awesome [TRC program](https://sites.research.google/trc/about/) and the [TensorFlow Model Garden](https://github.com/tensorflow/models) library.

Many thanks for providing TPUs!

Made from Bavarian Oberland with ❤️ and 🥨.