DA-BPE-CEREBRAS / README.md

MikkelWK

Create README.md

c5efc0e verified 23 days ago

preview code

raw

history blame contribute delete

274 Bytes

metadata

library_name: transformers
tags:
  - Danish
  - BPE Tokenization
  - CerebrasGPT

DA-BPE-CEREBRAS

This CerebrasGPT-111M model employs a standard Byte-Pair-Encoding (BPE) tokenizer for Danish text. It serves as a benchmark to compare against morphology-aware tokenizers.