README.md

---
license: cc0-1.0
datasets:
- Lichess/standard-chess-games
pipeline_tag: text2text-generation
tags:
- chess
---
# Model card for chessPT
A pretrained Decoder only transformer model for chess move prediction.

## Intended use
Predict new moves in a chess game based on PGN tokens.

## Implementation
The model implementation is based on Andrej Karpathy's [nanoGPT](https://github.com/karpathy/nanoGPT) following the webseries "Zero to Hero" on [youtube](https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ).

## Training
You can find the training script in the repositories files under `train.py`.
This also contains the used parameters
```python
context_size = 256
batch_size = 128
max_iters = 30_000
learning_rate = 3e-5
eval_interval = 100
eval_iters = 20
n_embed = 384
n_layer = 6
n_head = 6
dropout = 0.2
```