scripts/TRAIN.md

# Train

## Tokenizer

```bash
cd scripts
python -m venv venv
source venv/bin/activate
pip install -U -r requirements.in
```

```bash
python -B train_tokenizer.py
```

## Dataset

```bash
cd scripts
python -m venv venv-lit
source venv-lit/bin/activate
pip install -U -r requirements-lit.in
```

```bash
python -B prepare_pretrain_dataset.py
```

## Model

```bash
cd scripts
python -m venv venv-lit
source venv-lit/bin/activate
pip install -U -r requirements-lit.in
```

```bash
litgpt pretrain --config ./model.yaml
```