Yet Another Re-implementation Repository
AI/ML models and experiments, reimplemented for my own understanding.
Overview
I'm a career software engineer with a computer science background, but do not work with AI/ML in my daily duties. This repository aims to take concepts, models, and code from papers and around the web, and re-implement them using the techniques and best practices I've picked up over the years, and hopefully learn some things in the process. If you're like me and learned to code long before you learned about ML, this repo could help you, too.
Here on Huggingface, this repository holds the results of the experiments, not the code. To see the code, please go to my GitHub.
Implemented Models
| Model | Purpose | Paper |
|---|---|---|
| Rectified Flow | Image Synthesis | Scaling Rectified Flow Transformers for High-Resolution Image Synthesis, Esser et al. (2024) |
Prerequisites
Assuming python and CUDA are installed, you can ensure necessary packages are available via pip install -r requirements.txt
Rectified Flow
1-Channel
MNIST
Trained for 100 epochs in ~45 minutes.
Fashion MNIST
Trained for 100 epochs in ~45 minutes.
3-Channel
CIFAR-10
Trained for 100 epochs in ~3 hours.
CIFAR-100
Trained for 250 epochs in ~7.5 hours.
Training Commands
Basic Usage
This should be run from the root of the repository.
python -m yarr.trainers.rectified_flow
Options
Note: Weights and Biases is a freemium service for monitoring AI/ML training runs, using it will allow you to see details and samples throughout training. Use --wandb-entity to pass your team name to use it.
Usage: python -m yarr.trainers.rectified_flow [OPTIONS]
Train a Rectified Flow model on either MNIST, Fashion-MNIST, CIFAR-10, or CIFAR-100.
Options:
-lr, --learning-rate FLOAT Learning rate for the optimizer. [default: 0.001]
-e, --num-epochs INTEGER Number of epochs to train the model. [default: 100]
-w, --num-workers INTEGER Number of workers for the data loader. [default: 4]
-b, --batch-size INTEGER Batch size for training. [default: 250]
--wandb-entity TEXT Weights and Biases entity.
--resume Resume training from the latest checkpoint.
--fashion-mnist Use Fashion MNIST dataset instead of MNIST.
--cifar-10 Use CIFAR-10 dataset instead of MNIST.
--cifar-100 Use CIFAR-100 dataset instead of MNIST.
--help Show this message and exit.
License
All models in this repository are released into the public domain with no guarantees or warranty under the unlicense.
Contributions
While this repository is primarily for my own use and likely won't be useful as a library, I would welcome any recommendations others may have on other experiments to conduct or ways my implementations can be improved.
Citations and Acknowledgments
Simo Ryu (cloneofsimo), minRF for the inspiration and reference implementation.
@misc{ryu2024minrf,
author = {Simo Ryu},
title = {minRF: Minimal Implementation of Scalable Rectified Flow Transformers},
year = 2024,
publisher = {Github},
url = {https://github.com/cloneofsimo/minRF},
}