---
title: README
emoji: 📈
colorFrom: pink
colorTo: blue
sdk: static
pinned: false
---

We explore instruction-tuning to teach Stable Diffusion to follow instructions to translate or
process input images. With this method, we can prompt our model using an input image and an
“instruction”, such as - _Apply a cartoon filter to the natural image_.

<p align=center>
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/instruction-tuning-sd.png" width=600/>
</p>

Please follow our blog post to know more about this: TODO.


Below we provide links to our training code, models, and datasets:

* [Training and inference code](https://github.com/sayakpaul/instruction-tuned-sd)

* Models:
  * [instruction-tuning-sd/instruction-tuned-image-proc](https://huggingface.co/instruction-tuning-sd/instruction-tuned-image-proc)
  * [instruction-tuning-sd/instruction-tuned-cartoonizer](https://huggingface.co/instruction-tuning-sd/instruction-tuned-cartoonizer)
  * [instruction-tuning-sd/scratch-instruction-tuned-image-proc](instruction-tuning-sd/scratch-instruction-tuned-image-proc)
  * [instruction-tuning-sd/scratch-instruction-tuned-cartoonizer](https://huggingface.co/instruction-tuning-sd/scratch-instruction-tuned-cartoonizer)

* Datasets:
  * [Instruction-prompted cartoonization](https://huggingface.co/datasets/instruction-tuning-sd/cartoonizer-dataset)
  * [Instruction-prompted low-level image processing](https://huggingface.co/datasets/instruction-tuning-sd/instruct-tuned-image-processing) 

Happy instruction-tuning 🤗