awoo

Signed-off-by: Balazs Horvath <[email protected]>

README.md

@@ -30,11 +30,17 @@ The Yiff Toolkit is a comprehensive set of tools designed to enhance your creati
     - [Pony Training](#pony-training)
       - [Download Pony in Diffusers Format](#download-pony-in-diffusers-format)
       - [Sample Prompt File](#sample-prompt-file)
+      - [`--lowram`](#--lowram)
+      - [`--pretrained_model_name_or_path`](#--pretrained_model_name_or_path)
+      - [`--train_data_dir`](#--train_data_dir)
+      - [`--resolution`](#--resolution)
+      - [`--optimizer_type`](#--optimizer_type)
       - [`--dataset_repeats`](#--dataset_repeats)
       - [`--max_train_steps`](#--max_train_steps)
       - [`--shuffle_caption`](#--shuffle_caption)
       - [`--sdpa`](#--sdpa)
-      - [`--sample_sampler`](#--sample_sampler)
+      - [`--sample_prompts --sample_sampler --sample_every_n_steps`](#--sample_prompts---sample_sampler---sample_every_n_steps)
+    - [CosXL Training](#cosxl-training)
   - [Embeddings for 1.5 and SDXL](#embeddings-for-15-and-sdxl)
   - [ComfyUI Walkthrough any%](#comfyui-walkthrough-any)
   - [AnimateDiff for Masochists](#animatediff-for-masochists)
@@ -102,17 +108,20 @@ The Yiff Toolkit is a comprehensive set of tools designed to enhance your creati
 
 ### Installation Tips
 
-Firstly, download kohya_ss' [sd-scripts](https://github.com/kohya-ss/sd-scripts), you need to set up your environment either like [this](https://github.com/kohya-ss/sd-scripts?tab=readme-ov-file#windows-installation) tells you for Windows, or if you are using Linux or Miniconda on Windows, you are probably smart enough to figure out the installation for it. I recommend always installing the latest [PyTorch](https://pytorch.org/get-started/locally/) in the virtual environment you are going to use, which at the time of writing is `2.2.2`. I hope future me has faster PyTorch!
+---
 
-If someone told you to install `xformers` call them stinky, because ever since the fused implementation of `sdpa` landed in torch it has been the king of my benchmarks.
-For training you will have to go with either `--sdpa` or `--xformers`
+Firstly, download kohya_ss' [sd-scripts](https://github.com/kohya-ss/sd-scripts), you need to set up your environment either like [this](https://github.com/kohya-ss/sd-scripts?tab=readme-ov-file#windows-installation) tells you for Windows, or if you are using Linux or Miniconda on Windows, you are probably smart enough to figure out the installation for it. I recommend always installing the latest [PyTorch](https://pytorch.org/get-started/locally/) in the virtual environment you are going to use, which at the time of writing is `2.2.2`. I hope future me has faster PyTorch!
 
 ### Dataset Preparation
 
+---
+
 ⚠️ **TODO:** Awoo this section.
 
 ### Pony Training
 
+---
+
 I'm not going to lie, it is a bit complicated to explain everything. But here is my best attempt going through some "basic" stuff and almost all lines in order.
 
 #### Download Pony in Diffusers Format
@@ -125,7 +134,7 @@ git clone https://huggingface.co/k4d3/ponydiffusers
 
 #### Sample Prompt File
 
-A sample prompt file is used during training to sample images. A sample prompt for example might look like this for Pony.
+A sample prompt file is used during training to sample images. A sample prompt for example might look like this for Pony:
 
 ```py
 # anthro female kindred
@@ -136,6 +145,42 @@ score_9, score_8_up, score_7_up, score_6_up, rating_explicit, source_furry, solo
 score_9, score_8_up, score_7_up, score_6_up, rating_explicit, source_furry, solo, anthro male fox, glowing yellow eyes, night, crescent moon, tibetan necklace, gold bracers, blue and gold adorned loincloth, canine genitalia, knot, amazing_background, scenery porn, white marble ruins in the background, realistic, photo, photo (medium), photography (artwork) --n low quality, worst quality --w 1024 --h 1024 --d 1 --l 6.0 --s 40
 ```
 
+#### `--lowram`
+
+If you are running running out of RAM like I do with 2 GPUs and a really fat model, this option will help you save a bit of it and might get you out of OOM hell.
+
+#### `--pretrained_model_name_or_path`
+
+The directory containing the checkpoint you just downloaded. I recommend closing the path if you are using a local model with a `/`.
+
+```py
+    --pretrained_model_name_or_path="/ponydiffusers/" \
+```
+
+#### `--train_data_dir`
+
+The directory containing the dataset. We prepared this earlier together.
+
+```py
+    --train_data_dir="/training_dir" \
+```
+
+#### `--resolution`
+
+Always set this to match the model's resolution, which in Pony's case it is 1024x1024. If you can't fit into the VRAM, you can decrease it to `512,512` as a last resort.
+
+```py
+    --resolution="512,512" \
+```
+
+#### `--optimizer_type`
+
+The default optimizer is `AdamW` and there are a bunch of them added every month or so, therefore I'm not listing it. You can find the list if you really want. But `AdamW` is the best as of this writing so we use that!
+
+```py
+    --optimizer_type="AdamW" \
+```
+
 #### `--dataset_repeats`
 
 Repeats the dataset when training with captions, by default it is set to `1` so we'll set this to `0` with:
@@ -164,24 +209,32 @@ As you can tell, I have separated the caption part not just the tags with a `,`
 
 The choice between `--xformers` and `--spda` will depend on your GPU. You can benchmark it by repeating a training with both!
 
-#### `--sample_sampler`
+#### `--sample_prompts --sample_sampler --sample_every_n_steps`
 
 You have the option of generating images during training so you can check the progress, the argument let's you pick between different samplers, by default it is on `ddim`, so you better change it!
 
  You can also use `--sample_every_n_epochs` instead which will take precedence over steps. The `k_` prefix means karras and the `_a` suffix means ancestral.
 
 ```py
+    --sample_prompts=/training_dir/sample-prompts.txt
     --sample_sampler="euler_a" \
     --sample_every_n_steps=100
 ```
 
 My recommendation for Pony is to use `euler_a` for toony and for realistic `k_dpm_2`.
-Your options include the following:
+
+Your sampler options include the following:
 
 ```bash
 ddim, pndm, lms, euler, euler_a, heun, dpm_2, dpm_2_a, dpmsolver, dpmsolver++, dpmsingle, k_lms, k_euler, k_euler_a, k_dpm_2, k_dpm_2_a
 ```
 
+### CosXL Training
+
+The only difference between CosXL training is that you need to enable `--v_parameterization`, and you can't sample the images. 😹 I also don't recommend using the `block_dims` and `block_alphas` from Pony.
+
+---
+
 ## Embeddings for 1.5 and SDXL
 
 Embeddings in Stable Diffusion are high-dimensional representations of input data, such as images or text, that capture their essential features and relationships. These embeddings are used to guide the diffusion process, enabling the model to generate outputs that closely match the desired characteristics specified in the input.