patrickvonplaten commited on
Commit
efbe823
1 Parent(s): 4720714

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -6,6 +6,7 @@ tags:
6
  - controlnet
7
  - stable-diffusion
8
  - controlnet-v1-1
 
9
  duplicated_from: ControlNet-1-1-preview/control_v11p_sd15_mlsd
10
  ---
11
 
@@ -152,10 +153,13 @@ on a different type of conditioning:
152
  |[lllyasviel/control_v11e_sd15_shuffle](https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle)<br/> Trained with image shuffling | An image with shuffled patches or regions.|<a href="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/control.png"><img width="64" style="margin:0;padding:0;" src="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/control.png"/></a>|<a href="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/image_out.png"><img width="64" src="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/image_out.png"/></a>|
153
 
154
 
155
- ### Training
156
 
157
- TODO
 
 
 
158
 
159
- ### Blog post
160
 
161
- For more information, please also have a look at the [Diffusers ControlNet Blog Post](https://huggingface.co/blog/controlnet).
 
6
  - controlnet
7
  - stable-diffusion
8
  - controlnet-v1-1
9
+ - image-to-image
10
  duplicated_from: ControlNet-1-1-preview/control_v11p_sd15_mlsd
11
  ---
12
 
 
153
  |[lllyasviel/control_v11e_sd15_shuffle](https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle)<br/> Trained with image shuffling | An image with shuffled patches or regions.|<a href="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/control.png"><img width="64" style="margin:0;padding:0;" src="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/control.png"/></a>|<a href="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/image_out.png"><img width="64" src="https://huggingface.co/lllyasviel/control_v11e_sd15_shuffle/resolve/main/images/image_out.png"/></a>|
154
 
155
 
156
+ ## Improvements in MLSD 1.1:
157
 
158
+ - The training dataset of previous cnet 1.0 has several problems including (1) a small group of greyscale human images are duplicated thousands of times (!!), causing the previous model somewhat likely to generate grayscale human images; (2) some images has low quality, very blurry, or significant JPEG artifacts; (3) a small group of images has wrong paired prompts caused by a mistake in our data processing scripts. The new model fixed all problems of the training dataset and should be more reasonable in many cases.
159
+ - We enlarged the training dataset by adding 300K more images by using MLSD to find images with more than 16 straight lines in it.
160
+ - Some reasonable data augmentations are applied to training, like random left-right flipping.
161
+ - Resumed from MLSD 1.0 with continued training with 200 GPU hours of A100 80G.
162
 
163
+ ## More information
164
 
165
+ For more information, please also have a look at the [Diffusers ControlNet Blog Post](https://huggingface.co/blog/controlnet) and have a look at the [official docs](https://github.com/lllyasviel/ControlNet-v1-1-nightly).