awoo

Signed-off-by: Balazs Horvath <[email protected]>

README.md

@@ -49,7 +49,6 @@ The Yiff Toolkit is a comprehensive set of tools designed to enhance your creati
         - [`--save_model_as`](#--save_model_as)
         - [`--network_module`](#--network_module)
         - [`--network_args`](#--network_args)
-          - [`use_reentrant`](#use_reentrant)
           - [`preset`](#preset)
           - [`conv_dim` and `conv_alpha`](#conv_dim-and-conv_alpha)
           - [`module_dropout` and `dropout` and `rank_dropout`](#module_dropout-and-dropout-and-rank_dropout)
@@ -638,9 +637,6 @@ The arguments passed down to the network.
                "preset=full" \
                "conv_dim=256" \
                "conv_alpha=4" \
-               "dropout=None" \
-               "rank_dropout=None" \
-               "module_dropout=None" \
                "use_tucker=False" \
                "use_scalar=False" \
                "rank_dropout_scale=False" \
@@ -654,12 +650,6 @@ The arguments passed down to the network.
 
 ---
 
-###### `use_reentrant`
-
-- If `use_reentrant=False` is specified, checkpoint will use an implementation that does not require re-entrant autograd. You can learn more about checkpointing [here](https://pytorch.org/docs/stable/checkpoint.html). Note that future versions of PyTorch will default to `use_reentrant=False`, today the default is still `True`, so we set it to `False`. Easy!
-
----
-
 ###### `preset`
 
 The [Preset](https://github.com/KohakuBlueleaf/LyCORIS/blob/HEAD/docs/Preset.md)/config system added to LyCORIS for more fine-grained control.
@@ -698,7 +688,7 @@ conv_block_alphas = [conv_alpha] * num_total_blocks
 
 It’s called “rank” dropout because it operates on the rank of the input tensor, rather than its individual elements. This can be particularly useful in tasks where the rank of the input is important.
 
-If `rank_dropout` is set to `0`, it means that no dropout is applied to the rank of the input tensor `lx`. All elements of the mask would be set to `True` and when the mask gets applied to `lx` all of it's elements would be retained and when the scaling factor is applied after dropout it's value would just equal `self.scale` because `1.0 / (1.0 - 0)` is `1`. Basically, setting this to `0` effectively disables the dropout mechanism but it will still do some meaningless calculations.
+If `rank_dropout` is set to `0`, it means that no dropout is applied to the rank of the input tensor `lx`. All elements of the mask would be set to `True` and when the mask gets applied to `lx` all of it's elements would be retained and when the scaling factor is applied after dropout it's value would just equal `self.scale` because `1.0 / (1.0 - 0)` is `1`. Basically, setting this to `0` effectively disables the dropout mechanism but it will still do some meaningless calculations, and you can't set it to None, so if you really want to disable dropouts simply don't specify them! 😇
 
 ```python
 def forward(self, x):
@@ -1052,13 +1042,9 @@ accelerate launch --num_cpu_threads_per_process=2  "./sdxl_train_network.py" \
     --save_model_as="safetensors" \
     --network_module="lycoris.kohya" \
     --network_args \ 
-               "use_reentrant=False" \
                "preset=full" \
                "conv_dim=256" \
                "conv_alpha=4" \
-               "dropout=None" \
-               "rank_dropout=None" \
-               "module_dropout=None" \
                "use_tucker=False" \
                "use_scalar=False" \
                "rank_dropout_scale=False" \