File size: 1,890 Bytes
06e068b ac22bf9 06e068b e3731d3 ac22bf9 e3731d3 ac22bf9 e3731d3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
license: creativeml-openrail-m
base_model: runwayml/stable-diffusion-v1-5
tags:
- stable-diffusion
- stable-diffusion-diffusers
- text-to-image
- diffusers
- lora
- controlnet
- control-lora
inference: true
---
# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5). You can find the code repository in [HighCWu/ControlLoRA](https://github.com/HighCWu/ControlLoRA).
With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.
Each of the weights is fine-tuned on the [diffusiondb_2m_first_5k_canny](https://huggingface.co/datasets/HighCWu/diffusiondb_2m_first_5k_canny) and [mpii_100_openpose](https://huggingface.co/datasets/HighCWu/mpii_100_openpose) datasets with 30k steps on RTX4080 in 3 hours.
You can find some example images in the following.
![img_{i}](./docs/imgs/validation_25100_fe0b49075b45efdc311c.png)
*boys are playing with a frisbee in a field,
2009 cinematography, trending on artforum, running pose,
bruce springsteen, connected to heart machines, with tattoos,
beautiful - n 9, by Eric Dinyer, young child, midlands*
![img_{i}](./docs/imgs/validation_25000_6b3f8b37f2268ec0c09e.png)
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*
I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.
![img_{i}](./docs/imgs/7.png)
*portrait of female HighCWu as a cute pink hair girl*
|