File size: 1,890 Bytes
06e068b
ac22bf9
 
 
 
 
 
 
 
 
 
 
06e068b
e3731d3
 
ac22bf9
 
 
e3731d3
ac22bf9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e3731d3
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
---
license: creativeml-openrail-m
base_model: runwayml/stable-diffusion-v1-5
tags:
- stable-diffusion
- stable-diffusion-diffusers
- text-to-image
- diffusers
- lora
- controlnet
- control-lora
inference: true
---
# ControlLoRA text2image fine-tuning - Official Model Repository
These are ControlLoRA adaption weights for [runwayml/stable-diffusion-v1-5](https://huggingface.co/runwayml/stable-diffusion-v1-5). You can find the code repository in [HighCWu/ControlLoRA](https://github.com/HighCWu/ControlLoRA).

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the [diffusiondb_2m_first_5k_canny](https://huggingface.co/datasets/HighCWu/diffusiondb_2m_first_5k_canny) and [mpii_100_openpose](https://huggingface.co/datasets/HighCWu/mpii_100_openpose) datasets with 30k steps on RTX4080 in 3 hours. 

You can find some example images in the following. 

![img_{i}](./docs/imgs/validation_25100_fe0b49075b45efdc311c.png)
*boys are playing with a frisbee in a field, 
2009 cinematography, trending on artforum, running pose, 
bruce springsteen, connected to heart machines, with tattoos, 
beautiful - n 9, by Eric Dinyer, young child, midlands*

![img_{i}](./docs/imgs/validation_25000_6b3f8b37f2268ec0c09e.png)
*portrait of a dancing eagle woman, "
"beautiful blonde haired lakota sioux goddess, "
"intricate, highly detailed art by james jean, "
"ray tracing, digital painting, artstation, "
"concept art, smooth, sharp focus, illustration, "
"artgerm and greg rutkowski and alphonse mucha, "
"vladimir kush, giger, roger dean, 8 k*

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

![img_{i}](./docs/imgs/7.png)
*portrait of female HighCWu as a cute pink hair girl*