metadata

license: creativeml-openrail-m
base_model: runwayml/stable-diffusion-v1-5
tags:
  - stable-diffusion
  - stable-diffusion-diffusers
  - text-to-image
  - diffusers
  - lora
  - controlnet
  - control-lora
inference: true

ControlLoRA text2image fine-tuning - Official Model Repository

These are ControlLoRA adaption weights for runwayml/stable-diffusion-v1-5. You can find the code repository in HighCWu/ControlLoRA.

With ControlLoRA, a simple and small (~7M parameters, ~25M storage space) network, you could control the spatial information of stable diffusion.

Each of the weights is fine-tuned on the diffusiondb_2m_first_5k_canny and mpii_100_openpose datasets with 30k steps on RTX4080 in 3 hours.

You can find some example images in the following.

$img_{i}$ boys are playing with a frisbee in a field, 2009 cinematography, trending on artforum, running pose, bruce springsteen, connected to heart machines, with tattoos, beautiful - n 9, by Eric Dinyer, young child, midlands

$img_{i}$ portrait of a dancing eagle woman, " "beautiful blonde haired lakota sioux goddess, " "intricate, highly detailed art by james jean, " "ray tracing, digital painting, artstation, " "concept art, smooth, sharp focus, illustration, " "artgerm and greg rutkowski and alphonse mucha, " "vladimir kush, giger, roger dean, 8 k

I also uploaded a lora model fine-tuned on my selfies with 2k steps which could be use in the experiment of mixing LoRA and ControlLoRA.

$img_{i}$ portrait of female HighCWu as a cute pink hair girl