ControlNet

#35
by adamelliotfields - opened

Add a 🎮 Control tab with a control image input. Show the detected features beside the input image. For example, if using Canny, show the edge map.

Use controlnet_aux to get the features so you shouldn't have to do any OpenCV stuff.

For 1.5, going to try Tencent’s T2I-Adapter. It’s significantly smaller, and I’m going to implement ControlNet in SDXL and Flux anyways.

Edit: In Diffusers, the T2I-Adapter pipeline doesn't have img2img or IP-Adapter support yet.

Started with Canny since it runs fast on CPU. For now, users must click the Annotate button first to generate the feature map. Closed by 98afd85.

adamelliotfields changed discussion status to closed

Sign up or log in to comment