Spaces:

OzzyGT

/

diffusers-image-fill

like 258

Running on Zero

Hi, I could but I don't know if there's a way to use this controlnet with comfyui. If you can point me to a node that works with this I can release a workflow.

Also I'm working on a full diffusers port to ComfyUI where you can use all the releases that are for diffusers including this one, will that work for you?

I found that ComfyUI has support for it, but now I can't find a way to delete part of the image, comfyui by default only lets you mask the image. I'll see what I can do.

I am looking forward to the result of the ComfyUI workflow. Do you have any breakthroughs? Thanks so much!!

Here's a ton of workflows with the Union controlnet, easy to find if you know where to look! https://openart.ai/workflows/all?keyword=union

@OzzyGT why is delete needed? Just pass the mask to contrlnet apply node and you are done? There's some built-in and many custom mask nodes out there.

Hi, because what was asked was to make a comfy workflow of this space not how to use controlnet union. If you read the code and my guide you can see why is needed, there's a reason why people are liking to use this space and the ones derived from it, and that's because is not the usual or normal way to use the controlnet.

I haven't had the time to do this, but soon, I don't really like the idea of installing some random node, I have a project where I need to port diffusers to comfyui and making these spaces available is part of it.

If you really want to help the users, feel free to post here the ComfyUI workflow that can produce the same results, the idea here is to collaborate and help each other.

Hey @OzzyGT No need to get defensive! You did some awesome work and I was just trying to add to the conversation. I did skim your code, but I don't see any guide that you're referring to. Mind adding a link to it in your readme?

Please understand that all I had to go with to learn about your project was a random tweet from someone else that linked directly to the space without any context. I will definitely check out that guide, and if I do have anything to add in terms of porting the approach to comfy, I will do that.

As a pretty hardcore comfy user, I can't say I agree with avoiding all custom nodes, but I certainly do understand the desire for a clean reference implementation that would work out of the box for all. Luckily, that team has gone a long way towards making the ecosystem features fairly accessible.

@halr9000 I though it could sound defensive or aggressive that why I tried to edit it, my intention wasn't like that at all. I just tried to explain the reasons of the space and why people are asking for a comfyui port, but please read it with a friendly tone.

I like a lot ComfyUI but I'm more UI agnostic so I don't root for a specific one like most, I also use the other ones and try to keep up to date with the ones I like but ComfyUI has a lot of custom nodes made by different people (which is not bad) but for someone like me, that isn't ComfyUI centric, it would be a lot of work to keep up with all of them and to know which ones are good. I do have some of them installed from people I trust though.

Finally since I can code them myself, I like the freedom of doing it, kind of when people pass from auto1111 to ComfyUI, the next step after that is to just code what you want, I don't have to care if there's some node for it or if it's possible, I just do it.

I'll add the guide to the readme, thanks for the suggestion, this was an experiment, to publish a space with a guide but I'll start doing it more often, hopefully with also a ComfyUI workflow.

From what I understand, the image is cropped to a square. Can this be fixed to use vertical or horizontal photos?

Hi, this is a proof of concept for a guide I did to teach diffusers, it's a very minimal example to show what I was explaining. To answer your question, of course, this can be improved by a whole lot, you can adapt it to use any kind of images and also, the best improvement that can be made is to zoom in into the masked zone, do the fill and then paste it back on the original image, that way the infill would be a more lot detailed.

If you're asking if I'm willing to do it, probably, but when I have time and this is not my priority right now.

I just found out that the pinokio author added it as an app with what you want: https://x.com/cocktailpeanut/status/1840046540892623213