Inpainting results are not good
#1
by
chuckma
- opened
from diffusers import HunyuanDiT2DControlNetModel, HunyuanDiTControlNetPipeline
import torch
controlnet = HunyuanDiT2DControlNetModel.from_pretrained("chuckma/hunyuan-v1.2-inpaint", torch_dtype=torch.float16)
pipe = HunyuanDiTControlNetPipeline.from_pretrained("Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled",
controlnet=controlnet,
torch_dtype=torch.float16)
pipe.to("cuda")
import requests
from PIL import Image
from io import BytesIO
response = requests.get(
"https://huggingface.co/datasets/OzzyGT/testing-resources/resolve/main/outpainting/313891870-adb6dc80-2e9e-420c-bac3-f93e6de8d06b.png?download=true"
)
control_image = Image.open(BytesIO(response.content))
new_controlnet_image = Image.new("RGBA", control_image.size, "BLACK")
new_controlnet_image.alpha_composite(control_image)
new_controlnet_image
from diffusers.utils import load_image
new_controlnet_image = load_image(new_controlnet_image).resize((1024, 1024))
prompt = "high quality photo of a wolf playing basketball, highly detailed, professional, dramatic ambient light, cinematic, dynamic background, focus"
image = pipe(
prompt,
height=1024,
width=1024,
control_image=new_controlnet_image,
num_inference_steps=25,
).images[0]
image
Here are results, which are quite poor. I don't know why:
condition:
result:
I also tested other cases.Well, it's also not as good as shown in the home page.Here are my cases:
from diffusers.utils import load_image, make_image_grid
prompt = "An Asian man with long sleeve T-shirt, in the background there is a pink pig"
result_image_ = load_image("a_man.png")
final = hunyuan_inpaint_pipe(
prompt,
height=1024,
width=1024,
control_image=result_image_,
num_inference_steps=25,
controlnet_conditioning_scale=1.0,
).images[0]
make_image_grid([result_image_, final], rows=1, cols=2)