Spaces:
Running
on
Zero
Seek help when encountering problems
Hello, first of all, thank you very much for providing this space as a developer. This is the best inpaint model I have ever used.
I want to run your model on AWS g6.xlarge, but after clicking inpaint, I found an error message. Here is my error message. Can you help me find out where the problem is? Thank you.
usr/local/lib/python3.10/dist-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.42.0, however version 4.44.1 is available, please upgrade.
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/gradio/queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/dist-packages/gradio/route_utils.py", line 321, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1935, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1532, in call_function
prediction = await utils.async_iteration(iterator)
File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 671, in async_iteration
return await iterator.anext()
File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 664, in anext
return await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 647, in run_sync_iterator_async
return next(iterator)
File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 809, in gen_wrapper
response = next(iterator)
File "/root/diffusers-fast-inpaint/./app.py", line 55, in fill_image
) = pipe.encode_prompt(prompt, "cuda", True)
File "/root/diffusers-fast-inpaint/pipeline_fill_sd_xl.py", line 158, in encode_prompt
prompt_embeds = text_encoder(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py", line 1490, in forward
text_embeds = self.text_projection(pooled_output)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py", line 117, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::Half
Oh, to fix this, you might have to modify the app.py file.
Change this line
pipe = StableDiffusionXLFillPipeline.from_pretrained(
"SG161222/RealVisXL_V5.0_Lightning",
torch_dtype=torch.float16,
vae=vae,
controlnet=model,
variant="fp16",
).to("cuda")
to
pipe = StableDiffusionXLFillPipeline.from_pretrained(
"SG161222/RealVisXL_V5.0_Lightning",
vae=vae,
controlnet=model,
variant="fp16",
).to("cuda", torch.float16)
This should fix the problem. I haven't investigated the problem, but it seems the dtype argument for the pipeline isn't properly casting the model weights (this might be a diffusers issue). So directly allowing pytorch to handle the casting fixed the problem for me.
thanks a lot! I've seen this problem and it seems it's because of a recent update with transformers that introduced this breaking change, this space is safe because the version is locked but I'll try to test it and fix it when I have the time.
I really appreciate your feedback.
Thank you very much. With your help, my code has started running.
I have another question, can only support 512 * 512 images now? Can you support rectangular diagrams?