README.md

---
license: 
- apache-2.0
language:
- en
tags:
- Diffusion Models
- Stable Diffusion
- Perturbed-Attention Guidance
- PAG
---
# Perturbed-Attention Guidance for SD 1.5 (i2i)

The original Perturbed-Attention Guidance for unconditional models and SD1.5 by [Hyoungwon Cho](https://huggingface.co/hyoungwoncho) is availiable at [hyoungwoncho/sd_perturbed_attention_guidance](https://huggingface.co/hyoungwoncho/sd_perturbed_attention_guidance)

[Project](https://ku-cvlab.github.io/Perturbed-Attention-Guidance/) / [arXiv](https://arxiv.org/abs/2403.17377) / [GitHub](https://github.com/KU-CVLAB/Perturbed-Attention-Guidance)

This repository is just a simple implementation of the Perturbed-Attention Guidance (PAG) on Stable Diffusion 1.5 (SD 1.5) for the 🧨 diffusers library to "image-to-image".


## Quickstart

Loading Custom Pipeline:

```py
from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    custom_pipeline="jyoung105/sd15_perturbed_attention_guidance_i2i",
    torch_dtype=torch.float16
)
device="cuda"
pipe = pipe.to(device)
```

Unconditional sampling with PAG:
![image/jpeg](example_1.jpg)

```py
output = pipe(
        "",
        image=init_image,
        strength=0.6,
        num_inference_steps=50,
        guidance_scale=0.0,
        pag_scale=5.0,
        pag_applied_layers_index=['m0']
    ).images
```

Sampling with PAG and CFG:
![image/jpeg](example_2.jpg)
```py
output = pipe(
        "A man with hoodie on is looking at sky, photo",
        image=init_image,
        strength=0.6,
        num_inference_steps=50,
        guidance_scale=0.0,
        pag_scale=5.0,
        pag_applied_layers_index=['m0']
    ).images
```

## Parameters

`guidance_scale` : guidance scale of CFG (ex: `7.5`)

`pag_scale` : guidance scale of PAG (ex: `4.0`)

`pag_applied_layers`: layer to apply perturbation (ex: ['mid'])

`pag_applied_layers_index` : index of the layers to apply perturbation (ex: ['m0', 'm1'])