yichaodu commited on
Commit
da7c4fa
1 Parent(s): e949b44

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Aligned Diffusion Model via DPO
2
+
3
+ Diffusion Model Aligned with thef following reward model and DPO algorithm
4
+ ```
5
+ close-sourced vlm: claude3-opus gemini-1.5 gpt-4o gpt-4v
6
+ open-sourced vlm: internvl-1.5
7
+ score model: hps-2.1
8
+ ```
9
+
10
+ ## How to Use
11
+
12
+ You can load the model and perform inference as follows:
13
+ ```python
14
+ from diffusers import StableDiffusionPipeline, UNet2DConditionModel
15
+
16
+ pretrained_model_name = "runwayml/stable-diffusion-v1-5"
17
+
18
+ dpo_unet = UNet2DConditionModel.from_pretrained(
19
+ "path/to/checkpoint",
20
+ subfolder='unet',
21
+ torch_dtype=torch.float16
22
+ ).to('cuda')
23
+
24
+ pipeline = StableDiffusionPipeline.from_pretrained(pretrained_model_name, torch_dtype=torch.float16)
25
+ pipeline = pipeline.to('cuda')
26
+ pipeline.safety_checker = None
27
+ pipeline.unet = dpo_unet
28
+
29
+ generator = torch.Generator(device='cuda')
30
+ generator = generator.manual_seed(1)
31
+
32
+ prompt = "a pink flower"
33
+
34
+ image = pipeline(prompt=prompt, generator=generator, guidance_scale=gs).images[0]
35
+
36
+ ```