Upload README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,60 @@
|
|
1 |
---
|
2 |
license: creativeml-openrail-m
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: creativeml-openrail-m
|
3 |
+
base_model: kyujinpy/Tune-A-VideKO-disney
|
4 |
+
training_prompt: A bear is playing guitar
|
5 |
+
tags:
|
6 |
+
- tune-a-video
|
7 |
+
- text-to-video
|
8 |
+
- diffusers
|
9 |
+
- korean
|
10 |
+
inference: false
|
11 |
---
|
12 |
+
|
13 |
+
# Tune-A-VideKO-anything
|
14 |
+
Github: [Kyujinpy/Tune-A-VideKO](https://github.com/KyujinHan/Tune-A-VideKO)
|
15 |
+
|
16 |
+
## Model Description
|
17 |
+
- Base model: [kyujinpy/KO-stable-diffusion-disney](https://huggingface.co/kyujinpy/KO-stable-diffusion-disney)
|
18 |
+
- Training prompt: A bear is playing guitar
|
19 |
+

|
20 |
+
|
21 |
+
## Samples
|
22 |
+
|
23 |
+

|
24 |
+
Test prompt: ํ ๋ผ๊ฐ ๊ธฐํ๋ฅผ ์น๊ณ ์์ต๋๋ค, ๋ชจ๋ํ ๋์ฆ๋ ์คํ์ผ
|
25 |
+
|
26 |
+

|
27 |
+
Test prompt: ์์๊ธด ์์๊ฐ ๊ธฐํ๋ฅผ ์น๊ณ ์์ต๋๋ค, ๋ชจ๋ํ ๋์ฆ๋ ์คํ์ผ
|
28 |
+
|
29 |
+

|
30 |
+
Test prompt: ์ฌ์๊ฐ ๊ธฐํ๋ฅผ ์น๊ณ ์์ต๋๋ค, ๋ชจ๋ํ ๋์ฆ๋ ์คํ์ผ
|
31 |
+
|
32 |
+
## Usage
|
33 |
+
Clone the github repo
|
34 |
+
```bash
|
35 |
+
git clone https://github.com/showlab/Tune-A-Video.git
|
36 |
+
```
|
37 |
+
|
38 |
+
Run inference code
|
39 |
+
|
40 |
+
```python
|
41 |
+
from tuneavideo.pipelines.pipeline_tuneavideo import TuneAVideoPipeline
|
42 |
+
from tuneavideo.models.unet import UNet3DConditionModel
|
43 |
+
from tuneavideo.util import save_videos_grid
|
44 |
+
import torch
|
45 |
+
|
46 |
+
pretrained_model_path = "kyujinpy/KO-stable-diffusion-disney"
|
47 |
+
unet_model_path = "kyujinpy/Tune-A-VideKO-disney"
|
48 |
+
unet = UNet3DConditionModel.from_pretrained(unet_model_path, subfolder='unet', torch_dtype=torch.float16).to('cuda')
|
49 |
+
pipe = TuneAVideoPipeline.from_pretrained(pretrained_model_path, unet=unet, torch_dtype=torch.float16).to("cuda")
|
50 |
+
pipe.enable_xformers_memory_efficient_attention()
|
51 |
+
|
52 |
+
prompt = "์ฌ์๊ฐ ๊ธฐํ๋ฅผ ์น๊ณ ์์ต๋๋ค, ๋ชจ๋ํ ๋์ฆ๋ ์คํ์ผ"
|
53 |
+
video = pipe(prompt, video_length=14, height=512, width=512, num_inference_steps=50, guidance_scale=7.5).videos
|
54 |
+
|
55 |
+
save_videos_grid(video, f"./{prompt}.gif")
|
56 |
+
```
|
57 |
+
|
58 |
+
## Related Papers:
|
59 |
+
- [Tune-A-Video](https://arxiv.org/abs/2212.11565): One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
|
60 |
+
- [Stable Diffusion](https://arxiv.org/abs/2112.10752): High-Resolution Image Synthesis with Latent Diffusion Models
|