Add pipeline tag
#1
by
nielsr
HF staff
- opened
README.md
CHANGED
@@ -1,5 +1,6 @@
|
|
1 |
---
|
2 |
license: mit
|
|
|
3 |
language:
|
4 |
- en
|
5 |
---
|
@@ -51,8 +52,6 @@ language:
|
|
51 |
<a href="https://arxiv.org/abs/2501.10021">Paper</a>
|
52 |
</p>
|
53 |
|
54 |
-
|
55 |
-
|
56 |
-----
|
57 |
|
58 |
This huggingface repo contains the pretrained models of X-Dyna.
|
@@ -78,9 +77,6 @@ a) IP-Adapter encodes the reference image as an image CLIP embedding and injects
|
|
78 |
</p>
|
79 |
|
80 |
|
81 |
-
|
82 |
-
|
83 |
-
|
84 |
## π Requirements
|
85 |
* An NVIDIA GPU with CUDA support is required.
|
86 |
* We have tested on a single A100 GPU.
|
@@ -88,7 +84,6 @@ a) IP-Adapter encodes the reference image as an image CLIP embedding and injects
|
|
88 |
* **Recommended**: We recommend using a GPU with 80GB of memory.
|
89 |
* Operating system: Linux
|
90 |
|
91 |
-
|
92 |
## 𧱠Download Pretrained Models
|
93 |
Due to restrictions we are not able to release the model pretrained with in-house data. Instead, we re-train our model on public datasets, e.g. [HumanVid](https://github.com/zhenzhiwang/HumanVid), and other human video data for research use, e.g.[Pexels](https://www.pexels.com/). We follow the implementation details in our paper and release pretrained weights and other necessary network modules in this huggingface repository. The Stable Diffusion 1.5 UNet can be found [here](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5) and place it under pretrained_weights/unet_initialization/SD. After downloading, please put all of them under the pretrained_weights folder. Your file structure should look like this:
|
94 |
|
@@ -117,7 +112,6 @@ X-Dyna
|
|
117 |
|----...
|
118 |
```
|
119 |
|
120 |
-
|
121 |
## π BibTeX
|
122 |
If you find [X-Dyna](https://arxiv.org/abs/2501.10021) useful for your research and applications, please cite X-Dyna using this BibTeX:
|
123 |
|
@@ -133,10 +127,6 @@ If you find [X-Dyna](https://arxiv.org/abs/2501.10021) useful for your research
|
|
133 |
}
|
134 |
```
|
135 |
|
136 |
-
|
137 |
## Acknowledgements
|
138 |
|
139 |
-
We appreciate the contributions from [AnimateDiff](https://github.com/guoyww/AnimateDiff), [MagicPose](https://github.com/Boese0601/MagicDance), [MimicMotion](https://github.com/tencent/MimicMotion), [Moore-AnimateAnyone](https://github.com/MooreThreads/Moore-AnimateAnyone), [MagicAnimate](https://github.com/magic-research/magic-animate), [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), [ControlNet](https://arxiv.org/abs/2302.05543), [I2V-Adapter](https://arxiv.org/abs/2312.16693) for their open-sourced research. We appreciate the support from <a href="https://zerg-overmind.github.io/">Quankai Gao</a>, <a href="https://xharlie.github.io/">Qiangeng Xu</a>, <a href="https://ssangx.github.io/">Shen Sang</a>, and <a href="https://tiancheng-zhi.github.io/">Tiancheng Zhi</a> for their suggestions and discussions.
|
140 |
-
|
141 |
-
|
142 |
-
|
|
|
1 |
---
|
2 |
license: mit
|
3 |
+
pipeline_tag: image-to-video
|
4 |
language:
|
5 |
- en
|
6 |
---
|
|
|
52 |
<a href="https://arxiv.org/abs/2501.10021">Paper</a>
|
53 |
</p>
|
54 |
|
|
|
|
|
55 |
-----
|
56 |
|
57 |
This huggingface repo contains the pretrained models of X-Dyna.
|
|
|
77 |
</p>
|
78 |
|
79 |
|
|
|
|
|
|
|
80 |
## π Requirements
|
81 |
* An NVIDIA GPU with CUDA support is required.
|
82 |
* We have tested on a single A100 GPU.
|
|
|
84 |
* **Recommended**: We recommend using a GPU with 80GB of memory.
|
85 |
* Operating system: Linux
|
86 |
|
|
|
87 |
## 𧱠Download Pretrained Models
|
88 |
Due to restrictions we are not able to release the model pretrained with in-house data. Instead, we re-train our model on public datasets, e.g. [HumanVid](https://github.com/zhenzhiwang/HumanVid), and other human video data for research use, e.g.[Pexels](https://www.pexels.com/). We follow the implementation details in our paper and release pretrained weights and other necessary network modules in this huggingface repository. The Stable Diffusion 1.5 UNet can be found [here](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5) and place it under pretrained_weights/unet_initialization/SD. After downloading, please put all of them under the pretrained_weights folder. Your file structure should look like this:
|
89 |
|
|
|
112 |
|----...
|
113 |
```
|
114 |
|
|
|
115 |
## π BibTeX
|
116 |
If you find [X-Dyna](https://arxiv.org/abs/2501.10021) useful for your research and applications, please cite X-Dyna using this BibTeX:
|
117 |
|
|
|
127 |
}
|
128 |
```
|
129 |
|
|
|
130 |
## Acknowledgements
|
131 |
|
132 |
+
We appreciate the contributions from [AnimateDiff](https://github.com/guoyww/AnimateDiff), [MagicPose](https://github.com/Boese0601/MagicDance), [MimicMotion](https://github.com/tencent/MimicMotion), [Moore-AnimateAnyone](https://github.com/MooreThreads/Moore-AnimateAnyone), [MagicAnimate](https://github.com/magic-research/magic-animate), [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), [ControlNet](https://arxiv.org/abs/2302.05543), [I2V-Adapter](https://arxiv.org/abs/2312.16693) for their open-sourced research. We appreciate the support from <a href="https://zerg-overmind.github.io/">Quankai Gao</a>, <a href="https://xharlie.github.io/">Qiangeng Xu</a>, <a href="https://ssangx.github.io/">Shen Sang</a>, and <a href="https://tiancheng-zhi.github.io/">Tiancheng Zhi</a> for their suggestions and discussions.
|
|
|
|
|
|