File size: 3,989 Bytes
f37bfe5 ea5de63 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
license: apache-2.0
---
This repository contains a pruned and partially reorganized version of [CHAMP](https://fudan-generative-vision.github.io/champ/#/).
```
@misc{zhu2024champ,
title={Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance},
author={Shenhao Zhu and Junming Leo Chen and Zuozhuo Dai and Yinghui Xu and Xun Cao and Yao Yao and Hao Zhu and Siyu Zhu},
year={2024},
eprint={2403.14781},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/wZku1I_4L4VwWeXXKgXqb.mp4"></video>
Video credit: [Polina Tankilevitch, Pexels](https://www.pexels.com/video/a-young-woman-dancing-hip-hop-3873100/)
Image credit: [Andrea Piacquadio, Pexels](https://www.pexels.com/photo/man-in-black-jacket-wearing-black-headphones-3831645/)
# Usage
First, install the CHAMP package into your python environment. If you're creating a new environment for CHAMP, be sure you also specify the version of torch you want with CUDA support, or else this will try to run only on CPU.
```sh
pip install git+https://github.com/painebenjamin/champ.git
```
Now, you can create the pipeline, automatically pulling the weights from this repository, either as individual models:
```py
from champ import CHAMPPipeline
pipeline = CHAMPPipeline.from_pretrained(
"benjamin-paine/champ",
torch_dtype=torch.float16,
variant="fp16",
device="cuda",
).to("cuda", dtype=torch.float16)
```
Or, as a single file:
```py
pipeline = CHAMPPipeline.from_pretrained(
"benjamin-paine/champ",
torch_dtype=torch.float16,
variant="fp16",
device="cuda",
).to("cuda", dtype=torch.float16)
```
Follow this format for execution:
```
result = pipeline(
reference: PIL.Image.Image,
guidance: Dict[str, List[PIL.Image.Image]],
width: int,
height: int,
video_length: int,
num_inference_steps: int,
guidance_scale: float
).videos
# Result is a list of PIL Images
```
Starting values for `num_inference_steps` and `guidance_scale` are `20` and `3.5`, respectively.
Guidance keys include `depth`, `normal`, `dwpose` and `semantic_map` (densepose.) This guide does not provide details on how to obtain those samples, but examples are available in [the git repository.](https://github.com/painebenjamin/champ/tree/master/example) |