README.md

---
license: apache-2.0
---
This repository contains a pruned and partially reorganized version of [CHAMP](https://fudan-generative-vision.github.io/champ/#/).

```
@misc{zhu2024champ,
      title={Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance}, 
      author={Shenhao Zhu and Junming Leo Chen and Zuozhuo Dai and Yinghui Xu and Xun Cao and Yao Yao and Hao Zhu and Siyu Zhu},
      year={2024},
      eprint={2403.14781},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
```

<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64429aaf7feb866811b12f73/wZku1I_4L4VwWeXXKgXqb.mp4"></video>

Video credit: [Polina Tankilevitch, Pexels](https://www.pexels.com/video/a-young-woman-dancing-hip-hop-3873100/)

Image credit: [Andrea Piacquadio, Pexels](https://www.pexels.com/photo/man-in-black-jacket-wearing-black-headphones-3831645/)

# Usage

First, install the CHAMP package into your python environment. If you're creating a new environment for CHAMP, be sure you also specify the version of torch you want with CUDA support, or else this will try to run only on CPU.

```sh
pip install git+https://github.com/painebenjamin/champ.git
```

Now, you can create the pipeline, automatically pulling the weights from this repository, either as individual models:

```py
from champ import CHAMPPipeline
pipeline = CHAMPPipeline.from_pretrained(
  "benjamin-paine/champ",
  torch_dtype=torch.float16,
  variant="fp16",
  device="cuda"
).to("cuda", dtype=torch.float16)
```

Or, as a single file:

```py
from champ import CHAMPPipeline
pipeline = CHAMPPipeline.from_single_file(
  "benjamin-paine/champ",
  torch_dtype=torch.float16,
  variant="fp16",
  device="cuda"
).to("cuda", dtype=torch.float16)
```

Follow this format for execution:

```py
result = pipeline(
  reference: PIL.Image.Image,
  guidance: Dict[str, List[PIL.Image.Image]],
  width: int,
  height: int,
  video_length: int,
  num_inference_steps: int,
  guidance_scale: float
).videos
# Result is a list of PIL Images
```

Starting values for `num_inference_steps` and `guidance_scale` are `20` and `3.5`, respectively.

Guidance keys include `depth`, `normal`, `dwpose` and `semantic_map` (densepose.) This guide does not provide details on how to obtain those samples, but examples are available in [the git repository.](https://github.com/painebenjamin/champ/tree/master/example)