Model Details
This system provides a web demo for the following paper:
VToonify: Controllable High-Resolution Portrait Video Style Transfer (TOG/SIGGRAPH Asia 2022)
- Developed by: Shuai Yang, Liming Jiang, Ziwei Liu and Chen Change Loy
- Resources for more information:
Abstract
Recent advances in face manipulation using StyleGAN have produced impressive results. However, StyleGAN is inherently limited to cropped aligned faces at a fixed image resolution it is pre-trained on. In this paper, we propose a simple and effective solution to this limitation by using dilated convolutions to rescale the receptive fields of shallow layers in StyleGAN, without altering any model parameters. This allows fixed-size small features at shallow layers to be extended into larger ones that can accommodate variable resolutions, making them more robust in characterizing unaligned faces. To enable real face inversion and manipulation, we introduce a corresponding encoder that provides the first-layer feature of the extended StyleGAN in addition to the latent style code. We validate the effectiveness of our method using unaligned face inputs of various resolutions in a diverse set of face manipulation tasks, including facial attribute editing, super-resolution, sketch/mask-to-face translation, and face toonification.
Citation Information
@article{yang2023styleganex,
title = {StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces},
author = {Yang, Shuai and Jiang, Liming and Liu, Ziwei and and Loy, Chen Change},
journal = {arXiv preprint arXiv:2303.06146},
year = {2023},
}