What is it?

Multi-view diffusion is a type of diffusion model (e.g. Stable Diffusion). However, instead of being trained on regular images, it’s trained on multiple views of an object from different perspectives.

Problems

Multi-view diffusion is useful for 3D.

However, it doesn’t work out-of-the-box. It tends to suffer from something called the Janus problem, where objects have multiple faces (or, more generally, lack of consistency across views).

Solutions

State-of-the-art multi-view-diffusion models like MVDream address this problem using specialized techniques.

I won’t be going into technical detail on multi-view diffusion in this course, since it’s more related to diffusion than to 3D, but if you’d like to learn more, check out the Diffusion course.

In the next sections, we’ll be using a pre-trained multi-view diffusion model to generate multi-view images.

< > Update on GitHub