Introduction
ML-friendly 3D
Let’s take a step back and consider the generative 3D pipeline as a whole.
After multi-view diffusion comes ML-friendly 3D. This is some non-mesh representation of 3D that’s easy for AI to handle.
In the current 3D research ecosystem, this can be a lot of things:
- Gaussian Splatting: Detailed in this unit
- Triplanes: Latest state-of-the-art. Learn more in this Community Notebook
- NeRFs: Synthesize novel views with a neural network
- and more
This is all changing very rapidly. So, like in the case of multi-view diffusion, ML-friendly 3D can be treated like a black box, using pre-trained models.
Gaussian Splatting
In this unit, I’ll be diving deeper into one of these: Gaussian Splatting.
The reason I’m diving deeper into this is that, unlike the other representations, splats can be rendered in real-time, making them suitable for end-to-end 3D applications where everything is AI-compatible.
Let’s get started!
< > Update on GitHub