Non-meshes
While nearly all 3D is represented as meshes in real-world applications today, 3D machine learning research often uses non-mesh representations, which are later converted to meshes.
These non-mesh representations may be things like:
- Triplanes, such as in InstantMesh.
- NeRFs, such as in NeRFiller
- Splats, such as in LGM.
These approaches are constantly evolving and may even have changed by the time you’re reading this.
Fortunately, in most cases, this can be treated as a black box. You don’t need to understand the details of these non-mesh representations to use them in your work.
There is, however, one representation that stands out.
Gaussian Splatting
A special case of non-mesh representation is splats, or Gaussian Splatting.
This is because splats can be rendered in real-time, unlike the other non-mesh representations. They are also capable of features like animation, physics (hybrid), and lighting.
This means that theoretically, splats could replace meshes in real-world applications. However, the entire real-world 3D ecosystem is built around meshes, so it’s unlikely that splats will replace them. They are more likely to have a role in the 3D ecosystem alongside meshes, especially for anticipated applications like real-time generative 3D.
In this course
We’ll be covering both meshes and Gaussian splatting.
While current state-of-the-art uses triplanes, we won’t dive deep into these specifics in this course since they are constantly evolving.
Instead, we will focus on the building blocks of 3D machine learning research. Then, we’ll dive deeper into Gaussian Splatting and meshes, since they can be used in real-world applications today.
< > Update on GitHub