Comment by Brusco_RF
4 hours ago
I'm working on the vision component of a drone racing stack. Could I use GS to render my living room as a digital playground to train my vision models in?
I know nothing about the technology but the alternative is creating a 3d model of my living room which is also outside my skill-set.
Splats would work and it’s what a lot of automation folks use. But the risk is that your splats aren’t tight to the surface and that can cause false positives.
For training you can do a hybrid geometry plus splats workflow. Have geometry that you can constantly raycast against and have as an input to your vision training or to get accurate depth buffers.
The workflow for splats and photogrammetry are very similar.
> Could I use GS to render my living room as a digital playground to train my vision models in?
Yes, its what the autonomous car people are doing.
However you might want to do photogrammetry first (https://github.com/alicevision/meshroom opensource) as that produces a mesh which you can use to detect collisions easier. The downside is that transparent objects render really badly. but it is a lot faster to render
Ehhh if it's just for looking and you don't have anything lidar just go for splats they're way better behaved, mostly because they don't need to understand a concept of "surface" they just understand "splat with spherical harmonics of view-dependant color".