Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error

Key points

• In this work, we propose a post-training lifting method that works with one target class at a time and combines the information coming from all the views.1
• Target and non-target evidence are accumulated simultaneously, weighted by the visibility of each Gaussian in each view.1
• For the evaluation, the labels are transferred from the Gaussians to the mesh vertices that are both visible and annotated.1
• With this design, we can separate three sources of error: the 2D detector, the lifting and the transfer between representations.1

Sources (1)

• [1]Post-Training Semantic Lifting for 3D Gaussian Splatting: Separating Detector, Lifting and Representation Error
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:47 PM

  The same Gaussian of a 3D Gaussian Splatting model is seen from many views, and these views do not always agree on the class it belongs to.

  In this work, we propose a post-training lifting method that works with one target class at a time and combines the information coming from all the views.

Extractive summary: sentences quoted from the sources.