Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction

Key points

• Feed-forward 3D foundation models reconstruct scenes from unposed images, but the quadratic cost of global attention limits their scalability to large view sets.1
• Across VGGT, HunyuanWorld-Mirror, and $π^3$, we observe a broadly consistent depth-wise pattern: early stages are relatively tolerant to linear replacement, while sensitive middle-to-late stages tend to exhibit more concentrated attention.1
• Zebra3D exploits this structure by assigning linear-centered attention to tolerant stages and tile-sparse attention to sensitive stages, using landmark summaries to preserve unselected context.1
• Across all three backbones, Zebra3D achieves strong reconstruction and camera-pose accuracy at high attention sparsity, with training-free variants remaining close to their dense counterparts.1

Sources (1)

• [1]Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 09:09 PM

  We present Zebra3D, a layerwise sparse-linear hybrid attention framework that improves the efficiency of pretrained 3D foundation models without training from scratch.

  Feed-forward 3D foundation models reconstruct scenes from unposed images, but the quadratic cost of global attention limits their scalability to large view sets.

Extractive summary: sentences quoted from the sources.