Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models

Key points

• Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide.1
• We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD).1
• On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks.1
• Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.1

Sources (2)

• [1]Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
  Hugging Face Daily Papers · Oct 8, 12:00 AM

  Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence.

  Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide.

• [2]Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:22 PM · same content

Extractive summary: sentences quoted from the sources.