SpatialHarness: Test-Time Spatial Scaffolding for Fine Robotic Manipulation

Key points

• Frontier multimodal foundation models (e.g., GPT-6 Astra) have recently shown strong potential for direct robotic control, yet their performance on fine manipulation remains limited.1
• SpatialHarness maintains an online simulated scene synchronized with real-world execution, identifies task-critical spatial relationships, and renders complementary virtual views that expose them to a frozen multimodal policy.1
• To keep the simulated scene aligned during interaction, we develop interaction-aware scene synchronization that distinguishes static, held, and transition modes.1
• We evaluate SpatialHarness on four real-robot manipulation tasks spanning precise geometric alignment, object-relative placement, and articulated-object interaction.1

Sources (1)

• [1]SpatialHarness: Test-Time Spatial Scaffolding for Fine Robotic Manipulation
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:59 PM

  We introduce SpatialHarness, a test-time embodied harness that provides test-time spatial scaffolding for fine robotic manipulation without policy fine-tuning or changes to the physical sensing setup.

  Frontier multimodal foundation models (e.g., GPT-6 Astra) have recently shown strong potential for direct robotic control, yet their performance on fine manipulation remains limited.

Extractive summary: sentences quoted from the sources.