Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild
We introduce Mine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real-world locations.
Key points
- Advances in foundation models are driving efforts to introduce agents to assist people in the physical world.
- Such agents require agentic spatial intelligence: exploring unfamiliar environments, updating spatial understanding through interaction, and adapting actions based on feedback to sustain progress toward a sequence of goals.
- However, the second-best model, Claude Opus 5.5, completes 73.9% of tasks, while the strongest evaluated open-weight model, DeepSeek-V4.1-Flash, reaches 23.9%, highlighting substantial room for improvement in the agentic spatial intelligence of current models.
- Comprehensive analyses and ablation studies on Mine Odyssey reveal current models' limitations and provide insights for advancing agentic spatial intelligence.
Sources (1)
- [1]Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the WildarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 06:26 AM
We introduce Mine Odyssey, a benchmark for evaluating agentic spatial intelligence using Minecraft reconstructions of real-world locations.
Advances in foundation models are driving efforts to introduce agents to assist people in the physical world.
Extractive summary: sentences quoted from the sources.