ST-Bench: A Spatial-Temporal Benchmark for Multi-Agent System Generation on Scientific Research Tasks
We introduce ST-Bench, a benchmark designed to answer two questions: whether MAS outperform single agents on complex scientific data analysis tasks, and if so, by how much and at what additional cost.
Key points
- The rapid progress of LLM-based multi-agent systems (MAS) has shown that they largely outperform single agents on coding, math, and QA tasks, where executable tests provide a binary success signal.
- Whether this advantage transfers to real scientific data analysis remains untested.
- ST-Bench contains 100 data science tasks adapted from published Earth science studies across hydrology, agriculture, and wetland methane research, expanded into 2,067 queries grounded in additional published studies and validated by domain experts.
- Using ST-Bench, we evaluate five recent MAS generation methods under two training protocols, against single-agent baselines on the same GPT-5 backbone.
Sources (1)
- [1]ST-Bench: A Spatial-Temporal Benchmark for Multi-Agent System Generation on Scientific Research TasksarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:59 AM
We introduce ST-Bench, a benchmark designed to answer two questions: whether MAS outperform single agents on complex scientific data analysis tasks, and if so, by how much and at what additional cost.
The rapid progress of LLM-based multi-agent systems (MAS) has shown that they largely outperform single agents on coding, math, and QA tasks, where executable tests provide a binary success signal.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 4, 2026nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2
- Oct 2, 2026Chatham scales its capital markets expertise with OpenAI
- Oct 1, 2026Claude-shaped science
- Sep 29, 2026Introducing GPT-6.1 Sol
- Sep 28, 2026Holo4: powering generalist computer-use agents
- Sep 28, 2026Basis completes a tax workbook 2x faster with GPT-6 Astra