SqueezeBench: Saturation-Resilient Evaluation of Mathematical Capability

Key points

• AI systems are making rapid progress in mathematics, but this progress is hard to measure.1
• Each problem fixes an integer function $f(n)$ whose values are known only for small $n$, and the benchmark keeps a lower and an upper record on it.1
• A record is a computable Lean function with a machine-checked proof that it bounds $f$.1
• We release ten problems from extremal combinatorics with twenty proved initial records and the admission tools.1

Sources (1)

• [1]SqueezeBench: Saturation-Resilient Evaluation of Mathematical Capability
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 11:59 AM

  We present SqueezeBench, a benchmark that is resilient to saturation: a success raises the bar instead of removing the problem.

  AI systems are making rapid progress in mathematics, but this progress is hard to measure.

Extractive summary: sentences quoted from the sources.