ProtocolMatch: Protocol-Dependent Model Selection for Scientific Dynamics Forecasting

Key points

• Scientific dynamics forecasting is often framed as an architecture choice, although deployment is also determined by observed history, rollout feedback, compute budget, physical objective, and test distribution.1
• On driven quantum-spin dynamics, we compare recurrent, patched-attention, causal-attention, and low-rank linear predictors across three independently generated datasets.1
• The causal-attention--recurrence ordering reverses as the training set grows within a fixed two-spin task, while a linear predictor has the lowest mean error in the six-spin local-observable comparison.1
• Thus scientific model selection should return a predictor with its protocol and report accuracy, physical validity, and shifted-distribution reliability separately.1

Sources (1)

• [1]ProtocolMatch: Protocol-Dependent Model Selection for Scientific Dynamics Forecasting
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 03:23 PM

  We formulate protocol-dependent model selection and introduce ProtocolMatch, a compute-matched, validation-selected, and failure-preserving evaluation framework.

  Scientific dynamics forecasting is often framed as an architecture choice, although deployment is also determined by observed history, rollout feedback, compute budget, physical objective, and test distribution.

Extractive summary: sentences quoted from the sources.