What Output-Only Review Cannot Verify: Study Contracts for Research Agents
We propose study contracts that bind declared experimental choices, run obligations and claim scope to recorded execution evidence, and distinguish this contract-relative verification from scientific truth.
ProofPaper ↗
Key points
- Some defects in an AI-generated study can be identified from its artifacts; others require knowledge of what was approved before execution.
- A diagnostic using eight self-authored clean/mutated pairs illustrates the information boundary.
- A deterministic checker applying a registered, fault-specific rule to approved and executed objects detected all eight registered mutations.
- We identify full-information comparisons, legitimate-adaptation controls and closed-loop agent evaluations as necessary tests of whether contract checks improve useful compliant completion under optimization.
Sources (1)
- [1]What Output-Only Review Cannot Verify: Study Contracts for Research AgentsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:42 AM
We propose study contracts that bind declared experimental choices, run obligations and claim scope to recorded execution evidence, and distinguish this contract-relative verification from scientific truth.
Some defects in an AI-generated study can be identified from its artifacts; others require knowledge of what was approved before execution.
Extractive summary: sentences quoted from the sources.