From Evidence to Action: How Tool-Using Agents Fail
Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand.
ProofPaper ↗
Key points
- We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows.
- Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution.
- For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows.
- A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied.
Sources (1)
- [1]From Evidence to Action: How Tool-Using Agents FailarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 04:50 AM
Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand.
We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows.
Extractive summary: sentences quoted from the sources.