Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Models
In this paper, we argue that agent safety also depends on identifying required yet unperformed safety-critical actions, which we call obligations.
ProofPaper ↗
Key points
- Guard models are increasingly used to safeguard LLM-based agents, primarily by identifying actions that agents are forbidden to perform.
- However, identifying forbidden actions alone is insufficient to ensure agent safety.
- To close this gap, we introduce ObligationBench, the first benchmark for evaluating the capability of obligation identification, comprising 240 expert-validated trajectories covering issue resolution, feature development, and terminal operations.
- To address these limitations, we develop ObligationGuard using 40,000 synthetic training examples.
Sources (1)
- [1]Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard ModelsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 11:54 AM
In this paper, we argue that agent safety also depends on identifying required yet unperformed safety-critical actions, which we call obligations.
Guard models are increasingly used to safeguard LLM-based agents, primarily by identifying actions that agents are forbidden to perform.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution
- Oct 8, 2026REMORY: Learning Residual Memory for Context Compaction
- Oct 7, 2026How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense
- Oct 6, 2026Socio-Foundation: A Model for Generalizable Individual Behavior Simulation via Hierarchical Capability Distillation
- Oct 6, 2026MINDSET: Energy-based Schema Evolution for Long Conversational Agent Memory