APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces

Key points

• Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and business applications.1
• The benchmark includes six attack categories covering static and dynamic injections, task-specific script replacement, and attacker-provided generic scripts, and evaluates attack success, exposure, and task performance.1
• Opus 5 and Sonnet 5 have the lowest overall attack success rates (8% and 20%), while task-specific script replacement and attacker-provided generic script attacks remain substantially more effective, averaging 81-84% and 73% attack success, respectively.1
• These results show that completing the requested task is not sufficient evidence of safe behavior: an agent can produce a correct deliverable while also performing an unauthorized action in the workspace.1

Sources (1)

• [1]APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 02:05 PM

  We introduce APEX-Harm, a benchmark for evaluating prompt and script injection against long-horizon agents performing finance, consulting, and legal tasks in large workspaces.

  Professional agents increasingly work through command-line interfaces and multiple tools over shared workspaces containing files, messages, and business applications.

Extractive summary: sentences quoted from the sources.