NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents

Key points

• Prior defenses hand-write rules, query an LLM verifier per action, or compile policies through heavyweight formal machinery.1
• Naive compilation fails: extracted rules block the tool satisfying their own precondition, or read arguments their tool lacks.1
• NOMOS, a four-pass compiler, turns a natural-language policy into a deterministic tool-call gate; static verification with tool-schema-level checks alone (no prover, solver, or LLM) repairs or rejects 37% (airline) and 13% (retail) of candidates, without which most shipped rules are inoperable.1
• On $τ^2$-bench the gate cuts violations of reference-encoded clauses among state-changing calls from 66.3% to 2.6% (airline) and 30.8% to 6.9% (retail), raising airline task success significantly for $2 \le k \le 4$; a 26B on-premise compilation is not significantly worse than hand-written or frontier-compiled rules.1

Sources (1)

• [1]NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 12:23 AM

  Tool-using LLM agents violate the policies they are deployed to enforce, often silently.

  Prior defenses hand-write rules, query an LLM verifier per action, or compile policies through heavyweight formal machinery.

Extractive summary: sentences quoted from the sources.