When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection

Key points

• Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supported by the cited source.1
• Using recent New York State Court of Appeals decisions, we build a dataset of 3,396 parenthetical-style claims labeled as Supported, Refuted, or Not Found.1
• We cast this task as a three-way natural language inference problem and evaluate several state-of-the-art LLMs in a zero-shot setting.1
• Although the strongest models reach up to 0.97 accuracy, the results also show an important weakness: models still incorrectly mark unsupported claims as supported, even when the full opinion text is provided.1

Sources (1)

• [1]When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 10:43 PM

  We introduce PARCEL, a benchmark for checking whether a legal claim is supported by the underlying authority.

  Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supported by the cited source.

Extractive summary: sentences quoted from the sources.