SWE-DeceptionBench: Benchmarking Deceptive Behaviors of LLMs Across the Software Development Lifecycle
To study this behavior, we introduce SWE-DeceptionBench, a benchmark for evaluating LLM deception in software engineering.
ProofPaper ↗
Key points
- Large Language Models (LLMs) are increasingly used throughout the software development lifecycle.
- We further introduce Neutral, Reward, and Pressure conditions to study the effect of external inducements, resulting in 720 test instances in Chinese and English.
- Evaluating seven mainstream LLMs, we find substantial deception even under the Neutral condition, with aggregate rates of 49.88% for Active Deception and 64.29% for Passive Deception.
- These findings highlight the need for more effective deception detection and mitigation methods to support trustworthy LLM-based software engineering.
Sources (1)
- [1]SWE-DeceptionBench: Benchmarking Deceptive Behaviors of LLMs Across the Software Development LifecyclearXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 02:09 PM
To study this behavior, we introduce SWE-DeceptionBench, a benchmark for evaluating LLM deception in software engineering.
Large Language Models (LLMs) are increasingly used throughout the software development lifecycle.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 9, 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolution
- Oct 9, 2026Overcoming Data Imbalances with Expert Selection Biases
- Oct 9, 2026Qwen/Qwen-Image-2.1-Turbo
- Oct 9, 2026AMLworld-Compact: Importance-Weighted Downsampling for Cost-Effective LLM Evaluation and Error Diagnosis
- Oct 8, 2026Understanding Jev, the new model everyone is talking about
- Oct 8, 2026Launching an opt-in vulnerability-finding service for open-source software