Characterizing Overconfident Failure in LLM-Based Code Generation
Large language models (LLMs) are increasingly used for automated code generation, but generated programs can appear syntactically plausible while still failing execution-based correctness checks.
ProofPaper ↗
Key points
- This paper studies the dilemma of overconfidence in code LLMs where incorrect programs are often generated with token-level confidence comparable to correct programs.
- We study this dilemma across four open-source code models and three execution-based benchmarks.
- Our analysis begins by investigating whether existing uncertainty metrics provide reliable proxies for execution correctness in code generation.
- We then characterize overconfidence at both global and local token levels, asking whether incorrect programs remain indistinguishable from correct ones under confidence and entropy summaries, including selective generation and the limits of instruction tuning.
Sources (1)
- [1]Characterizing Overconfident Failure in LLM-Based Code GenerationarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 06:07 AM
Large language models (LLMs) are increasingly used for automated code generation, but generated programs can appear syntactically plausible while still failing execution-based correctness checks.
This paper studies the dilemma of overconfidence in code LLMs where incorrect programs are often generated with token-level confidence comparable to correct programs.
Extractive summary: sentences quoted from the sources.