Navier-Stokes lost in translation: Why Lean verification of AI autoformalisation does not guarantee correct natural language proofs

Key points

• In this process, an AI system translates the text from a natural language (NL) into a formal language such as Lean.1
• In particular, we highlight that the problem of resolving ambiguities in mathematical NL text, which is necessary in order to provide semantically faithful translation, is arbitrarily high up in the Solvability Complexity Index (SCI) hierarchy/arithmetical hierarchy (the SCI $= \infty$).1
• To demonstrate the effect of this result we provide several examples of AI mistranslations of NL statements and proofs into Lean in practice, resulting in mismatches between NL proofs and their Lean verifications'.1
• In particular, we show that the formalised Lean proof does not correspond to the NL proof of blow-up of solutions to the Navier-Stokes equations.1

Sources (1)

• [1]Navier-Stokes lost in translation: Why Lean verification of AI autoformalisation does not guarantee correct natural language proofs
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 10:58 AM

  Autoformalisation is increasingly used to verify mathematical texts, including those generated by AI, as in OpenAI's announced proof of blow-up of solutions to the Navier-Stokes equations.

  In this process, an AI system translates the text from a natural language (NL) into a formal language such as Lean.

Extractive summary: sentences quoted from the sources.