zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models

Key points

• Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographically proving the full training process is prohibitively expensive at Transformer scale.1
• 2) Then the trainer proves the objective value attained by the committed model on those sequences.1
• We build on sumcheck- and lookup-based arguments to certify Transformer computations, while supporting next-token loss and task-specific audit objectives.1
• Across four model families, operator-level benchmarks yield proving times of 41-59 seconds for 1.1-1.5B-parameter models and 131 seconds at 13B for the covered operators, with verification below half a second at a sequence length of 512.1

Sources (1)

• [1]zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 12:35 PM

  We present zkLLMPoT, a zero-knowledge framework that certifies auditor-defined properties of a trained checkpoint through forward evaluation rather than verification of its optimization trajectory. zkLLMPoT includes 2 phases: 1) The trainer fixes the architecture and the model weights are committed.

  Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographically proving the full training process is prohibitively expensive at Transformer scale.

Extractive summary: sentences quoted from the sources.