Conformal Prediction under Partial Verification

Key points

• We develop a partial verification method that returns exactly the same prediction sets as complete verification.1
• We characterize calibration certificates, the verified information sufficient to determine the conformal threshold, and design a procedure that coordinates verification across calibration examples.1
• For finite thresholds at high coverage, its verification cost is less than twice the minimum certificate cost when candidates are checked in order.1
• Across retrieval, mathematical solutions, and configuration evaluation, it reduces verification cost by 15-82% compared with verifying calibration examples one at a time, while producing identical prediction sets.1

Sources (1)

• [1]Conformal Prediction under Partial Verification
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:32 PM

  Conformal prediction provides prediction sets with finite-sample guarantees, but the label verification required for calibration can be expensive.

  We develop a partial verification method that returns exactly the same prediction sets as complete verification.

Extractive summary: sentences quoted from the sources.