CopRoute: Dependence-Aware Conformal Routing for Language Models

Key points

• Existing risk-controlled routing policies bound the all-miss risk (the probability that every model in the selected set fails) by calibrating a single conformal threshold and applying it to a nested family of model sets.1
• Models that share training data or architecture tend to fail on the same queries, so two individually strong models can be jointly redundant while a weaker but complementary model covers the queries that the others miss.1
• To account for this dependence, which existing risk-controlled policies ignore, we propose CopRoute, which orders models by how many new queries each one answers correctly beyond those already selected, decides per query how far down that order to go, and then picks the smallest cutoff that still meets the target risk.1
• We show that CopRoute's ordering attains, on the design data, the best guarantee any efficient fixed-ordering algorithm can give for the average number of calls to the first correct model, and that a separate calibration step certifies the all-miss risk of the deployed sets.1

Sources (1)

• [1]CopRoute: Dependence-Aware Conformal Routing for Language Models
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 09:27 AM

  Set-based language model routing aims to increase the coverage of the event where at least one model answers a query correctly while controlling the number of inference calls made.

  Existing risk-controlled routing policies bound the all-miss risk (the probability that every model in the selected set fails) by calibrating a single conformal threshold and applying it to a nested family of model sets.

Extractive summary: sentences quoted from the sources.