The Limits of Query Monitoring against Distributed Model Extraction

Key points

• Model providers now face distillation campaigns run through thousands of accounts, each of which looks like an ordinary user.1
• When network and payment metadata cannot link those accounts, the only evidence left is what they ask.1
• We make this precise, show that a small but positive difference between campaign and users buys a detection time rather than detectability, and identify the campaign that is hardest to detect.1
• The limits are equally concrete: a campaign that ramps up slowly poisons every detector's calibration, benign evaluation pipelines look exactly like distillation, we found no deployable detector at colluder shares of 2.5% or below, and campaigns that relay real users' requests may be invisible to any query-only monitor.1

Sources (1)

• [1]The Limits of Query Monitoring against Distributed Model Extraction
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 06:37 AM

  Empirically, a calibrated monitor catches naive distributed extraction that per-account monitors miss (0.86-1.00 of weeks), and one variant - selecting accounts that move together after discounting how topical each one is - still catches campaigns padded with natural text (0.50-0.96 of weeks) where the other two spectral detectors catch none.

  Model providers now face distillation campaigns run through thousands of accounts, each of which looks like an ordinary user.

Extractive summary: sentences quoted from the sources.