Detecting a Shift Is Not Enough: Exact Minimax Limits of Linear Representation Repair
We cast its removal as a statistical decision problem: from noisy differences between paired calibration measurements in $\mathbb{R}^d$, learn one linear map, applied to both sources under a hard distortion budget, that leaves as little of the shift as possible on fresh data.
Key points
- A mean shift between two data sources can be easy to detect but hard to remove without substantially changing their representations.
- We derive the exact finite-sample minimax risk over all such maps, $(d-k) \mathbb{E}[1/(d+2J)]$ with $J\simPois(κ/2)$, where the budget allows deleting $k$ directions and $κ$ is the calibration signal-to-noise ratio.
- This exposes a detection-repair gap: detecting the shift needs only $κ\gg\sqrt d$, whereas removing a fixed fraction of it at constant distortion needs $κ\asymp d$, as for estimating its direction.
- On paired clinical and wearable sleep EEG, where differences between participants act as calibration noise, the formula predicts the device shift left in new participants, and more recordings per person soon stop helping.
Sources (1)
- [1]Detecting a Shift Is Not Enough: Exact Minimax Limits of Linear Representation RepairarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 10:03 AM
We cast its removal as a statistical decision problem: from noisy differences between paired calibration measurements in $\mathbb{R}^d$, learn one linear map, applied to both sources under a hard distortion budget, that leaves as little of the shift as possible on fresh data.
A mean shift between two data sources can be easy to detect but hard to remove without substantially changing their representations.
Extractive summary: sentences quoted from the sources.