Subspace Uncertainty and Sharp Sampling Thresholds on the Boolean Cube

Key points

• Random inputs can undersample regions essential for prediction, delaying the parametric rate even when the model is known.1
• For fixed $q0<1/2$, $1\le k\le q0d$, and sufficiently large fixed $A$, the worst-subspace sample threshold for minimax error $Aσ^2(m+t)/n$ with confidence $1-e^{-t}$, $t\ge\log4$, is \[ N=(m+t)\exp\{E{d,k}+O(k^{1/3})\}, \quad E{d,k}=dΨ(k/d), \] where $Ψ(q)=\log2-\mathsf H(\tfrac12-\sqrt{q(1-q)})$ and $\mathsf H$ is binary entropy with natural logarithms.1
• We sharpen the Polyanskiy--Samorodnitsky uncertainty principle in two respects.1
• Second, we construct a subspace of dimension $\binom d{\lfloor k^{1/3}\rfloor}$ such that every function in the subspace has at least a fraction $1-ρ$ of its energy on the same set, whose probability is at most $\exp\{-E{d,k}+C{ρ,q0}k^{1/3}\}$.1

Sources (1)

• [1]Subspace Uncertainty and Sharp Sampling Thresholds on the Boolean Cube
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:23 PM

  We study Gaussian regression under squared population $L_2$ loss in a known $m$-dimensional subspace of degree-at-most-$k$ functions on the $d$-dimensional Boolean cube.

  Random inputs can undersample regions essential for prediction, delaying the parametric rate even when the model is known.

Extractive summary: sentences quoted from the sources.