A Geometry-Based Capacity Theory for Finite-Feature Associative Memory

Key points

• For random or approximately isotropic values, retrieval interference separates into finite-feature noise, which decreases with feature dimension, and structural interference, which is determined by squared kernel overlap among stored keys and persists in the infinite-feature limit.1
• This yields a fit-free prediction of retrieval quality, reveals a geometry-dependent capacity ceiling, and predicts the feature budget required for a target retrieval quality.1
• When stored values are correlated, we show that retrieval depends jointly on the key kernel and value Gram matrix, and derive finite-feature approximations that account for this interaction.1
• We validate the theory on synthetic, visual, and medical-image representations.1

Sources (1)

• [1]A Geometry-Based Capacity Theory for Finite-Feature Associative Memory
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 08:07 PM

  We develop a geometry-based capacity theory for exact-key retrieval in compressed finite-feature Hebbian associative memory.

  For random or approximately isotropic values, retrieval interference separates into finite-feature noise, which decreases with feature dimension, and structural interference, which is determined by squared kernel overlap among stored keys and persists in the infinite-feature limit.

Extractive summary: sentences quoted from the sources.