Kernel Autoresearch for Open-Ended Model Discovery

Key points

• Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma.1
• A fixed grammar of base kernels and operators guarantees validity but limits the search to structures expressible by those building blocks.1
• Our experiments demonstrate that the discovered kernels encode reusable inductive biases that generalize to unseen tasks.1
• On held-out black-box optimization families, a discovered kernel outperforms a meta-learned deep kernel trained on the same episodes.1

Sources (1)

• [1]Kernel Autoresearch for Open-Ended Model Discovery
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 04:48 PM

  We propose Kernel Autoresearch (Kernaut), which treats kernel design as open-ended model discovery.

  Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma.

Extractive summary: sentences quoted from the sources.