HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification

Key points

• DNNs exhibit robustness and generalisation issues not seen in humans.1
• Inductive bias could help with these issues by providing in-built mechanisms to improve generalisation, and hence, reduce reliance on learning from data.1
• Using HAND a ConvNeXt-tiny required 25 training epochs to reach the same accuracy on ImageNet1k as the unmodified model achieved after 200 epochs.1
• Results generalised across CNN architectures and training data-sets.1

Sources (1)

• [1]HAND: A Biologically-Inspired Activation Function that Improves Generalisation and Sample Efficiency in Image Classification
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 09:03 AM

  We incorporate a biologically-inspired inductive bias into a new activation function, HAND (Homeostasis, Accelerating Nonlinearity, and Divisive-nomalisation), and show its effectiveness with CNNs trained on image classification.

  DNNs exhibit robustness and generalisation issues not seen in humans.

Extractive summary: sentences quoted from the sources.