MotherTree: Meta-learning on synthetic data improves decision tree training
We introduce MotherTree, a tabular transformer that meta-learns decision tree induction: given a training set for a new task, it outputs a hard, axis-aligned decision tree, equivalent in form to classically trained trees, in a single forward pass.
Key points
- Conventional decision tree algorithms produce effective, transparent models that can be audited, communicated, and deployed independently of the training data, but require learning every new task from scratch.
- In contrast, tabular foundation models demonstrate that meta-learning from a synthetic prior distribution enables strong in-context prediction for previously unseen tasks, especially in small-sample regimes.
- MotherTree is pre-trained on a synthetic prior using stochastic gradient descent without requiring reference trees for supervision.
- These results show that meta-learning can provide effective inductive biases for learning stand-alone, small decision tree classifiers.
Sources (1)
- [1]MotherTree: Meta-learning on synthetic data improves decision tree trainingarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:34 PM
We introduce MotherTree, a tabular transformer that meta-learns decision tree induction: given a training set for a new task, it outputs a hard, axis-aligned decision tree, equivalent in form to classically trained trees, in a single forward pass.
Conventional decision tree algorithms produce effective, transparent models that can be audited, communicated, and deployed independently of the training data, but require learning every new task from scratch.
Extractive summary: sentences quoted from the sources.