arXiv:2606. 30995v1 Announce Type: new Abstract: Recent work has shown that well-optimized individual decision trees can match complex black box models in some settings, primarily in noisy domains.
By Zakk Heile, Hayden McTavish, Margo Seltzer, Cynthia Rudin
The paper presents a scalable method for meta‑learning interpretable decision trees by generating synthetic pre‑training data. It samples near‑optimal trees to create large, realistic datasets and uses the MetaTree transformer architecture to match the performance of real‑world pre‑training or costly optimal tree training. This approach cuts computational costs, offers flexible data generation, and supports efficient, scalable meta‑learning of decision trees.
By Kyaw Hpone Myint, Zhe Wu, Alexandre G. R. Day, Giri Iyengar
The paper introduces a three‑stage framework that uses Large Language Models (LLMs) to generate rules and assemble them into decision trees for few‑shot tabular classification. By distilling LLM knowledge into interpretable trees, the method avoids the high inference costs and limited interpretability of direct LLM application while outperforming traditional decision trees in low‑data settings. Experiments on real‑world datasets show superior accuracy and interpretability with lower prompting overhead compared to existing baselines.
By Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li
While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs...
arXiv:2605. 18147v2 Announce Type: replace Abstract: Predictive models play a pivotal role in credit risk management, guiding critical decisions through accurate estimation of default probabilities and losses.
By Bart Baesens, Andreas Goethals, Stefan Lessmann, Simon De Vos, Cristi\'an Bravo, David Martens, Victor Medina-Olivares, Christophe Mues, Maria Oskarsd\'ottir, Seppe vanden Broucke, Tony Van Gestel, Tim Verdonck, Wouter Verbeke
MotherTree is a tabular transformer that meta‑learns to produce hard, axis‑aligned decision trees in a single forward pass. It is pre‑trained on a synthetic prior distribution using stochastic gradient descent, without needing reference trees for supervision. On controlled benchmarks, MotherTree matches or surpasses size‑matched trees from traditional algorithms and serves as a strong initializer for task‑specific fine‑tuning.
By Ziyuan Wang, Fredrik D. Johansson