arXiv Machine Learning October 8, 2026 By Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

Read the original on arXiv Machine Learning →

The paper introduces a three‑stage framework that uses Large Language Models (LLMs) to generate rules and assemble them into decision trees for few‑shot tabular classification. By distilling LLM knowledge into interpretable trees, the method avoids the high inference costs and limited interpretability of direct LLM application while outperforming traditional decision trees in low‑data settings. Experiments on real‑world datasets show superior accuracy and interpretability with lower prompting overhead compared to existing baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

1d ago

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs...

llms safety

arXiv Machine Learning

Jun 26

LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots

arXiv:2605. 24417v2 Announce Type: replace Abstract: Supervised classification on tabular data remains a central machine learning task, but its dependence on large labeled datasets limits its applicability in data-scarce settings.

By Daria Grushina, Kseniia Kuvshinova, Alina Kostromina, Aziz Temirkhanov, Mile Mitrovic, Dmitry Simakov

llms benchmarks

arXiv Machine Learning

Sep 30

TabFM: A Zero-Shot Foundation Model for Tabular Data

arXiv:2609.37959v1 Announce Type: new Abstract: Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We p...

By Weihao Kong, Erez Louidor Ilan, Shuxin Nie, Taman Narayan, Rajat Sen, Yichen Zhou, Deqing Fu, Samet Oymak, Abhimanyu Das

llms benchmarks

arXiv Machine Learning

Jul 1

Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again

arXiv:2606. 30995v1 Announce Type: new Abstract: Recent work has shown that well-optimized individual decision trees can match complex black box models in some settings, primarily in noisy domains.

By Zakk Heile, Hayden McTavish, Margo Seltzer, Cynthia Rudin

benchmarks safety

arXiv Computation and Language

Oct 1

Marginal Response Surface Elicitation for Zero-Label Tabular Learning

arXiv:2609.39639v1 Announce Type: new Abstract: Tabular learning uses structured data to predict target outcomes. Traditionally, this process has relied on labeled data. However, large language model...

By Liangyu Teng, Yicheng Ding, Jing Liu, Hengsong Liu, Juncen Guo, Hongru Li, Jingyu Zhang, Liang Song

llms benchmarks

arXiv AI

Jun 24

Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules

arXiv:2508. 10971v2 Announce Type: replace-cross Abstract: Knowledge graphs (KGs) can be enhanced through rule mining; however, the resulting logical rules are often difficult for humans to interpret due to their inherent complexity and the idiosyncratic labeling conventions of individual KGs.

By Nasim Shirvani-Mahdavi, Chengkai Li

llms fine-tuning

One email a morning, machine-written

Related stories

From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification

LLMTabBench: Evaluating LLMs on Binary Tabular Classification From Zero to Few Shots

TabFM: A Zero-Shot Foundation Model for Tabular Data

Multistage Defer Trees for Hybrid Interpretability: If at First You Can't Succeed, Tree Again

Marginal Response Surface Elicitation for Zero-Label Tabular Learning

Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules