The paper introduces TOPOFE, a framework that treats automatic feature engineering for tabular data as a graph-structured multi-island evolutionary search. Each island explores a semantically coherent family of transformations using LLM-guided mutation and crossover, while a Prompt Adaptation Memory steers proposals based on accept/reject feedback. TOPOFE dynamically learns a directed topology graph to coordinate cross-island transfer, enabling the discovery of compositional feature programs that outperform state‑of‑the‑art methods on 29 datasets and produce lower redundancy and higher representational coverage.
By Sha Li, Naren Ramakrishnan
arXiv:2607. 27389v1 Announce Type: new Abstract: Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance.
By Bingheng Li, Junyang Cai, Yupeng Zhang, Bistra Dilkina, Jayant Kalagnanam, Dzung T. Phan
arXiv:2606. 02384v1 Announce Type: new Abstract: Progress in tabular machine learning has largely focused on increasingly sophisticated model architectures.
By Andrej Tschalzev, Nick Erickson, Yuyang Wang, Huzefa Rangwala, Stefan L\"udtke, Heiner Stuckenschmidt, Christian Bartelt
SymboLLM-FE combines symbolic regression and large language models to automate feature engineering for tabular data. It first extracts mathematically expressive formulas that correlate strongly with the target, then refines them with LLMs to improve interpretability. Experiments on six real‑world datasets and four Kaggle competitions show that SymboLLM‑FE outperforms existing AutoFE methods while reducing the number of costly LLM calls.
By Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Yu-Feng Li, Lan-Zhe Guo
arXiv:2606. 09004v1 Announce Type: new Abstract: Feature engineering remains essential for tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for automating this process, giving rise to LLM-powered AuTomated Tabular feature Engineering (LATTE).
By Ankai Hao, Ke Chen, Huan Li, Lidan Shou
The paper presents an iterative framework that uses large language models (LLMs) to automatically extract interpretable, schema‑bound categorical features from unstructured text for use in tabular prediction models. A generator LLM proposes semantic definitions, an extractor LLM materializes the features, and a downstream tabular model evaluates their predictive performance, with error‑driven natural‑language feedback guiding the search. Across three public datasets, the error‑driven loop speeds up feature discovery up to three times and the resulting features outperform any subset when combined with TF‑IDF and dense embeddings, while also providing instance‑level interpretability through SHAP importance rankings and a semantic audit trail.
By Merwan Barlier, Blaz Skrlj
arXiv:2607. 16255v1 Announce Type: cross Abstract: A crucial step in machine learning pipelines is to present each entity with features or attributes that are representative of the characteristics of the processed entities.
By Aria Nourbakhsh, Beno\^it Alcaraz, Christoph Schommer
arXiv:2607. 01548v1 Announce Type: cross Abstract: Large language models are increasingly used as open-ended search operators in evolutionary optimization.
By Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz, Petros Mol, Abhimanyu Das, Karthik Duraisamy, Samet Oymak
The paper introduces a framework for learning feature transformations on tabular data that addresses three limitations of prior generative methods: neglect of hierarchical feature relationships, bias from order-sensitive embeddings, and reliance on gradient-based search. It combines a permutation‑invariant hierarchical module using self‑attention pooling to capture interactions across features, operations, and abstraction levels, with a policy‑guided multi‑objective reinforcement learning strategy that starts from strong seeds and optimizes both predictive accuracy and transformation efficiency. Experiments on diverse tabular benchmarks show the approach outperforms strong baselines, and the authors provide public code and data.
By Rui Liu, Tao Zhe, Yanyong Huang, Sankha Narayan Guria, Xiao Luo, Wei Fan, Yanjie Fu, Dongjie Wang
InsightSR is a new framework that integrates Large Language Models (LLMs) with the PySR genetic programming engine to refine symbolic regression search spaces. It employs two LLM-guided pathways: a Semantic Seed Pathway that generates dimensionally consistent functional skeletons, and a Structural Feature Pathway that suggests nonlinear feature transformations. Over successive iterations, these pathways expand the input space and shift the search toward shallow, semantically informed trees, with a feedback loop that evaluates and refines candidate features. The method achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, outperforming existing genetic programming and neural-symbolic approaches while preserving strong out-of-distribution generalization.
By Yating Ling, Wenjing Cun, Zhitang Chen
Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data.
arXiv:2603. 02221v2 Announce Type: replace-cross Abstract: In clinical tabular prediction, classical machine learning models with feature engineering often outperform neural methods.
By Zizheng Zhang, Yiming Li, Justin Xu, Jinyu Wang, Rui Wang, Lei Song, Jiang Bian, David W Eyre, Jingjing Fu