The paper presents an iterative framework that uses large language models (LLMs) to automatically extract interpretable, schema‑bound categorical features from unstructured text for use in tabular prediction models. A generator LLM proposes semantic definitions, an extractor LLM materializes the features, and a downstream tabular model evaluates their predictive performance, with error‑driven natural‑language feedback guiding the search. Across three public datasets, the error‑driven loop speeds up feature discovery up to three times and the resulting features outperform any subset when combined with TF‑IDF and dense embeddings, while also providing instance‑level interpretability through SHAP importance rankings and a semantic audit trail.
By Merwan Barlier, Blaz Skrlj
arXiv:2608. 05207v1 Announce Type: new Abstract: Frozen pretrained forecasters often fail in structured, recurring ways that are costly to repair through fine-tuning.
By Fangxin Wang, Ziyi Zhang, Diyi Zhuang, Langzhou He, Shiyu Wang, Baichuan Mo, Philip S. Yu
SymboLLM-FE combines symbolic regression and large language models to automate feature engineering for tabular data. It first extracts mathematically expressive formulas that correlate strongly with the target, then refines them with LLMs to improve interpretability. Experiments on six real‑world datasets and four Kaggle competitions show that SymboLLM‑FE outperforms existing AutoFE methods while reducing the number of costly LLM calls.
By Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Yu-Feng Li, Lan-Zhe Guo
arXiv:2607. 23286v1 Announce Type: new Abstract: Automatic feature engineering (AutoFE) for tabular learning can be naturally formulated as a program synthesis problem, where the objective is to discover predictive feature transformations from an exponentially large search space.
By Sha Li, Naren Ramakrishnan
arXiv:2512. 13374v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) open new perspectives for automation in optimization, yet little is known about whether their internal representations capture problem structure or algorithmic behavior.
By Francesca Da Ros, Luca Di Gaspero, Kevin Roitero
arXiv:2607. 27389v1 Announce Type: new Abstract: Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance.
By Bingheng Li, Junyang Cai, Yupeng Zhang, Bistra Dilkina, Jayant Kalagnanam, Dzung T. Phan
arXiv:2607. 01548v1 Announce Type: cross Abstract: Large language models are increasingly used as open-ended search operators in evolutionary optimization.
By Ege Onur Taga, Yilin Zhuang, M. Emrullah Ildiz, Petros Mol, Abhimanyu Das, Karthik Duraisamy, Samet Oymak
arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.
By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique
arXiv:2604. 19753v2 Announce Type: replace Abstract: We propose a feature-free approach to algorithm selection: instead of hand-crafted instance features, we use pretrained text embeddings.
By Stefan Szeider
Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data.
The paper proposes a method for discovering interpretable, decision‑relevant features from unstructured data by leveraging large language models (LLMs). It does so by conducting pairwise comparisons of items, asking an LLM to explain the distinguishing features, and then applying a statistical choice model to aggregate these explanations into feature relevance scores. Experiments on a synthetic hiring dataset and the Reddit ChangeMyView debate dataset demonstrate that the approach outperforms direct LLM feature extraction and matches baseline predictive accuracy while providing interpretable insights.
By Baptiste Arnaudo, Amine Bennouna, S\'ebastien Martin
arXiv:2606. 09004v1 Announce Type: new Abstract: Feature engineering remains essential for tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for automating this process, giving rise to LLM-powered AuTomated Tabular feature Engineering (LATTE).
By Ankai Hao, Ke Chen, Huan Li, Lidan Shou