SymboLLM-FE combines symbolic regression and large language models to automate feature engineering for tabular data. It first extracts mathematically expressive formulas that correlate strongly with the target, then refines them with LLMs to improve interpretability. Experiments on six real‑world datasets and four Kaggle competitions show that SymboLLM‑FE outperforms existing AutoFE methods while reducing the number of costly LLM calls.
arXiv:2603. 02221v2 Announce Type: replace-cross Abstract: In clinical tabular prediction, classical machine learning models with feature engineering often outperform neural methods.
arXiv:2603.02221v3 Announce Type: replace-cross Abstract: In clinical tabular prediction, classical machine learning models with feature engineering often outperform neural methods. LLMs are increasi...
arXiv:2606. 02384v1 Announce Type: new Abstract: Progress in tabular machine learning has largely focused on increasingly sophisticated model architectures.
InsightSR is a new framework that integrates Large Language Models (LLMs) with the PySR genetic programming engine to refine symbolic regression search spaces. It employs two LLM-guided pathways: a Semantic Seed Pathway that generates dimensionally consistent functional skeletons, and a Structural Feature Pathway that suggests nonlinear feature transformations. Over successive iterations, these pathways expand the input space and shift the search toward shallow, semantically informed trees, with a feedback loop that evaluates and refines candidate features. The method achieves a 95% exact recovery rate on the Feynman benchmark and 80.18% accuracy on the LLM-SRBench LSR-Transform task, outperforming existing genetic programming and neural-symbolic approaches while preserving strong out-of-distribution generalization.
arXiv:2607. 23286v1 Announce Type: new Abstract: Automatic feature engineering (AutoFE) for tabular learning can be naturally formulated as a program synthesis problem, where the objective is to discover predictive feature transformations from an exponentially large search space.
Neural Symbolic Regression (NSR) uses neural networks as functional preconditioners to learn smooth, noise‑robust approximations of target functions in an interaction‑aware nonlinear feature space. A subsequent LASSO step extracts sparse, interpretable closed‑form expressions, while distributed hyperparameter optimization with Ray Tune and ASHA scheduling improves predictive accuracy and symbolic fidelity. Experiments on the Nguyen benchmark demonstrate that NSR outperforms SINDy and untuned neural baselines in RMSE, noise robustness, and out‑of‑distribution generalization, with ablation studies highlighting the importance of feature interactions, neural depth, and tuning strategies.
arXiv:2607. 27389v1 Announce Type: new Abstract: Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance.
The paper presents an iterative framework that uses large language models (LLMs) to automatically extract interpretable, schema‑bound categorical features from unstructured text for use in tabular prediction models. A generator LLM proposes semantic definitions, an extractor LLM materializes the features, and a downstream tabular model evaluates their predictive performance, with error‑driven natural‑language feedback guiding the search. Across three public datasets, the error‑driven loop speeds up feature discovery up to three times and the resulting features outperform any subset when combined with TF‑IDF and dense embeddings, while also providing instance‑level interpretability through SHAP importance rankings and a semantic audit trail.
The paper introduces LLM-BlockFE, an offline framework that uses large language models to generate executable feature programs from long unstructured text, eliminating the need for LLM calls during real‑time inference. It builds feature programs by adding immutable code blocks, evaluates candidates with a downstream model, and employs a block‑level rollback mechanism with depth‑calibrated credit allocation to avoid suboptimal greedy search. Experiments on public and private datasets show AUC gains of 0.0069 to 0.0358 over the best baseline, and real‑world deployments in financial risk‑control applications report KS improvements of 0.02 to 1.56 percentage points.
arXiv:2606. 09004v1 Announce Type: new Abstract: Feature engineering remains essential for tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for automating this process, giving rise to LLM-powered AuTomated Tabular feature Engineering (LATTE).
arXiv:2607. 01548v1 Announce Type: cross Abstract: Large language models are increasingly used as open-ended search operators in evolutionary optimization.
arXiv:2605.09424v2 Announce Type: replace Abstract: Generative modelling is a demanding test of foundation models, because it requires robust, holistic representation learning for a given data modali...