← Back to all news
arXiv Machine Learning By Ziming Dai, Dabiao Ma, Ziheng Guo, Jack Dong, Zimu Zhou

Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search

Read the original on arXiv Machine Learning →

The paper introduces LLM-BlockFE, an offline framework that uses large language models to generate executable feature programs from long unstructured text, eliminating the need for LLM calls during real‑time inference. It builds feature programs by adding immutable code blocks, evaluates candidates with a downstream model, and employs a block‑level rollback mechanism with depth‑calibrated credit allocation to avoid suboptimal greedy search. Experiments on public and private datasets show AUC gains of 0.0069 to 0.0358 over the best baseline, and real‑world deployments in financial risk‑control applications report KS improvements of 0.02 to 1.56 percentage points.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 21

LLMs as Feature Engineers for Text-and-Tabular Prediction

The paper presents an iterative framework that uses large language models (LLMs) to automatically extract interpretable, schema‑bound categorical features from unstructured text for use in tabular prediction models. A generator LLM proposes semantic definitions, an extractor LLM materializes the features, and a downstream tabular model evaluates their predictive performance, with error‑driven natural‑language feedback guiding the search. Across three public datasets, the error‑driven loop speeds up feature discovery up to three times and the resulting features outperform any subset when combined with TF‑IDF and dense embeddings, while also providing instance‑level interpretability through SHAP importance rankings and a semantic audit trail.

By Merwan Barlier, Blaz Skrlj
llmsragsafety
More like this →
arXiv Machine Learning
Aug 31

SymboLLM-FE: LLM-Accelerated Symbolic Regression for Automated Feature Engineering on Tabular Data

SymboLLM-FE combines symbolic regression and large language models to automate feature engineering for tabular data. It first extracts mathematically expressive formulas that correlate strongly with the target, then refines them with LLMs to improve interpretability. Experiments on six real‑world datasets and four Kaggle competitions show that SymboLLM‑FE outperforms existing AutoFE methods while reducing the number of costly LLM calls.

By Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Yu-Feng Li, Lan-Zhe Guo
llmssafety
More like this →
arXiv AI
Aug 13

Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selection

arXiv:2512. 13374v2 Announce Type: replace Abstract: Recent advances in Large Language Models (LLMs) open new perspectives for automation in optimization, yet little is known about whether their internal representations capture problem structure or algorithmic behavior.

By Francesca Da Ros, Luca Di Gaspero, Kevin Roitero
llmsbenchmarks
More like this →