The paper investigates a new failure mode in relational in-context learning called support‑set target leakage, where target‑derived features appear only in the labeled support set and not in the query. The authors create 14 synthetic leaker types across 20 columns, evaluate a frozen relational encoder on RelBench databases, and use Integrated Gradients to identify and mitigate the most harmful leakers. Results show that target‑table leakers degrade performance most, while one‑ and two‑hop leakers are less consistently exploited, and that IG can partially recover performance by removing high‑ranked leakers.
arXiv:2609.36417v1 Announce Type: new Abstract: Relational in-context learning (ICL) uses labeled support examples and their linked relational context to predict labels for new queries. This creates...
arXiv:2602. 13697v2 Announce Type: replace-cross Abstract: Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes.
arXiv:2606. 04320v1 Announce Type: cross Abstract: Relational Foundation Models (RFMs) promise a single pre-trained predictor that, given any relational database, returns predictions in one forward pass via relational in-context learning (ICL).
arXiv:2606. 30336v1 Announce Type: new Abstract: We introduce FlexTab, a flexible encoder-decoder architecture for in-context learning on tabular data that pairs a single, task-agnostic encoder with a suite of task-specific decoders.
STEER is a sampling method for relational foundation models that reduces inference cost by focusing on the most relevant tables for a prediction task. It uses a large language model to rank foreign‑key edges in the database schema into relevance tiers, then assigns traversal probabilities based on these tiers. Evaluated on three state‑of‑the‑art RFMs, STEER cuts inference context size by roughly 40% on average while preserving or improving accuracy.
We introduce FlexTab, a flexible encoder-decoder architecture for in-context learning on tabular data that pairs a single, task-agnostic encoder with a suite of task-specific decoders. Unlike existing tabular in-context learners, which entangle feature representations with a specific prediction target, our design produces \textit{target-agnostic} row embeddings that can be leveraged across a wide range of downstream tasks within a table-native in-context learning setup.
The paper evaluates 17 paradigm-level configurations of in‑context learning (ICL) text‑to‑SQL pipelines across five common modules, measuring each module’s marginal accuracy contribution and cost for four different backbone models. It finds that execution‑feedback refinement consistently improves accuracy at low cost, while other modules only help under specific backbone conditions. The study also shows that investing in a more elaborate pipeline for a mid‑tier backbone can be more cost‑effective than upgrading to a high‑capability model with a lean pipeline, providing a tiered, cost‑aware guideline that generalizes to additional backbones.
GEAR is a two‑stage framework that distills tabular foundation models into lightweight MLP or tree‑based predictors for efficient CPU deployment. In the first stage, synthetic covariates are used as teacher‑query locations to train the student on soft TFM targets, expanding coverage beyond observed rows. The second stage re‑anchors the student to the target distribution using real labels and out‑of‑fold teacher predictions, preventing self‑labeling leakage and improving performance. Experiments on TALENT and TabArena show that GEAR‑distilled MLPs outperform supervised MLPs by up to 2.00 AUC points on binary tasks and 1.35 on multiclass tasks, and also outperform CatBoost, while dramatically reducing inference time and memory usage.
arXiv:2609.01292v1 Announce Type: cross Abstract: Relational Deep Learning (RDL) has become a powerful paradigm for learning from multi-tabular data. However, manually defining RDL prediction tasks i...
The paper introduces RefineICL, an attention‑gated, feed‑forward‑network‑free framework that refines representations in situ for tabular foundation models. By using support labels to guide episode‑specific updates, the method transfers learned corrections to unlabeled queries without altering model parameters, achieving state‑of‑the‑art performance on AMLB29 and TabArena benchmarks. Experiments and internal interventions demonstrate that intermediate support updates are essential for constructing task‑specific predictors in context.
RelICL: Training-free Relational Learning with Tabular Foundation Models proposes a new method for relational learning that addresses two key issues of deep feature synthesis—feature explosion and interaction blindness—by propagating and fusing information step by step through the schema graph using a tabular foundation model. The approach retains the benefits of DFS while improving scalability and performance. Experiments on RelBench tasks show that RelICL performs on par with the strongest DFS-based approach.
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.