arXiv AI
Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Impact, Detection, and Mitigation
The paper investigates a new failure mode in relational in-context learning called support‑set target leakage, where target‑derived features appear only in the labeled support set and not in the query. The authors create 14 synthetic leaker types across 20 columns, evaluate a frozen relational encoder on RelBench databases, and use Integrated Gradients to identify and mitigate the most harmful leakers. Results show that target‑table leakers degrade performance most, while one‑ and two‑hop leakers are less consistently exploited, and that IG can partially recover performance by removing high‑ranked leakers.
arXiv AI
Jun 4
OpenRFM: Dissecting Relational In-Context Learning
arXiv:2606. 04320v1 Announce Type: cross Abstract: Relational Foundation Models (RFMs) promise a single pre-trained predictor that, given any relational database, returns predictions in one forward pass via relational in-context learning (ICL).
arXiv Machine Learning
Jun 5
No Need to Train Your RDB Foundation Model
arXiv:2602. 13697v2 Announce Type: replace-cross Abstract: Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes.
arXiv Computation and Language
Sep 21
COAL-SQL: Coverage-Guided Augmentation and Failure-Driven Learning for Text-to-SQL Post-Training
arXiv:2609.20842v1 Announce Type: new Abstract: Text-to-SQL translates natural-language questions into executable SQL queries, but open-source large language models still require task-specific post-t...
arXiv Machine Learning
Sep 24
What Do Tabular Foundation Models Compute In Context? In-Situ Representation Refinement through Attention-Gated Updates
The paper introduces RefineICL, an attention‑gated, feed‑forward‑network‑free framework that refines representations in situ for tabular foundation models. By using support labels to guide episode‑specific updates, the method transfers learned corrections to unlabeled queries without altering model parameters, achieving state‑of‑the‑art performance on AMLB29 and TabArena benchmarks. Experiments and internal interventions demonstrate that intermediate support updates are essential for constructing task‑specific predictors in context.
arXiv Machine Learning
Aug 20
GEAR: Generative Expansion and Real Anchoring for Two-Stage Distillation of Tabular Foundation Models
GEAR is a two‑stage framework that distills tabular foundation models into lightweight MLP or tree‑based predictors for efficient CPU deployment. In the first stage, synthetic covariates are used as teacher‑query locations to train the student on soft TFM targets, expanding coverage beyond observed rows. The second stage re‑anchors the student to the target distribution using real labels and out‑of‑fold teacher predictions, preventing self‑labeling leakage and improving performance. Experiments on TALENT and TabArena show that GEAR‑distilled MLPs outperform supervised MLPs by up to 2.00 AUC points on binary tasks and 1.35 on multiclass tasks, and also outperform CatBoost, while dramatically reducing inference time and memory usage.