arXiv AI
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
Oct 13
RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions
Foundation models for tables and relational databases are increasingly used for predictive tasks across various scientific and industrial domains. However, adapting existing models to unseen datasets...
arXiv Machine Learning
Oct 3
STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling
STEER is a sampling method for relational foundation models that reduces inference cost by focusing on the most relevant tables for a prediction task. It uses a large language model to rank foreign‑key edges in the database schema into relevance tiers, then assigns traversal probabilities based on these tiers. Evaluated on three state‑of‑the‑art RFMs, STEER cuts inference context size by roughly 40% on average while preserving or improving accuracy.
arXiv AI
Sep 15
Beyond Quacking: Deep Integration of Language Models and RAG into DuckDB
The paper introduces FlockMTL, an extension for database management systems that deeply integrates large language models and retrieval‑augmented generation into DuckDB. It provides model‑driven scalar and aggregate functions, cost‑based optimizations like batching and caching, and new SQL DDL abstractions (PROMPT and MODEL) to treat LLMs as first‑class schema objects. These features aim to simplify the development of knowledge‑intensive analytical applications by reducing the effort required to orchestrate heterogeneous data systems and manage LLM context.
arXiv AI
Jun 4
OpenRFM: Dissecting Relational In-Context Learning
arXiv:2606. 04320v1 Announce Type: cross Abstract: Relational Foundation Models (RFMs) promise a single pre-trained predictor that, given any relational database, returns predictions in one forward pass via relational in-context learning (ICL).
arXiv Machine Learning
Jun 5
No Need to Train Your RDB Foundation Model
arXiv:2602. 13697v2 Announce Type: replace-cross Abstract: Relational databases (RDBs) contain vast amounts of heterogeneous tabular information that can be exploited for predictive modeling purposes.
arXiv Machine Learning
Aug 3
PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining
arXiv:2607. 29129v1 Announce Type: new Abstract: Relational Foundation Models (RFMs) require large-scale synthetic relational databases for pretraining, but existing approaches tightly couple data generation with the model training pipeline.