← Back to all news
arXiv AI By Mohammad Sadeq Abolhasani, Viswanath Ganapathy

Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations

Read the original on arXiv AI →

The paper investigates whether prior‑data‑fitted networks (PFNs) can be extended to graph‑based relational models. The authors pretrain a message‑passing foundation model, Griffin, on large synthetic relational databases and evaluate its frozen encoder on 16 real‑world relational classification tasks. A curriculum that starts with schema‑guided synthetic data and then uses schema‑agnostic data yields the best in‑context performance, outperforming real‑data‑only pretraining and other mixtures.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 5

When Does Synthetic Relational Data Teach Models to Use Relations? Tracing Predictive Structure from Pretraining Data to Model Behavior

The paper investigates why certain synthetic relational datasets lead to better relational foundation models. By training four Relational Transformer checkpoints on data from four different generators, the authors trace a measurable property of the data—specifically the predictive necessity of cross‑table information—to the emergence of relational computation in the models. They find that the RelDiff generator yields the largest predictive gain from foreign‑key‑linked parents, and its model uniquely responds to foreign‑key interventions, a dependence that persists across random initializations and grows with corrupted links. Disrupting this mechanism during downstream inference eliminates RelDiff’s advantage on relational tasks while leaving structure‑insensitive models largely unchanged, thereby linking synthetic data properties to learned mechanisms and downstream behavior.

By Shivam Dubey, Mohamed Bouadi, Nassim Bouarour, Varun Kulkarni, Aditya Tanna, Vinay Kumar Sankarapu
llmsbenchmarks
More like this →
arXiv AI
Jun 4

OpenRFM: Dissecting Relational In-Context Learning

arXiv:2606. 04320v1 Announce Type: cross Abstract: Relational Foundation Models (RFMs) promise a single pre-trained predictor that, given any relational database, returns predictions in one forward pass via relational in-context learning (ICL).

By Zhikai Chen, Junyu Yin, Jialiang Gu, Siheng Xiong, Xiaoze Liu, Ruowang Zhang, Keren Zhou, Kai Guo
llms
More like this →