arXiv Machine Learning

Model-to-Data Distillation for Graph Neural Networks

arXiv Machine Learning

Jun 9

Heterophily-Aware Adaptive Knowledge Distillation for Hypergraph Neural Networks

arXiv:2606. 08978v1 Announce Type: new Abstract: Hypergraph knowledge distillation aims to retain the predictive performance of a hypergraph neural network (HNN) teacher while reducing inference costs through a lightweight student model.

By Joohee Cho, David Yoon Suk Kang, Yunyong Ko

efficiency

arXiv AI

Aug 26

Too much of a good thing -- when knowledge distillation promotes overfitting, and how to avoid it

The paper investigates how knowledge distillation (KD) applied at intermediate layers of a neural network can affect overfitting and model performance. While traditional KD focuses on the final output, this study explores block‑wise KD across eleven datasets, finding that on standard datasets the last block suffices, but on fine‑grained, data‑scarce settings intermediate supervision significantly improves accuracy. The authors also analyze optimal supervision granularity using attention maps, Centered Kernel Alignment, and Grad‑CAM, and examine teacher‑student fine‑tuning strategies.

By Irene Trigueros-Lorca, Leonardo Concepci\'on, Christian Wagner, Isaac Triguero, Daniel Molina

fine-tuning efficiency safety

arXiv AI

Jun 17

Handling Feature Heterogeneity with Learnable Graph Patches

arXiv:2606. 17667v1 Announce Type: cross Abstract: In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM).

By Yifei Sun, Yang Yang, Xiao Feng, Zijun Wang, Haoyang Zhong, Chunping Wang, Lei Chen

rag

arXiv Machine Learning

Aug 26

TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernel

arXiv:2606.22975v2 Announce Type: replace Abstract: Text-attributed graphs (TAGs) are widely used in many real-world domains, and learning on TAGs requires jointly modeling text semantics and graph s...

By Yeongho Kim, Yeonje Choi, Kijung Shin

llms efficiency benchmarks

arXiv Machine Learning

Jul 21

Node4All: Learning Node Representation Beyond Datasets

arXiv:2607. 17272v1 Announce Type: new Abstract: Node representation learning has advanced rapidly, yet most existing methods rely on per-dataset training and hyperparameter tuning.

By Dooho Lee, Jaemin Yoo

llms benchmarks

arXiv Machine Learning

Sep 2

MUGEN: Generating Unlearnable Graph Examples for Multiple Learning Tasks

MUGEN is a framework that generates unlearnable graph examples capable of protecting multiple downstream tasks—node classification, graph classification, and link prediction—simultaneously. It achieves this by perturbing a single clean dataset with a shared GNN encoder and task‑specific heads, guided by a Task‑Aligned Separability Objective (TASO) and a Type‑Adaptive Perturbation (TAP) that handles both discrete and continuous node attributes. Experiments on five benchmarks, four GNN backbones, and three learning paradigms show that MUGEN’s perturbations transfer across models and remain effective even under adversarial training and data augmentation.

By Ziyan Liu, Chengshuai Zhao, Huan Liu

benchmarks safety

arXiv AI

Aug 6

NodeJEPA: Structure-Conditioned Latent Prediction for Node-Level Graph Self-Supervised Learning

arXiv:2608. 04381v1 Announce Type: cross Abstract: Self-supervised learning on graphs is largely shaped by contrastive methods that depend on carefully designed augmentations, and by generative methods that reconstruct node attributes in the input space.

By Tinghe Zhang, Jian Xu, Jiaheng Chen, Jiaxing Li, Yucheng Xiao, Qiang Wang

rag diffusion fine-tuning benchmarks

arXiv Machine Learning

4d ago

The Universal Classifier for Graph Learning

arXiv:2609.36302v1 Announce Type: new Abstract: While foundation models have revolutionized natural language processing and computer vision by leveraging universal vocabularies, Graph Machine Learnin...

By Ben Finkelshtein, Andr\'{e} Linhares, Petar Veli\v{c}kovi\'{c}, Bryan Perozzi, Mikhail Galkin

diffusion computer-vision nlp

arXiv Machine Learning

Sep 10

Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning

arXiv:2609.08709v1 Announce Type: new Abstract: Designing foundation models for graphs is challenging due to the irregular structure of graphs and the different sizes and characteristics of embedding...

By Jonathan Frank, David Richerby, Ansgar Scherp

llms rag benchmarks

arXiv Machine Learning

Sep 22

Reforge: Low-Latency Distributed GNN Serving with Selective Embedding Recomputation

arXiv:2501.08547v2 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) have been widely adopted for their ability to compute expressive node representations in graph datasets. However...

By Geon-Woo Kim, Donghyun Kim, Jeongyoon Moon, Henry Liu, Tarannum Khan, Anand Iyer, Daehyeok Kim, Aditya Akella

rag benchmarks

arXiv Machine Learning

2d ago

Measuring Structure in Graph Benchmark Datasets Using Graph Invariants

arXiv:2605.06462v2 Announce Type: replace Abstract: Progress in graph learning is hindered by benchmark practices that conflate the contributions of node features and graph structure, making it hard...

By Richard von Moos, Mathieu Alain, Bastian Rieck

llms benchmarks

arXiv Machine Learning

Sep 10

$\alpha$-Graph: Attention-Infused Normalizing Flow Approach to Tractable Graph Modeling

arXiv:2609.07961v1 Announce Type: new Abstract: Graph modeling, a crucial task for representing complex relationships in graph-structured data, has achieved significant success in recent years. Howev...

By Thanh-Dat Truong, Sarah Alharbi, Susan Gauch, Xinghui Zhao, Marios Savvides, Khoa Luu

benchmarks

Related stories