arXiv AI By Vedant Badoni, Danqi Chen, Xinyi Wang

Hubs or Fringes: Pretraining Data Selection via Web Graph Centrality

Read the original on arXiv AI →

arXiv:2606. 11499v1 Announce Type: cross Abstract: The performance of modern language models depends critically on pretraining data composition.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI
Jun 17

Handling Feature Heterogeneity with Learnable Graph Patches

arXiv:2606. 17667v1 Announce Type: cross Abstract: In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM).

By Yifei Sun, Yang Yang, Xiao Feng, Zijun Wang, Haoyang Zhong, Chunping Wang, Lei Chen
arXiv Machine Learning
Aug 11

PreGress: Ranking-Native Pre-training and Prompting for Graph Node Ranking

arXiv:2608. 09016v1 Announce Type: cross Abstract: Node ranking is a fundamental problem in graph information retrieval, measuring the relative importance of nodes and supporting a wide range of applications such as influence analysis, recommendation, and graph-based retrieval augmented generation.

By Lujie Ban, Jiasheng shi, Yingli Zhou, Kaiwen Xue, Daiyin Wang, Xubin Li, Shuanghua Li, Chenhao Ma
arXiv AI
Jul 28

CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data

arXiv:2607. 22662v1 Announce Type: new Abstract: Open-web corpora curated via highly selective filters, such as FineWeb-Edu and DCLM, constitute the core of LLM pretraining data and have significantly advanced LLM performance.

By Peiguang Li, Yongwei Zhou, Juncheng Diao, Yuchun Fan, Jian Yang, Jianxiao Yang, Zhongda Su, Shuguang Jiao, Xiao Wei, Zhiye Zou, Gan Dong, Zhizhao Zeng, Rongxiang Weng, Jingang Wang, Xunliang Cai
arXiv Machine Learning
Jun 11

GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMs

arXiv:2606. 11898v1 Announce Type: cross Abstract: Research on Text-Attributed Graphs (TAGs) has gained significant attention recently due to its broad applications across various real-world data scenarios, such as citation networks, e-commerce platforms, social media, and web pages.

By Hengyi Feng, Zeang Sheng, Meiyi Qiang, Meiyi Qiang, Wentao Zhang