The paper introduces GONE, a benchmark for evaluating knowledge unlearning in large language models using structured knowledge graphs, and presents Neighborhood-Expanded Distribution Shaping (NEDS), a framework that leverages graph connectivity to separate forgotten facts from their semantic neighborhood. GONE disentangles direct fact removal, reasoning-based leakage, and catastrophic forgetting, while NEDS achieves high unlearning efficacy and locality on LLaMA-3-8B and Mistral-7B. The dataset is publicly available on Hugging Face.
By Chahana Dahal, Ashutosh Balasubramaniam, Zuobin Xiong
The paper introduces EXYGEN, a framework that enables conversational access to large knowledge graphs by combining VoID descriptions, ShEx schemas, retrieved triples, and example question‑query pairs in a retrieval‑augmented generation pipeline. On the SciQA benchmark, this approach achieves an exact‑match score of 0.419 without fine‑tuning any large language model, and shows that larger general‑purpose LLMs can outperform smaller code‑specialized ones when provided sufficient context. To scale metadata generation for very large KGs, the authors propose a predicate‑coverage‑aware parallel graph sampling strategy that preserves structural diversity, reduces runtime by over 80× on OpenCitations Meta and GESIS, and is the only tractable method for obtaining complete metadata on ORKG.
By Harshdeep Singh, Yurui Zhu, Giovanni Colavizza, Matteo Romanello
RePro is a web‑recycling technique that trains a small language model (as little as 1 B parameters) with reinforcement learning to produce high‑quality, faithful rephrasings of pretraining data. The method uses one quality reward and three faithfulness rewards to preserve core semantics and structure while converting organic data into better training examples. Experiments show that RePro boosts downstream accuracy by 3.7–14.5 % over organic‑only baselines and improves data efficiency 2–3×, outperforming prior prompting‑based recycling approaches.
By Zichun Yu, Chenyan Xiong
CORTEX is a novel framework that transforms web‑scale corpus construction from flat document filtering into structured knowledge organization using an Ontological Corpus Graph (OCG). The OCG comprises a quality‑refined content layer, a lightweight ontology layer that evolves via LLMs, and a cross‑domain alignment layer that supports arbitrary taxonomic resolution. Experiments demonstrate CORTEX’s effectiveness, and the authors release a 24.14 B‑token refined corpus, its OCG, and a cross‑domain benchmark called CortexBench for evaluating large language models.
By Chengtao Gan, Xiaoke Guo, Yushan Zhu, Zhaoyan Gong, Zhiqiang Liu, Songze Li, Huajun Chen, Wen Zhang
arXiv:2606. 11499v1 Announce Type: cross Abstract: The performance of modern language models depends critically on pretraining data composition.
By Vedant Badoni, Danqi Chen, Xinyi Wang
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.
By Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein
arXiv:2606. 28601v1 Announce Type: cross Abstract: Recent progress in Text-to-SQL has been driven by stronger language models and prompting strategies, yet performance on real enterprise benchmarks such as Spider 2.
By Jingwen Liu, Weibin Liao, Xin Gao, Junfeng Zhao, Yasha Wang
arXiv:2606. 20089v1 Announce Type: cross Abstract: Persian pretrained language models (PLMs) are still limited by the scarcity of large-scale, high-quality pretraining corpora and by insufficient evaluation beyond standard classification and NER tasks.
By Arash Ghafouri, Mahdi Firouzmandi, Hossein Saberi, Mohammad Reza Hasani Ahangar
arXiv:2606. 15216v1 Announce Type: cross Abstract: Diversity plays a critical role in data selection, improving performance under fixed data budgets by reducing redundancy and repetition.
By Clarence Lee, Yejin Choi, Luke Zettlemoyer, Pang Wei Koh, Hai Leong Chieu
arXiv:2608. 08994v1 Announce Type: cross Abstract: Retrieving relevant evidence from noisy web data is challenging, particularly in sensitive domains containing incomplete reports, heterogeneous language, and irrelevant content.
By Joshua Castillo, Santosh Nukavarapu, Ravi Mukkamala
arXiv:2511. 05385v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) utilizes external knowledge to augment Large Language Models' (LLMs) reliability.
By Chao Zhang, Yuhao Wang, Derong Xu, Haoxin Zhang, Yuanjie Lyu, Yuhao Chen, Shuochen Liu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, Enhong Chen
Retrieval in the SQL setting has largely been studied as the task of finding, within a large collection of SQL statements, the statement that answers a natural-language question. At scale, however, a more fundamental retrieval problem precedes generation: schema retrieval, identifying the tables and columns a question requires in a database that may contain thousands of them, far more than fit in a model's context.