arXiv AI

Co-LMLM: Continuous-Query Limited Memory Language Models

arXiv:2607. 07707v1 Announce Type: cross Abstract: Limited memory language models (LMLMs) externalize factual knowledge during pretraining to a knowledge base (KB), rather than memorizing it in their weights.

arXiv AI

Jun 24

Representation Interventions Enable Lifelong Knowledge Memory Control in LLMs

arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.

By Xuyuan Liu, Shengyu Chen, Xinshuai Dong, Yanchi Liu, Xujiang Zhao, Haoyu Wang, Yujun Yan, Haifeng Chen, Zhengzhang Chen

llms benchmarks

arXiv AI

Jun 26

KARLA: Knowledge-base Augmented Retrieval for Language Models

arXiv:2606. 26807v1 Announce Type: new Abstract: We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during token generation.

By Francois Crespin (IP Paris, LTCI), Fabian M. Suchanek (IP Paris, LTCI), Nils Holzenberger

llms rag

arXiv AI

Aug 11

To Memorize or to Retrieve: Scaling the Interaction Between Pretraining and Retrieval

arXiv:2604. 00715v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) improves language model (LM) performance by providing relevant context at test time for knowledge-intensive situations.

By Karan Singh, Michael Yu, Varun Gangal, Zhuofu Tao, Sachin Kumar, Emmy Liu, Steven Y. Feng

llms rag benchmarks

arXiv AI

Jun 16

Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift

arXiv:2606. 15734v1 Announce Type: cross Abstract: Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities.

By Weihang Su, Jiacheng Kang, Jingyan Xu, Qingyao Ai, Jianming Long, Hanwen Zhang, Bangde Du, Xinyuan Cao, Min Zhang, Yiqun Liu

rag

arXiv Machine Learning

Jun 9

A Unified Structured Query Understanding Framework for Industrial Semantic Search

arXiv:2605. 27441v2 Announce Type: replace-cross Abstract: Query understanding in large-scale industrial search systems is typically implemented as a cascade of disparate, task-specific components.

By Ping Liu, Qianqi Shen, Jianqiang Shen, Chunnan Yao, Kevin Kao, Rajat Arora, Dan Xu, Baofen Zheng, Yunxiang Ren, Benjamin Le, Ali Hooshmand, Igor Lapchuk, Juan Bottaro, Raghavan Muthuregunathan, Caleb Johnson, Liangjie Hong, Jingwei Wu, Wenjing Zhang

llms rag efficiency

arXiv Machine Learning

Jul 13

Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem

arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.

By Amit Peleg, Naman Deep Singh, Naama Pearl, Bibhabasu Mohapatra, Matthias Hein

llms benchmarks safety

arXiv AI

Jun 16

Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning

arXiv:2606. 15598v1 Announce Type: new Abstract: Text-to-SQL aims to translate natural language questions into executable SQL queries over structured databases, enabling non-expert users to access data intuitively.

By Feng Lyu, Jinfeng Cen, Sijing Duan, Hao Wu, Shucheng Li, Weixu Zhang, Haolun Wu

llms fine-tuning benchmarks data-engineering

arXiv AI

1d ago

LLMs Get Smarter from Targeted Synthetic Multilingual Data

arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.

By Ishika Agarwal, Arkajyoti Charaborty, Tanner Sorensen, Neha Gupta, Andreas Stolcke

llms fine-tuning

arXiv AI

Jun 15

Knowledge Graph Enhanced Memory-Augmented Retrieval for Long Context Modeling

arXiv:2606. 14047v1 Announce Type: cross Abstract: Long-context language modeling requires not only extending context windows but maintaining coherent understanding of entity states and relationships across thousands of tokens -- a challenge that semantic similarity alone cannot address.

By Ghadir Alselwi, Basem Suleiman, Hao Xue, Shoaib Jameel, Hakim Hacid, Flora D. Salim, Imran Razzak

llms rag

arXiv AI

Jun 6

Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)

arXiv:2606. 05901v1 Announce Type: cross Abstract: Large language models (LLMs) have fundamentally transformed the landscape of Natural Language Processing.

By Christopher J. Wedge, Joshua Stutter, Danny Dixon, Jacek Ca{\l}a

llms rag agents nlp fine-tuning benchmarks safety

arXiv AI

22h ago

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval

arXiv:2608. 17632v1 Announce Type: cross Abstract: Large language models (LLMs) can both expand underspecified queries and encode text as dense representations, suggesting a unified model for query expansion and retrieval.

By Jingyuan Wang, Richong Zhang, Zhijie Nie, Mingxin Li, Yanzhao Zhang

llms rag benchmarks

arXiv AI

Jun 2

Grokers: Bottom-Up Inductive Comprehension and Write-Time Intelligence over Typed Knowledge Graphs

arXiv:2606. 00050v1 Announce Type: new Abstract: We present Grokers, an architecture for building persistent, structured comprehension of typed knowledge graphs through bottom-up inductive traversal of dependency subgraphs.

By Gregory Magarshak

llms rag agents

Related stories