Hugging Face Trending Papers

How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

A LoRA adapter is a few megabytes that almost everyone treats as a skill rather than a record of the data behind it. We put that assumption on a scale.

arXiv Machine Learning

Jul 24

How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

arXiv:2607. 21351v1 Announce Type: new Abstract: A LoRA adapter is a few megabytes that almost everyone treats as a skill rather than a record of the data behind it.

By Kaizhen Tan, Heqing Du, Yang Feng

reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 10

What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

arXiv:2607. 08032v1 Announce Type: new Abstract: Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions.

By Ashwin Gerard Colaco, Nada Lahjouji

llms agents efficiency benchmarks

arXiv AI

Jul 28

Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

arXiv:2607. 24667v1 Announce Type: new Abstract: A language model with a bounded working memory must repeatedly decide which stored items to keep.

By Maruthi Vemula, Neeraj Praneeth Gajula

llms nlp benchmarks

arXiv Machine Learning

Jun 25

Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory

arXiv:2606. 25115v1 Announce Type: new Abstract: On-device language-model agents improve by accumulating experience in retrieved memory rather than by updating weights.

By Beining Wu, Zihao Ding, Jun Huang, Yanxiao Zhao

llms agents robotics benchmarks

arXiv AI

Jun 19

Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices

arXiv:2606. 19528v1 Announce Type: cross Abstract: Fine-tuning of Large Language Models (LLMs) using Low-Rank Adaptation (LoRA) on an end-user's data offers personalized experiences while keeping data private, but faces severe memory constraints on consumer hardware.

By Hassan Dbouk, Matthias Reisser, Prathamesh Mandke, Likhita Arun Navali, Christos Louizos

llms fine-tuning efficiency

arXiv AI

Jun 29

Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents

arXiv:2606. 27472v1 Announce Type: cross Abstract: Large language model (LLM) agents operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised.

By Vedant Patel

llms agents reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 14

Can a Language Model Learn Facts Continually in Its Weights?

arXiv:2607. 11020v1 Announce Type: cross Abstract: Continual learning promises a language model that keeps acquiring knowledge after training, with each new fact written into its weights.

By Charles O'Neill

llms

arXiv AI

Jun 30

Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs

arXiv:2606. 28479v1 Announce Type: cross Abstract: CSIRTs increasingly fine tune language models on vulnerability scan records, but these records expose internal network topology and create privacy risks under regulations such as GDPR and LGPD.

By Cristhian Kapelinski, Diego Kreutz

llms fine-tuning safety

arXiv Machine Learning

Jul 14

MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers

arXiv:2607. 10034v1 Announce Type: new Abstract: Large language models (LLMs) store factual knowledge in their parameters.

By Roberto Garcia, Jerry Liu, Ronny Junkins, Sabri Eyuboglu, Atri Rudra, Christopher R\'e

llms rag safety

arXiv AI

Jun 6

LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs

arXiv:2606. 06286v1 Announce Type: cross Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use.

By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech

llms safety

arXiv AI

Jun 16

Fine-Tuning a 7B Advisor on Free-Tier GPUs: An Adapter-Handoff Recipe and a Synthetic-Data Reliability Caution

arXiv:2504. 15610v4 Announce Type: replace Abstract: Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on.

By Md Millat Hosen

llms fine-tuning data-engineering

arXiv Machine Learning

Jul 21

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.

By Anik Jha

llms agents fine-tuning efficiency benchmarks

Related stories