arXiv Machine Learning

In-Cell Learning: Deployed Language Models Can Learn New Knowledge Without Changing a Single Stored Bit

arXiv Machine Learning

2d ago

Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs

arXiv:2608.20873v1 Announce Type: new Abstract: Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint,...

By Zifeng Liu, Zhiyong Du, Yaxin Lu, Yiming Mao, Zhenhe Wang, Wenqi Shi, Zhengkun Jing

llms fine-tuning efficiency

arXiv AI

2d ago

Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs

arXiv:2608.20953v1 Announce Type: cross Abstract: Serving large language models cheaply increasingly means shipping models that are both structurally compressed to a fraction of their parameters and...

By Bakbergen Ryskulov, Iker Garc\'ia-Ferrero, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Rom\'an Or\'us

llms efficiency benchmarks

arXiv Machine Learning

Jul 14

Can a Language Model Learn Facts Continually in Its Weights?

arXiv:2607. 11020v1 Announce Type: cross Abstract: Continual learning promises a language model that keeps acquiring knowledge after training, with each new fact written into its weights.

By Charles O'Neill

llms

arXiv AI

Jul 24

Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing

arXiv:2607. 20433v1 Announce Type: cross Abstract: While language models remain frozen at their training state, the world evolves continuously.

By Jea Kwon, Jiwon Kim, Dong-kyum Kim, Meeyoung Cha

llms reinforcement-learning fine-tuning

arXiv AI

Jun 29

Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents

arXiv:2606. 27472v1 Announce Type: cross Abstract: Large language model (LLM) agents operate over long, multi-session interactions in which facts change: a user moves, a price updates, a plan is revised.

By Vedant Patel

llms agents reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 7

LGQ: Learnable Geometric Quantization for Image Tokenization

arXiv:2602. 16086v3 Announce Type: replace-cross Abstract: Recent collapse-free quantizers such as FSQ achieve stable training by replacing the learnable codebook with an engineered geometry: a fixed scalar grid whose structure is dictated by the codebook size K.

By Idil Bilge Altun, Mert Onur Cakiroglu, Elham Buxton, Mehmet Dalkilic, Hasan Kurban

diffusion nlp efficiency benchmarks

arXiv Computer Vision

1d ago

Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner

arXiv:2608.23268v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-le...

By Jieke Wang, Tiancheng Shen, Yibo Yang, Ming-Hsuan Yang

llms agents multimodal

arXiv Machine Learning

Jul 7

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

arXiv:2607. 02893v1 Announce Type: new Abstract: Low-bit quantization shrinks language models but treats precision as a single global hyper-parameter: every weight uses the same bit-width.

By Hamish Ogilvy

llms efficiency

arXiv Machine Learning

6d ago

Compress and Forget: bitsandbytes Quantization Amplifies Proactive Interference in LLMs

The study investigates how post‑training quantization (PTQ) affects proactive interference (PI) in large language models. Using bitsandbytes, the authors compare FP16, INT8, and INT4/NF4 precision across three instruction‑tuned models and find that INT4 quantization markedly degrades accuracy under high interference, with INT8 also incurring a smaller penalty in two of the three models. The degradation is linked to increased same‑key intrusion errors and originates in the quantized transformer backbone rather than the output layer.

By Shayan Shahrabi-Farahani (Shahid Beheshti University, Tehran, Iran), Dara Rahmati (Shahid Beheshti University, Tehran, Iran)

llms nlp efficiency benchmarks

arXiv Machine Learning

Jul 21

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

arXiv:2607. 16721v1 Announce Type: new Abstract: The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token.

By Anik Jha

llms agents fine-tuning efficiency benchmarks

arXiv AI

Jul 15

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

arXiv:2607. 12463v1 Announce Type: new Abstract: Coding agents must integrate external tool returns into ongoing reasoning - a capability that standard left-to-right pretraining on code exposes only in its forward direction.

By Yubo Wang, Jiarong Liang, Yuxuan Zhang, Xuye Liu, Cong Wei, Yuyu Zhang, Ping Nie, Wenhu Chen

agents benchmarks safety

arXiv Machine Learning

2d ago

When Graph-JEPA Learns the Wrong Thing: Diagnosing and Repairing Category-Conditional Collapse

The paper investigates a failure mode in Graph-JEPA, a joint‑embedding predictive model trained on a large scientific‑reasoning graph. Despite achieving high linear‑probe accuracy and effective rank, the learned representation contains almost no usable instance information, as shown by retrieval metrics. The authors diagnose the issue to variance allocation in the objective, propose a repair that restores near‑perfect information recovery, and demonstrate that the problem persists even after repair, highlighting limitations in the evaluation metrics used.

By Gollam Rabby, S\"oren Auer

rag

Related stories