arXiv Computation and Language November 4, 2026 By Sajjad Pakdamansavoji, Mohammad Reza Rezaei, Ilija Subasic, Pedro Garzon, Radu Paradovschi, Zhao Chen

Online Label-Free Gradient Matching for Continual Language Model Training

Read the original on arXiv Computation and Language →

The paper introduces OLGA, an online label‑free gradient matching method for continual language model training. OLGA synthesizes examples whose gradients match the model’s own next‑token predictions, enabling efficient updates on small unlabeled data sets. It achieves significant accuracy gains on five text classification benchmarks, outperforms existing selection‑based baselines in both offline and online settings, and scales to large data by combining selection and distillation in a hybrid pipeline.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Hugging Face Trending Papers

Sep 1

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

Large language models (LLMs) struggle to classify text into taxonomies with many semantically similar labels, as the distinctions are domain-specific and not captured by pre-training. To handle large...

llms rag nlp fine-tuning benchmarks

arXiv AI

Sep 2

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

arXiv:2609.01564v1 Announce Type: cross Abstract: Large language models (LLMs) struggle to classify text into taxonomies with many semantically similar labels, as the distinctions are domain-specific...

By Manish Gupta, Chaitanya Giri, Jayasimha Talur

llms rag nlp fine-tuning benchmarks

arXiv Machine Learning

Sep 30

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

arXiv:2609.37076v1 Announce Type: new Abstract: Large language models trained on vast corpora inherently risk memorizing harmful content that may later re-emerge in their outputs. To mitigate this is...

By Puning Yang, Qizhou Wang, Junchi Yu, Bo Han, Xiuying Chen

llms

arXiv Machine Learning

Jul 22

A Classifier That Teaches Itself: Self-Improving, Frozen-gate Training (SIFT) for Dynamic Document Classification

arXiv:2607. 18358v1 Announce Type: cross Abstract: Document classification is a solved problem in the laboratory and an unsolved one in the enterprise.

By Bogdan Raduta, Horia Velicu, Alexandru Preda, Serban Chiricescu

llms safety

arXiv Computation and Language

Oct 1

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters...

By Hao Liang, Zhengyang Zhao, Mingrui Chen, Meiyi Qiang, Lu Ma, Rongyi Yu, Hengyi Feng, Shixuan Sun, Zimo Meng, Xiaochen Ma, Xuanlin Yang, Qifeng Cai, Ruichuan An, Bohan Zeng, Zhen Hao Wong, Chengyu Shen, Runming He, Zhaoyang Han, Yaowei Zheng, Fangcheng Fu, Conghui He, Bin Cui, Zhiyu Li, Weinan E, Wentao Zhang

llms rag benchmarks

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

A Classifier That Teaches Itself: Self-Improving, Frozen-gate Training (SIFT) for Dynamic Document Classification

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning