arXiv Machine Learning
Jun 3
Alignment-Aware Decoding
arXiv:2509. 26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing.
Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts
Ready2Blend is a method that blends natural-language instructions with learned alignment prompts to enable continual alignment of large language models without retraining the backbone. It uses AlignFormer to map each requirement to a fixed-length prompt stored in a modular bank, while keeping the backbone and prior prompts frozen. The approach achieves 93.1–98.5% of joint‑training performance, retains prior knowledge, and reduces training time by up to 4.3×, also allowing weighted personalization and order‑free composition.
Related stories
arXiv Machine Learning
Jul 14
FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
arXiv:2601. 22146v2 Announce Type: replace-cross Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data.
arXiv AI
Aug 11
Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages
arXiv:2603. 12658v2 Announce Type: replace-cross Abstract: Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs.
arXiv AI
Sep 2
Prompt-Robust Language Models: Which Training Strategies Work?
The paper investigates how different training strategies affect the prompt sensitivity of large language models. It reproduces and compares methods such as refined data construction and robustness objectives, finding that while robustness fine‑tuning improves over standard fine‑tuning and in‑context learning, the prompt gap remains large (40–57%). Notably, newer techniques like CoIN and PPCL often underperform a simple data‑construction approach that uses one template per batch, and diagnostics suggest that mixed‑template batches force the optimizer to reconcile conflicting updates rather than learn a prompt‑agnostic representation.
arXiv Machine Learning
Sep 10
RePro: Training Language Models to Faithfully Recycle the Web for Pretraining
RePro is a web‑recycling technique that trains a small language model (as little as 1 B parameters) with reinforcement learning to produce high‑quality, faithful rephrasings of pretraining data. The method uses one quality reward and three faithfulness rewards to preserve core semantics and structure while converting organic data into better training examples. Experiments show that RePro boosts downstream accuracy by 3.7–14.5 % over organic‑only baselines and improves data efficiency 2–3×, outperforming prior prompting‑based recycling approaches.
arXiv Computation and Language
Aug 24
Scaling Unsupervised Word Alignment to Documents via Structural Constraints
arXiv:2608.21023v1 Announce Type: new Abstract: Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full document...
arXiv Computation and Language
Sep 1
REER-PT: Reverse-Engineered Reasoning for Perplexity-Guided Pre-training Data Augmentation
arXiv:2608.30627v1 Announce Type: new Abstract: As language-model compute continues to scale, high-quality training data is becoming an increasingly important bottleneck. Conventional next-token pred...
arXiv AI
Jun 10
PromptEmbedder: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting
arXiv:2605. 28066v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability.
arXiv AI
Jul 29
REPREC: Representation Driven Parameter-Efficient Recommendation System
arXiv:2607. 24845v1 Announce Type: cross Abstract: Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task.
arXiv Computation and Language
Aug 25
CALIBURN: Self-Calibrated LLM Unlearning Alignment
CALIBURN is a new approach to large language model (LLM) unlearning that measures a model’s confidence in undesirable knowledge and uses this measure to fine‑tune unlearning gradient updates. By doing so, it offers more precise control over what is forgotten while better preserving the model’s overall utility. Experiments on benchmarks such as MUSE and WMDP show that CALIBURN outperforms existing methods in balancing knowledge removal with utility retention.
arXiv AI
Jun 10
GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuning
arXiv:2507. 14725v4 Announce Type: replace-cross Abstract: Prompt-based continual learning (CL) offers a parameter-efficient way to adapt large language models (LLMs) across task sequences.
arXiv Computation and Language
Sep 23
DA-Cramming: Enhancing Cost-Effective Language Model Pretraining with Dependency Agreement Integration
The paper introduces DA-Cramming, a cost‑effective pretraining method that incorporates dependency agreement information into BERT‑style language models. It builds on the Cramming technique to enable training with a single GPU in a day, using a dual‑stage workflow and four submodels to embed chunk‑level dependency agreements. Experiments show that this approach outperforms prior methods on a range of tasks.