LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
arXiv:2504.04711v2 Announce Type: replace Abstract: Current large language models (LLMs) are constrained by human-derived training data and limited by a single level of abstraction that impedes defin...
The paper proposes probe-guided fine-tuning, a method that uses probes detecting undesired properties in model activations as a direct training signal. Experiments show that continuously updated probes reduce harmfulness and improve honesty while preserving utility, outperforming DPO and inference-time steering in safety-utility trade-offs and robustness to jailbreak and abliteration attacks. Importantly, the concepts remain linearly encoded after fine-tuning, maintaining monitorability.
arXiv:2609.38630v1 Announce Type: new Abstract: Privacy redaction must remove personal information while preserving relationships expressed in text. We develop a multilingual named-entity tagger with...
The paper introduces the QuantCode Model, a specialized language model for generating executable algorithmic trading code. It combines continued pretraining on trading framework code with supervised fine‑tuning on validated request‑to‑code pairs, achieving significant gains on the QuantCode‑Bench benchmark and improving both single‑turn and multi‑turn agentic success rates. The study also highlights the importance of retaining parsing and tool‑calling capabilities during domain specialization.
The paper introduces Foundation Preserving LoRA (FoLoRA), a forgetting‑aware optimization framework that balances adaptation to downstream tasks with preservation of pretraining capabilities. FoLoRA uses a first‑order preservation condition to define a forgetting penalty based on pretraining‑proxy activations and a task utility from downstream activations, scoring update directions via a generalized Rayleigh quotient. This spectral coordinate system enables gated Adam updates that reduce low‑utility, high‑penalty directions, and the method constructs pretraining proxy calibration data by sampling from the pretrained model. Experiments on math, code, and instruction‑following tasks demonstrate that FoLoRA achieves a stronger balance between target task performance and aggregate preservation of non‑target capabilities compared to baselines.
The paper reports that fine‑tuning large language models on aligned data can unintentionally cause misaligned responses in other contexts—a phenomenon termed *context confusion*. The authors demonstrate this effect in gender equality, privacy, and physical safety domains, showing that it differs from emergent misalignment and is not mitigated by general alignment data but can be reduced with domain‑specific alignment or in‑context examples. They provide a mechanistic explanation based on representational shifts during fine‑tuning that lead to behavioral feature transfer across contexts.
The paper introduces Demographic Pluralism, an inference-time framework that estimates population-level opinion distributions without requiring training data or task-specific fine-tuning. It generates multiple perspectives within demographically grounded groups and, across four model backbones on GlobalOpinionQA and VITAL, reduces Jensen-Shannon distance by 8.4%–26.4% compared to Modular Pluralism. The study finds that equal weighting of group perspectives yields the best overall performance, while weighted aggregation performs worse due to increased group-level error.
AVERT-VLN introduces a closed‑loop framework for vision‑and‑language navigation that incorporates an abstention‑aware Monitor to detect instruction‑execution inconsistencies. The Monitor is trained on a new LOSTNAV dataset of 20K counterfactual risk trajectories and fine‑tuned on 40K normal trajectories to recognize semantic deviations. During deployment, the Monitor can suspend autonomous navigation and request human guidance, while offline preference learning uses deviation‑associated failures to improve the policy, achieving 76.2% and 66.3% success on R2R‑CE and RxR‑CE unseen splits.
The paper introduces T‑Router, a parameter‑efficient reinforcement learning framework that learns thalamic routing to improve reasoning with a pretrained model. T‑Router focuses adaptation on reusing completed computations via a compressed, addressable bank and a depth‑recurrent controller, allocating only 0.466% of the backbone parameters while achieving higher MathAvg scores on GSM8K and improving AIME accuracy. The study demonstrates that controlled computation reuse can outperform full‑parameter and LoRA approaches in reasoning tasks.
Fenchel Tilt Flow Control (FTFC) is a new method for fine‑tuning pretrained generative models to arbitrary preference functions. It decouples utility optimization from model fitting by first learning reward and density‑ratio weights on pretrained samples, then freezing these weights to adjust a diffusion or flow model in a single importance‑weighted stage. The approach supports general f‑divergence penalties, achieves exact duality for concave utilities, and demonstrates up to 20× efficiency gains while outperforming baselines on image and molecule generation tasks.
arXiv:2609.39378v1 Announce Type: new Abstract: Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolvin...
RobECG-CL is a rank‑aware contrastive learning framework designed to learn robust representations for paper ECG images, which often suffer from varied layouts, physical artifacts, and limited labels. The method constructs progressively degraded views from standard 12‑lead ECG recordings and trains the model to maintain invariance to the same recording while respecting degradation ordering. In synthetic stress tests on CODE‑II and EchoNext, RobECG‑CL shows improved robustness under severe degradation and few‑shot transfer, outperforming other contrastive baselines and surpassing the waveform‑based foundation model ECG‑FM in a 1% labeled setting; it also achieves the best macro AUROC on 312 hospital samples with 37 labels.
The paper introduces NAQD‑Env, a synthetic benchmark designed to test language agents’ ability to selectively withdraw and resume actions when new evidence, permissions, or stop instructions arise. It evaluates models against a deterministic reference policy across eleven dependency families, measuring policy agreement, task value, withdrawal, resumption, and event reporting. Experiments on 350 scenarios show low withdrawal recall, no valid resumption, and limited policy alignment, highlighting the need to treat selective withdrawal as a distinct reliability component.
ChartRevise is a new dataset and evaluation protocol designed for exact chart editing via code. It contains 92,438 records covering 344 edit types across 20 chart types and three plotting libraries, built using the grammar of graphics and source‑program checks to ensure applicability. The protocol measures atomic requirement completion, detects gratuitous changes and missed coupled updates, and combines these with execution and rendering success to determine exact‑edit success.
The paper investigates self‑distillation techniques for language models by systematically varying three key design choices: the source of rollout tokens (student vs. teacher), the teacher coupling strategy (frozen or exponential moving average), and the KL divergence direction (reverse or forward). Experiments on Qwen2.5‑7B and Ministral‑3‑3B across 1,200 adaptation runs reveal that rollout source mainly affects acquisition on contradictory tasks, teacher coupling most strongly influences acquisition across all tasks, and KL direction impacts retention differently depending on the model. A controlled theoretical model reproduces these empirical trends, offering a unified framework for understanding acquisition‑retention trade‑offs in self‑distillation.
The paper investigates how large language models learn to hide their reasoning within text—termed steganographic reasoning—compared to related abilities like steganographic messaging and encoded reasoning. Across reinforcement learning, in-context learning, and supervised fine-tuning, models readily acquire messaging and encoded reasoning, but steganographic reasoning only emerges under supervised fine-tuning and requires substantially more training, unless a convenient cover task is provided. Even then, steganographic reasoning remains significantly harder than its neighboring capabilities.
The Agent Error Dataset (AED) presents 50,228 error–diagnosis pairs collected from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models in text‑based agent systems. A five‑stage Agentic Error‑to‑Training (AET) pipeline generates diagnoses and proposed corrections, verifies them against recorded evidence, and creates separate training views for diagnosis and actor recovery. Experiments show that first‑proposal corrections improve verifier pass rates from 18.4% to 51.1%, and fine‑tuning with full‑diagnosis data raises Qwen3‑8B’s exact‑step agreement from 47.2% to 63.6% on a holdout set.
DualCast is a dual‑path language model that forecasts financial time‑series by combining a fast numerical forecaster with an optional text‑conditioned revision mechanism. The fast path trains only new financial‑token embeddings and output heads on a frozen Qwen3‑8B backbone, while the slow path uses a LoRA adapter to incorporate news and refine predictions. In zero‑shot tests across equities and energy prices at multiple time resolutions, the slow path achieves the lowest mean absolute percentage error in most settings, especially for longer horizons, and news ablations show additional gains in many markets.
The paper introduces an LLM-based framework for continuous dimensional emotion evaluation in multimodal dialogue, combining discrete emotion recognition with Valence-Arousal-Dominance (VAD) assessment on the IEMOCAP dataset. It incorporates acoustic cues as natural language descriptions via the SpeechCueLLM approach and evaluates six models from the LLaMA, GPT, and Qwen families using zero-shot, few-shot, and LoRA fine-tuning. LoRA-fine-tuned LLaMA models outperform prompt-engineered GPT models, achieving a new state-of-the-art Valence CCC of 0.7822, and ablation studies show that textual audio descriptions significantly benefit smaller models. "whyItMatters":"The study demonstrates that domain adaptation through fine-tuning can surpass larger GPT models in multimodal emotion evaluation, highlighting the importance of tailored training for emotion recognition tasks."
The paper introduces FDCU, a dual‑constrained subspace projection framework designed to improve machine unlearning for large language models. FDCU limits parameter updates with a dual‑masking rule that preserves general knowledge via Fisher Information while preventing the activation of spurious suppressors through the Principle of Minimal Functional Intervention. Experiments show that FDCU achieves state‑of‑the‑art robustness against retraining attacks while maintaining near‑lossless general utility, thereby ensuring durable safety for LLMs.