arXiv AI September 3, 2026 By Khumaisa Nur'aini, Ayu Purwarianti, Alham Fikri Aji, Derry Wijaya

Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languages

The paper introduces a method that adapts Contextual Decomposition for Transformers (CD‑T) to discover neural circuits without relying on clean counterfactuals, using label‑balanced activation means and task‑directional relevance scoring. These circuits are then employed in Circuit‑Targeted Supervised Fine‑Tuning (CT‑SFT), which restricts parameter updates to task‑relevant heads and LayerNorm, leading to competitive performance on low‑resource language adaptation tasks such as NusaX cross‑lingual sentiment transfer. CT‑SFT consistently avoids catastrophic forgetting and preserves source‑language and related‑task performance, offering a more controlled alternative to global fine‑tuning, as further validated on the XNLI benchmark.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Aug 21

EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation

arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.

By Ting-Wei Li, Sirui Chen, Jiaru Zou, Yingbing Huang, Tianxin Wei, Jingrui He, Hanghang Tong

llms benchmarks safety

arXiv Machine Learning

Jul 7

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.

By Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang

llms fine-tuning multimodal

Hugging Face Trending Papers

Jul 6

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining.

llms fine-tuning multimodal

arXiv AI

Aug 6

Calibrating Transformer Attention via Task-Space Sensitivity Feedback

arXiv:2512. 20661v2 Announce Type: replace Abstract: Transformer-based pre-trained language models (PLMs) excel in text classification but suffer from attention dilution and attention sink effects, forcing models to over-focus on task-irrelevant tokens.

By Yawei Liu

llms nlp reinforcement-learning benchmarks safety

arXiv AI

1d ago

CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations

CroCo introduces cross‑lingual contrastive preference tuning on self‑generations, extending prior English‑only methods to 14 high‑ and low‑resource languages. A reward model trained solely on English preferences, applied to a multilingual base, yields effective within‑language rankings and improves performance in both monolingual and multilingual settings without catastrophic forgetting. The approach requires on‑policy data; off‑policy responses and online preference optimization offer limited gains, yet on structured tasks CroCo matches or surpasses the base model in most languages, and on open‑ended generation it wins 28/30 judge evaluations across 15 languages.

By Mike Zhang, Ali Basirat, Desmond Elliott

llms reinforcement-learning fine-tuning

arXiv Machine Learning

3d ago

Cross-lingual Functional Vectors for Emotion Detection in Large Language Models

arXiv:2608.29613v1 Announce Type: cross Abstract: Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-spe...

By Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, Shogo Okada

llms benchmarks

One email a morning, machine-written

Related stories

EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Calibrating Transformer Attention via Task-Space Sensitivity Feedback

CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations

Cross-lingual Functional Vectors for Emotion Detection in Large Language Models