arXiv AI September 3, 2026 By Khumaisa Nur'aini, Ayu Purwarianti, Alham Fikri Aji, Derry Wijaya

Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languages

The paper introduces a method that adapts Contextual Decomposition for Transformers (CD‑T) to discover neural circuits without relying on clean counterfactuals, using label‑balanced activation means and task‑directional relevance scoring. These circuits are then employed in Circuit‑Targeted Supervised Fine‑Tuning (CT‑SFT), which restricts parameter updates to task‑relevant heads and LayerNorm, leading to competitive performance on low‑resource language adaptation tasks such as NusaX cross‑lingual sentiment transfer. CT‑SFT consistently avoids catastrophic forgetting and preserves source‑language and related‑task performance, offering a more controlled alternative to global fine‑tuning, as further validated on the XNLI benchmark.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Aug 21

EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation

arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.

By Ting-Wei Li, Sirui Chen, Jiaru Zou, Yingbing Huang, Tianxin Wei, Jingrui He, Hanghang Tong

llms benchmarks safety

arXiv Computation and Language

Sep 22

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

arXiv:2606.18389v2 Announce Type: replace Abstract: Large language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated dat...

By Jan Cegin, Daniil Gurgurov, Yusser Al Ghussin, Simon Ostermann

llms fine-tuning

arXiv AI

Oct 7

Cross-Lingual Activation Steering for Multilingual Language Models

arXiv:2601.16390v2 Announce Type: replace-cross Abstract: Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant la...

By Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar

llms benchmarks safety

arXiv Machine Learning

Jul 7

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.

By Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang

llms fine-tuning multimodal

Hugging Face Trending Papers

Jul 6

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities. Standard cross-entropy fine-tuning promotes only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible alternatives, potentially distorting the rich local preference structure learned during pretraining.

llms fine-tuning multimodal

arXiv AI

Aug 6

Calibrating Transformer Attention via Task-Space Sensitivity Feedback

arXiv:2512. 20661v2 Announce Type: replace Abstract: Transformer-based pre-trained language models (PLMs) excel in text classification but suffer from attention dilution and attention sink effects, forcing models to over-focus on task-irrelevant tokens.

By Yawei Liu

llms nlp reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

Cross-Lingual Activation Steering for Multilingual Language Models

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure

Calibrating Transformer Attention via Task-Space Sensitivity Feedback