Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,201 stories · RSS feed

arXiv AI

Oct 5

Not Until the Evidence Says So: Teaching LLM Investigators When to Close a Case

arXiv:2610.03190v1 Announce Type: cross Abstract: Accident, defect and outage investigations end with a decision that ordinary question answering never faces: whether the evidence gathered so far is...

By Tingzhu Bi, Ping Wang, Meng Ma

llms nlp reinforcement-learning fine-tuning safety

arXiv AI

Oct 5

Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval

arXiv:2610.02438v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong performance in code generation, where success depends on both recalling relevant algorithmic kn...

By Nickil Maveli, Antonio Vergari, Shay B. Cohen

llms fine-tuning benchmarks

arXiv AI

Oct 5

FinDialogLens: Event Extraction over Multi-Party Dialogue for Missed-Trade Identification in Financial Chatrooms

arXiv:2610.02455v1 Announce Type: cross Abstract: Multi-party financial chatrooms are vital for sales-and-trading professionals, but their complexity makes manual recovery of missed trades infeasible...

By Chin-Lun Fu, Hong Ni, Behrouz Madahian

llms fine-tuning

arXiv Machine Learning

Oct 5

BaCP: Backbone Contrastive Pruning for Preserving Representations in Extremely Sparse Neural Networks

arXiv:2610.02524v1 Announce Type: new Abstract: Unstructured pruning at extreme sparsity often suffers from representational collapse, causing sharp drops in accuracy. To address this, we study Backb...

By Mohammad Haroon Khawaja, Muhammad Haseeb, Mohammad Fatim Shoaib, Muhammad Tahir

rag fine-tuning efficiency

arXiv Computation and Language

Oct 5

Specializing Without Forgetting: Analyzing Knowledge Preservation in Multilingual Model Adaptation

arXiv:2606.00284v2 Announce Type: replace Abstract: While continual pretraining (CPT) is a practical way to extend large language models to new languages, na\"ive finetuning often erodes existing cap...

By Sanchit Ahuja, Terra Blevins

llms fine-tuning

arXiv AI

Oct 5

Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective

arXiv:2610.03185v1 Announce Type: new Abstract: On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also col...

By Han Cui, Jianhao Yan, Yun Luo, Hongbo Zhang, Zhizhang Fu, Yue Zhang

llms reinforcement-learning fine-tuning efficiency

arXiv AI

Oct 5

Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite

arXiv:2610.02826v1 Announce Type: new Abstract: Successful trajectories on difficult tasks provide valuable supervision for model improvement, but specialized harnesses introduce interventions that m...

By Zongxia Li, Yucheng Shi, Zhongzhi Li, Junyao Yang, Ruhan Wang, Chengsong Huang, Fuxiao Liu, Haitao Mi, Jordan Boyd-Graber, LeoweiLiang

fine-tuning

arXiv AI

Oct 5

MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile Manipulation

arXiv:2610.03476v1 Announce Type: cross Abstract: Long-horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and...

By Chenzhi Liu, Yue Zhang, Jiehong Lin, Jianan Wang, Bo Wang, Zhongrui Wang, Xiaojuan Qi

llms agents robotics fine-tuning multimodal

arXiv AI

Oct 5

KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning

arXiv:2605.14907v2 Announce Type: replace Abstract: Knowledge graph (KG) foundation models aim to generalize to graphs with unseen entities and relations by learning transferable relational structure...

By Yisen Gao, Jiaxin Bai, Haoyu Huang, Zhongwei Xie, Yufei Li, Hong Ting Tsang, Sirui Han, Yangqiu Song

llms fine-tuning

arXiv AI

Oct 5

A Language Model from 1913: Pretraining on Historical Text

arXiv:2606.02991v2 Announce Type: replace-cross Abstract: While modern language models increasingly rely on ever-larger web corpora, we show that pretraining on historical text (e.g., pre-1913 text)...

By Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber, Yixuan Wang, Junchi Yu, Freda Shi, Philip Torr, Yao Lu

llms fine-tuning benchmarks data-engineering

arXiv Computation and Language

Oct 5

Text-Centric Post-Training for Omni-Modal Reasoning

arXiv:2610.02819v1 Announce Type: new Abstract: Improving joint audio-visual reasoning in Omni Large Language Models typically incurs substantial data construction and training costs. Our diagnostics...

By Ziyang Cheng, Yuhao Wang, Hongcheng Liu, Qimin Wu, Jingru Fan, Chen Qian, Yanfeng Wang, Yu Wang

llms reinforcement-learning fine-tuning multimodal

arXiv Computation and Language

Oct 5

Unmasking Propaganda: A Comparative Analysis of Masked and Causal Language Models

arXiv:2610.03077v1 Announce Type: new Abstract: Propaganda detection is an essential task in natural language processing (NLP), particularly in the context of manipulative political communications. H...

By Claudiu Creanga, Ioachim Lihor, Liviu P. Dinu

llms nlp fine-tuning benchmarks

arXiv AI

Oct 5

Right Order, Wrong Scale: Auditing LLM Judges for Occupational AI Measurement

arXiv:2610.02492v1 Announce Type: new Abstract: LLM judges are increasingly used to assess whether AI outputs meet workplace requirements, but agreement on response rankings does not establish agreem...

By Harry Lyu, Neil Thompson

llms fine-tuning safety

arXiv AI

Oct 5

DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents

arXiv:2610.02320v1 Announce Type: cross Abstract: Computer-use agents need to reliably ground action targets in complex desktop scenes, where multiple applications, overlapping windows, and visually...

By A. Said Gurbuz, Ahmed Nassar, Sunghwan Hong, Marc Pollefeys, Peter W. J. Staar

llms agents fine-tuning multimodal benchmarks

arXiv AI

Oct 5

EXAM2: Extending Audio Understanding in Multilingual and Multimodal Analysis

arXiv:2608.23758v3 Announce Type: replace-cross Abstract: Recent large audio language models (LALMs) have achieved impressive progress in audio understanding. However, existing evaluations remain lar...

By Jiawen Wang, Xiaoxue Gao, Zi Haur Pang, Nancy F. Chen

llms fine-tuning multimodal benchmarks

arXiv AI

Oct 5

SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining

arXiv:2610.03265v1 Announce Type: cross Abstract: Large-scale pre-training has improved the generalization of neural operators across diverse PDEs. However, existing PDE foundation models still strug...

By Dengdi Sun, Xiaoya Zhou, Xiao Wang, Wanli Lyu, Jin Tang, Bin Luo

fine-tuning benchmarks

arXiv AI

Oct 5

Revisiting Visual Representation Enhancement of VLMs via Kernel Canonical Correlation Analysis

arXiv:2610.02718v1 Announce Type: cross Abstract: Vision-language models such as CLIP exhibit strong semantic generalization, but remain limited in fine-grained visual perception. A recent work named...

By Peilin Yang, Xiaoyu Liu, Jian Sun, Qinghua Tao

llms rag fine-tuning multimodal safety

arXiv Machine Learning

Oct 5

The Neuro-Physical Inverter: A Modular Framework for Magnetotelluric Inversion Coupling Ensemble Conditioning with Residual Learning

arXiv:2610.03225v1 Announce Type: new Abstract: We present the Neuro-Physical Inverter (NPI), a modular, uncertainty-aware framework for geophysical inversion that couples ensemble-based conditioning...

By Jae Deok Kim, Sai Ravela, Rob. L. Evans

fine-tuning

arXiv Machine Learning

Oct 5

Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning

arXiv:2605.10247v2 Announce Type: replace Abstract: Applying Large Language Models (LLMs) to graph-structured data usually involves multi-step pipelines in which textual node attributes are compresse...

By Dario Vajda

llms fine-tuning benchmarks

arXiv Computation and Language

Oct 5

Hindsight-Guided Rationale Distillation for Rare Disease Diagnosis

arXiv:2610.03176v1 Announce Type: new Abstract: We study hindsight-guided distillation for rare disease diagnosis on ZebraMap: a 1.5B student is fine-tuned on chain-of-thought traces from a 8B teache...

By Aarav Singh, Animesh Pathak, Navyansh Singh

reinforcement-learning fine-tuning efficiency safety

Get the daily briefing