Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

3,511 stories · RSS feed

arXiv AI
Jul 14

The Nuts and Bolts of Natural Language to SQL Translation: A Systematic Analysis of Model Pipeline Optimisation Approaches and their Interactions

arXiv:2607. 10911v1 Announce Type: cross Abstract: In the age of large language models, Natural Language to SQL (NL2SQL) translation remains an open problem with many useful applications.

By Filip Klubicka, Vasudevan Nedumpozhimana, Sneha Rautmare, Bora Caglayan, Mingxue Wang, John D. Kelleher
arXiv AI
Jul 14

Stable On-Policy Distillation through Adaptive Target Reformulation

arXiv:2601. 07155v3 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is a widely adopted technique for transferring knowledge from large language models to smaller student models; however, conventional supervised KD often suffers from a distribution mismatch between training and inference.

By Ijun Jang, Jewon Yeom, Juan Yeo, Hyunggyu Lim, Taesup Kim
arXiv AI
Jul 14

DiffEEG: A Self-Supervised Denoising Diffusion Model for Learning EEG Generic Representations

arXiv:2607. 11578v1 Announce Type: cross Abstract: Deep learning for EEG-based seizure detection faces critical challenges: severe annotation scarcity and extreme class imbalance, where ictal events comprise less than 10\% of clinical recordings.

By Abdulkader Helwan, Lina Abou-Abbas, Hussein El Amouri, Belkacem Chikhaoui, Khadidja Henni
arXiv Machine Learning
Jul 14

Empowering Long-form Omni-modal Understanding with Robust Audio Perception

arXiv:2607. 10299v1 Announce Type: new Abstract: Recent advances in large-scale multimodal models have drivenremarkable progress in vision-language tasks; however, comprehensiveomni-modal understanding remains under-explored, largely due to thescarcity of datasets with rich, explicitly aligned auditory cues.

By Kaiying Yan, Luoyi Sun, Xiao Zhou, Weidi Xie