Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,205 stories · RSS feed

arXiv Computation and Language
Sep 1

Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation

The paper investigates how to choose language models (teachers) for generating multilingual synthetic data used to fine‑tune smaller student models. By evaluating 10 teacher models across six diverse languages and training 240 students, the authors find that teacher effectiveness is not driven by model size but by data qualities such as prompt diversity, length, and fluency, which explain most of the variance in student performance. Practical guidelines are offered, including matching teacher and student families and using translated prompts to improve outcomes for low‑resource languages.

By Lester James V. Miranda, Ivan Vuli\'c, Anna Korhonen
arXiv Computation and Language
Sep 1

LLP: LLM-Based Product Pricing in E-commerce

The paper introduces LLP, a Large Language Model–based generative framework for pricing second‑hand products on consumer‑to‑consumer platforms. LLP retrieves similar items to capture market dynamics, then uses LLMs to generate price suggestions, refined through supervised fine‑tuning and group relative policy optimization. A confidence‑based filter rejects unreliable predictions, and experiments show LLP outperforms prior methods, achieving higher static adoption rates when deployed on Xianyu.

By Hairu Wang, Sheng You, Qiheng Zhang, Xike Xie, Shuguang Han, Yuchen Wu, Fei Huang, Jufeng Chen
arXiv Machine Learning
Sep 1

Learning diverse attacks on large language models for robust red-teaming and safety tuning

arXiv:2405.18540v3 Announce Type: replace-cross Abstract: Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe and responsible deployment of larg...

By Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Esmeralda S. Whitammer, Moksh Jain
arXiv Computation and Language
Sep 1

PaperGym: Rubric-Centered Evolution for Research-Plan Generation

PaperGym is a framework that transforms each research paper into a training environment for AI research planning, using rubrics extracted from the paper’s method and experiments as a critic. It synthesizes research questions from the goal and background, and derives evaluation criteria from the method and experiments, reducing criterion leakage to 3.7%. Experiments with Qwen models show that training with PaperGym’s rubric improves benchmark performance by up to 5.6 points and outperforms existing datasets and fine‑tuning baselines.

By Yuhan Wang, Zhengxi Lu, Yuchen Yan, Kaitao Song, Wenqi Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
arXiv Computation and Language
Sep 1

CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions

CoVA‑SFT is a new large‑scale dataset comprising 51.9K samples and over 222K multimodal reasoning steps that teach models to interleave text and visual abstractions across five layout families and 17 complex tasks. It includes explicit rationale formulations, agentic renderings, and verification loops to help models build and maintain internal visual workspaces for purely textual reasoning problems. A companion benchmark, CoVA‑Bench, contains 1,700 held‑out test samples for reproducible evaluation, and models fine‑tuned on CoVA‑SFT outperform all interleaved CoT baselines by more than 2× on average, though they still lag behind strong text‑only CoT baselines.

By Tsung-Han Wu, Heekyung Lee, Anya Ji, Haoming Chen, Trevor Darrell, Joseph E. Gonzalez, David M. Chan
arXiv AI
Sep 1

Stratified Consistency Distillation for Natural Language Formalization

The paper introduces Stratified Consistency Distillation, a fine‑tuning method to improve natural‑language to logical‑formula translations for neurosymbolic reasoning. It generates multiple candidate translations with a large LLM, clusters them by semantic equivalence, and selects pseudo‑labels using entropy‑based strategies—majority voting for low entropy, LLM-as‑Judge for medium entropy, and unification or abstention for high entropy. The selected pseudo‑labels are then used to fine‑tune a smaller model, yielding significant gains in Pass@K and a new Equivalent Logical Similarity metric.

By Zhichao Hou, Ferhat Erata, Joe Lilien, MohamadAli Torkamani