NormTune introduces a norm‑guided approach to parameter‑efficient fine‑tuning (PEFT) for reasoning‑intensive tasks, addressing a performance gap caused by under‑adaptation in weight‑update norms. By adaptively strengthening layers with insufficient updates, NormTune consistently improves various PEFT methods across challenging benchmarks. Experiments show significant accuracy gains—up to 24.24% with rank‑8 LoRA on S1K‑1.1—while rank‑256 variants match full fine‑tuning performance.
By Zhangming Li, Qinghao Hu, Weifan Guan, Yifan Zhang, Peisong Wang, Jian Cheng
The paper introduces FlowGDA, a flow‑matching framework for gradual domain adaptation that models adaptation as continuous transport on a statistical manifold. FlowGDA learns time‑dependent velocity fields to generate smooth domain trajectories, avoiding adversarial Min–Max optimization by using a simulation‑free regression objective. The method incorporates source‑anchored transport, semantic consistency loss, and a two‑stage optimization scheme to prevent semantic drift, achieving superior performance over state‑of‑the‑art techniques.
By Yubo Huang, Yushe Cao, Zixi Wang, Jingzehua Xu, Xin Lai, Xiongyu Hu, Shuai Zhang
The paper introduces a unified training framework for automatic PowerPoint generation, combining cross‑validated data synthesis, rule‑based teacher enhancement, and a mixed‑teacher supervised fine‑tuning strategy. It further adds a reinforcement learning component with verifiable aesthetic metrics to refine slide visual attributes. Experiments show significant performance gains on PresentBench, outperforming existing models and improving aesthetic quality.
By Yuhang Xie, Cheng Li, Chenliang Li, Mingyu Xu, Mingwei Sun, Suhang Zheng, Ming Yan, Xiaoxiao Xu, Lin Qu
The paper introduces a post‑hoc Feature Selection Layer (FSL) that can be added to frozen pre‑trained neural networks on tabular data to highlight the most important features. This lightweight, trainable module fine‑tunes feature weights based on the original model’s outputs without altering its architecture or parameters. Experiments on synthetic and real‑world datasets show that the post‑hoc FSL preserves predictive performance while improving interpretability, outperforming other post‑hoc methods such as Integrated Gradients and SHAP.
By Jo\~ao Kenji Suwa, Jo\~ao Pedro Silveira e Silva, Bruno Iochins Grisci
The paper investigates how performance predictors can improve model routing for large language models (LLMs). It shows that a predictor only outperforms a model’s self‑confidence when the candidate model is suboptimal or the predictor adds significant capacity. The authors provide theoretical bounds, extend the analysis to two‑model cascades, and confirm their findings experimentally across 16 LLMs on four benchmarks, noting that the advantage diminishes as models are fine‑tuned.
By Guanhua Zhang, Moritz Hardt
The paper evaluates cost‑aware abstention for routing IT support tickets, comparing four non‑LLM classifiers and four large language models on a 47,837‑ticket, eight‑class dataset. Using a cost model (c_ok=0, c_triage=1, c_wrong=5), a TF‑IDF logistic regression model with a 0.60 threshold reduced test cost from 0.763 to 0.472 and lowered wrong routes from 15.3% to 3.5%. The study also shows that calibration‑selected policies outperform always‑route and always‑defer strategies in most settings and introduces new metrics such as correctness AUROC and tie‑aware risk‑coverage area.
By Sachin Gupta
FIGFLEX is a mixed‑precision accelerator co‑designed with a low‑bit LLM quantizer that performs floating‑point‑integer (FP‑INT) GEMM. It introduces a shift‑add, radix‑4 reformulation of FP‑INT multiplication that fuses multiply‑accumulate into a precision‑flexible partial‑product reduction, and a novel INT3 optimization that reclaims idle datapath leaves. The accompanying mixed‑precision quantizer assigns channel‑group bit‑widths based on GPTQ error, using power‑of‑two column scaling to keep activations out of the FP16 subnormal range, enabling the quantized model to run on FIGFLEX with the same accuracy as on a GPU while achieving higher throughput per area on 3‑bit mixed‑precision workloads.
By Faraz Tahmasebi, Haocheng Xu, Andrew Trauger, Yomal Perera, Zhenyu Tang, Nael Chowdhury, Sitao Huang
ReDence is an open‑source hybrid framework that combines a multimodal agentic Investigator model with a specialized Decider model to detect modern social engineering scams. It automates investigative planning, evidence interpretation, and decision‑making across multiple scam categories, achieving 96% accuracy and operating at least four times faster than comparable agentic models. The system outputs structured, evidence‑grounded reports that enable analysts to audit decisions and support mitigation efforts.
By Jaishnoor Kaur, Josef Hor\'a\v{c}ek, Ian Waskom, Sayak Saha Roy
The paper investigates how open‑book scoring evaluates document‑level LoRA adapters by training one adapter per Wikipedia article on Llama‑3.1‑8B‑Instruct. Results show that open‑book scores largely reflect a shared reading behaviour: a random adapter can recover most of a specialist’s advantage when the document is provided, but only a negligible fraction when the document is omitted in a cloze probe. While specialists improve cloze recall and their weight updates align with the documents they read, selecting adapters by weight‑space proximity or text embedding does not outperform random choice, and merged adapters recover most of the open‑book advantage but only a small portion of the cloze advantage. The study also documents four silent errors that yielded clean but incorrect results.
By Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp
MarkTune is a model‑embedded watermarking framework that improves the trade‑off between generation quality and detectability. By regularizing toward a high‑quality reference distribution, it decouples quality preservation from proximity to the base model and uses the GaussMark detection statistic as a reward during on‑policy fine‑tuning. Empirical results show that MarkTune enhances the quality‑detectability frontier, maintains downstream task performance, and remains robust against text‑editing and open‑weight attacks across diverse datasets.
By Yizhou Zhao, Zhiwei Steven Wu, Adam Block
NADS (Navigator‑Guided Data Selection) is a fine‑tuning framework for large language models that mitigates catastrophic forgetting when the original pretraining data are unavailable. It first trains a navigator model without preservation regularization to detect capability drift, then scores synthetic data candidates using the predictive divergence between the navigator and the pretrained model. Finally, it selects a diverse, utility‑aware subset of synthetic data and fine‑tunes while distilling pretrained behavior, achieving a better balance between new‑task performance and general capability preservation with reduced selection overhead.
By Jianhao Zhang, Ou Wu, Yi Du
The paper investigates how to add new skills to large multimodal models (LMMs) without erasing previously learned abilities. By sequentially fine‑tuning on five target skills and tracking performance on eight held‑out benchmarks across three model families, the authors discover that forgetting on one skill can partially recover when the model is later tuned on a different skill. They attribute this to shifts in the output token distribution and propose two simple, selective tuning recipes—updating only self‑attention projection layers or only the MLP Gate&Up while freezing the Down projection—that achieve a better learning‑forgiveness trade‑off than full‑LLM tuning or other common forgetting‑mitigation methods.
whyItMatters":"The study shows that carefully choosing which model components to fine‑tune can enable LMMs to acquire new skills while preserving prior knowledge, offering a simpler and more effective alternative to existing forgetting‑mitigation techniques."
By Zhen Zhu, Yiming Gong, Yao Xiao, Yaoyao Liu, Derek Hoiem
GFlowPO introduces a probabilistic framework for optimizing language model prompts by treating prompt search as posterior inference over latent prompts, regularized by a meta-prompted reference-LM prior. The method first fine‑tunes a lightweight prompt‑LM using an off‑policy Generative Flow Network objective that reuses past prompt evaluations for sample‑efficient exploration. It then applies a training‑free Dynamic Memory Update to refine the meta‑prompt with diverse and top‑performing prompts, leading to consistent improvements over recent discrete prompt optimization baselines on few‑shot classification, instruction induction, and question‑answering tasks.
By Junmo Cho, Suhan Kim, Sangjune An, Minsu Kim, Dong Bok Lee, Heejun Lee, Sung Ju Hwang, Hae Beom Lee
The study investigates whether post‑training makes language models’ writing more predictable and whether this effect carries over when the same models act as judges. Using two 7‑billion‑parameter families (OLMo‑2 and Zephyr), the authors track changes from base models through supervised fine‑tuning (SFT) and direct preference optimization (DPO). They find that as writers, models produce increasingly familiar stories, but as judges, the models’ preference for predictable writing barely shifts, while they become more sensitive to story length, display order, and lose consistency on creative‑writing questions.
By Arman Nik Khah, Arvin Bahreini
SIM-UT is a multi‑turn, user‑simulation evaluation framework designed to uncover realistic failure modes in large‑language‑model (LLM) medical triage chatbots. The study applied SIM‑UT to three WhatsApp‑based health chatbots in India, South Africa, and Kenya, comparing a medically fine‑tuned model (MedGemma) with general‑purpose models (ChatGPT and Claude) across more than 5,000 multilingual simulated conversations. A prominent failure identified was de‑escalation risk, where chatbots incorrectly down‑rated serious symptoms after users minimized them, potentially leading to dangerous clinical outcomes.
By Deepali Mittal, Dheeraj Mundhra, Yulia Sidorova, Tersia de Kock, Oli Munns, Sofie S Meyer
The paper introduces a new benchmark of 13,044 spontaneous Nepali‑English code‑switched podcast utterances, totaling 15.45 h, and evaluates a two‑stage synthetic‑to‑real adaptation framework for ASR. Stage 1 uses supervised fine‑tuning on 0.95 h of synthetic speech, reducing Mixed Error Rate from 23.40 % to 21.52 % and switch‑point error rate from 35.89 % to 28.77 %. Stage 2 applies Group Relative Policy Optimization with script and transcription rewards, yielding modest but statistically significant further improvements, particularly in English script confusion and switch‑point accuracy. The study also shows that an auxiliary script reward term offers no benefit beyond MER‑only optimization in this setting, and that the synthetic SFT provides the main cross‑domain transfer to spontaneous human speech.
By Firoj Paudel
IndoNutriQA is a newly released Indonesian clinical nutrition question‑answer benchmark comprising 911 expert‑curated question‑answer pairs drawn from real clinical‑nutrition reports. Each pair is grounded in a clinically relevant context and enriched with structured metadata such as anthropometric assessment, medical diagnosis, and document provenance. The dataset supports generative, extractive, and retrieval‑augmented QA, and includes fine‑tuned baselines and zero‑shot evaluations of contemporary large language models.
By Tirana Noor Fatyanosa, Ummu Ditya Erliana, Mahendra Data, Muhammad Nurwegiono, Eleazar Tadeo Eman, Alham Fikri Aji
CrisiLangs is a multilingual extension of the CrisiText dataset, adding Italian, Bulgarian, German, and Greek translations of warning messages for 1,000 crisis events, resulting in 5,000 event‑message pairs across five languages. The resource was used to fine‑tune a multilingual model, which outperformed a zero‑shot baseline on ROUGE‑L, BLEU, and BERTScore in every language and configuration. Human preference studies showed that low decoding temperatures were favored across all languages, while the impact of emergency guidelines varied by language.
By Giacomo Gonella, Stefano Menini, Marco Guerini
The paper introduces the Bashkir Web Corpus, a 71,567‑document collection (46.9 M tokens) and three labeled datasets for text classification, and benchmarks eight large language models (from 82 M to 8 B parameters) on binary, multiclass, and multilabel tasks. It evaluates three adaptation strategies—full fine‑tuning, QLoRA, and in‑context learning—alongside continual pretraining, finding that QLoRA on 3.8–8 B models outperforms encoder‑only fine‑tuning for binary classification, while continual pretraining’s impact varies by model and task. The study also reports tokeniser fertility differences, shows that Cyrillic tokenisers pretrained on Russian perform best, and demonstrates that LLM‑generated ratings correlate with native judgments for grammar and coherence but not for language correctness, underscoring the need for native evaluation.
By Mullosharaf K. Arabov, Svetlana S. Khaybullina
SCOPE compiles natural‑language policies into neurosymbolic predicate programs that combine small neural models, deterministic extractors, and logical composition. The system is jointly optimized with adversarially generated counterexamples, achieving higher accuracy than prompt‑optimization, task‑decomposition, and workflow‑generation baselines on real‑world policy benchmarks. Programs trained with the weakest executor transfer to other models, enabling deployment on inexpensive models with stable, interpretable predictions.
By Arnon Mazza, Elad Levi