LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
Text summarization is an important task for advancing natural language processing (NLP) research and applications in Indic languages. Despite progress in extractive and abstractive summarization, both...
AudioFM is a self‑supervised framework that learns audio representations by combining recording‑level contrastive learning with patch‑wise masked latent discrimination. The model uses visible acoustic context for a student to predict masked patch representations and matches them to full‑view teacher targets from other recordings, while also incorporating weaker supervision from temporal neighbors to promote local consistency. Evaluations show that AudioFM achieves state‑of‑the‑art results across diverse audio domains in linear probing, fine‑tuning, and zero‑shot transfer, excelling in event detection, scene‑level understanding, and domain generalizability.
The study measures the exact probability that language models assign to each integer from 1 to N when prompted for a random number, eliminating sampling noise by summing over all spellings. Across 18 open‑weight models and 29 decoding settings, instruction tuning consistently makes the distribution less uniform, introducing a round‑number avoidance heuristic and sharpening endpoint avoidance. While tuned models rarely output the extreme values 1 or N and largely avoid multiples of 10, they do not converge on a single favorite integer across models.
The paper introduces a structured text representation using SVG to encode map geometry for landmark-based route instruction generation. It shows that large language models can use this representation to select landmarks at turns and near destinations, performing as well or better than image-based inputs. Fine‑tuning smaller local LLMs further improves accuracy, demonstrating the efficiency of structured text for spatial reasoning in navigation.
The paper presents a Japanese discourse parsing system built on Segmented Discourse Representation Theory (SDRT) by translating existing English SDRT corpora with a large language model (LLM). Comparative experiments with prompt-based and fine-tuned LLMs show that Japanese and English parsers achieve similar performance, indicating cross‑lingual applicability of SDRT. Ablation studies reveal that Japanese discourse markers have little influence, while predictions rely mainly on contextual discourse relations rather than clause content, explaining the comparable results across typologically distinct languages.
CAMR (Compatibility-Aware Memory Replay) is an adaptive framework for continual learning in large language models that jointly manages supervision of new examples and replay of historical data. It uses compatibility checks and a learned memory gate to decide among three actions: general replay of incoming examples, reinforcing compatible related examples, and skipping supervision for mastered duplicates. Evaluated on Qwen3-8B-Base, CAMR outperforms existing methods on WikiBigEdit factual updates and achieves top scores on multiple TRACE tasks while preserving earlier-task performance across diverse domains.
The paper introduces Amplify, Locate, Estimate (ALE), a technique for efficiently estimating rare failure probabilities in safety‑fine‑tuned large language models. ALE amplifies failure rates by exploiting changes in token log‑probabilities, then uses a score‑function estimator and importance sampling to extrapolate back to the original rare rates. Experiments on 41 targets show ALE’s estimates closely match reference values, achieving accurate predictions for failure rates as low as 10⁻⁵ with far fewer labeled rollouts than direct sampling.
The paper introduces REGLAT, an ensemble-based model that integrates named entity recognition (GLiNER) with a fine‑tuned MARBERT classifier to detect harmful content in Arabic. It classifies user prompts into seven harm categories, achieving a macro‑F1 of 0.886 and accuracy of 0.897 on the ArGuard 2026 development set, with slightly lower scores on blind test sets. The study highlights that combining explicit entity extraction with contextual language modeling improves fine‑grained harmful content detection, though distinguishing closely related categories remains difficult.
The paper introduces Watermark Basin Alignment (WM‑Basin), a black‑box method for detecting distilled large language models (LLMs). It exploits a persistent watermark basin—a key‑specific geometric signal embedded in the teacher’s parameter space during fine‑tuning—that remains detectable even when token‑level signals vanish. WM‑Basin maps suspect model outputs to gradient representations, removes ordinary fine‑tuning directions, and tests alignment with the watermark basin, achieving significantly higher detection rates than token‑level baselines across a range of teacher and student models.
The paper describes a winning entry (MAI) in the MetaFine Open Competition, where a separate policy π_{0.5} was fine‑tuned for each of five tasks using a task‑specific training mixture and a set number of actions per prediction. Training began from official demonstrations and expert procedures, and local evaluation revealed frequent grasping, pushing, and insertion failures. These issues were mitigated by refining the demonstrated motions and adding expert continuations from states reached by the policy, while retaining the original demonstrations in training; checkpoints were evaluated on every instruction to ensure balanced performance, resulting in a first‑place score of 96.66.
The paper explores whether images with embedded watermarks can still be edited without damaging the watermark. It introduces SafeMark, a framework that adds a watermark‑decoding loss to the training objective of diffusion editors, ensuring that semantic edits preserve watermark integrity. Experiments across datasets and editing methods show that SafeMark maintains high watermark bit accuracy while producing quality edits, proving that watermark preservation and semantic editability can coexist.
The paper introduces INTRA, a hardware‑aware static sparse attention mechanism that interleaves non‑contiguous token groups to enable efficient long‑range communication while reducing quadratic cost. INTRA uses Scatter and Gather layers to create short mediated paths across layers, and a FlashAttention‑style kernel computes structured indices at runtime for dense packing of selected tokens. Experiments on an A100 GPU show significant speedups—up to 8.4× over MoBA and 10.4× over density‑constrained NSA—while maintaining or improving image quality and long‑context performance on models like FLUX.1‑dev and LLaMA‑3.1‑8B.
The paper discusses how software engineering (SE) expertise can guide the training of large language models (LLMs) for coding tasks. Using a case study of a 92‑billion‑parameter LLM, the authors created RefactorForge, a pipeline that leverages SE knowledge and tools to generate verified training data from 306 Python repositories. Fine‑tuning the model on this data improved performance on several benchmarks, including RefactorBench and SWE‑Bench ProMax, and the gains extended to other coding‑agent harnesses.
The paper introduces LBS (Look Beyond Screens), a vision‑language model that learns transferable visual observation behaviors from GUI grounding. LBS is first supervised with teacher‑generated GUI trajectories and then fine‑tuned via reinforcement learning to make multi‑turn crop‑and‑inspect decisions. Experiments show that LBS models based on Qwen3.5‑35B‑A3B and Qwen3.8‑27B outperform their baselines on PerceptionBench, HR‑Bench, and ChartQAPro, indicating that the learned observation strategies generalize beyond GUI‑specific imitation.
The paper introduces a hierarchical low‑rank reparameterization technique that reorganizes the trainable parameters of parameter‑efficient fine‑tuning (PEFT) methods into an approximately square matrix and applies recursive low‑rank decomposition. This multi‑level compression drastically reduces the number of trainable parameters while preserving strong task performance. Experiments on LLaMA‑3.1‑8B and Qwen2.5‑7B show that, when combined with LoRA or DoRA, the method outperforms baseline PEFT approaches and remains competitive even after compressing the original parameter count by an order of magnitude.
MVRCap is a new video captioning framework that treats caption generation as a reinforcement learning problem guided by structured verification. It introduces a Fine‑Grained Verifiable Reward system built on three pillars—Hierarchical Concept Coverage, Factuality Verification, and Conciseness Constraint—supported by the newly created MVRCap‑QA and MVRCap‑SFT datasets. Experiments show that MVRCap outperforms existing methods on benchmarks such as VDC, CAPability, and VC4VG.
TextCloak is a reinforcement‑learning driven framework that generates unlearnable examples (UEs) to protect textual data from unauthorized fine‑tuning of large language models (LLMs). It transforms clean text into perturbations that preserve meaning and naturalness while degrading the utility of models trained on them. Experiments on six datasets and nine LLMs show that TextCloak consistently impairs unauthorized fine‑tuning and remains robust across different architectures, training settings, and adaptive attacks.
The paper introduces a system that enables first‑order LoRA training on phone NPUs by running all frozen‑weight multiplications on the NPU while keeping adapters and the optimizer on the CPU. It reuses weight tiles for input‑gradient computation, selects precision based on gradient sensitivity, and fuses remaining CPU operations to overlap with NPU execution. The approach is evaluated on Qualcomm Snapdragon devices with Qwen and Llama LLMs, demonstrating background training for nightly personalization and benchmark workloads.
The paper investigates how to adapt recurrent neural networks for surface electromyography (sEMG) gesture recognition across different users and new gesture classes using continual learning (CL). Three CL strategies—experience replay, elastic weight consolidation (EWC), and naive fine‑tuning—were evaluated on the NinaPro DB1 dataset under domain‑ and class‑incremental drift, with experience replay achieving the highest final accuracy (83.3%) and the lowest forgetting (1.8–5.8%). The study also shows that training on shuffled, independently‑windowed segments and incorporating hand‑crafted time‑domain features can significantly improve per‑task accuracy, but only experience replay maintains this advantage under class‑incremental drift.
The study presents an open‑source multimodal model that classifies bacterial versus fungal microbial keratitis using 17,158 slit‑lamp images from 1,645 patients in the US and India. The model, which integrates cross‑modality contrastive learning, modality‑specific fine‑tuning, and seven clinical variables, achieved a macro‑F1 of 83.06% and an AUROC of 89.9% overall, outperforming five comparator methods. However, performance varied markedly by country—AUROC 83.9% in India versus 67.0% in the US—highlighting geographic confounding and the need for independent geographic validation.