LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
arXiv:2605.20682v3 Announce Type: replace Abstract: Small defects in unfamiliar product categories can be difficult to distinguish from benign appearance variations in a global view. Additional reaso...
arXiv:2607.25962v2 Announce Type: replace Abstract: Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appea...
Navalia is a framework that advances cowork agents by combining verifiable task synthesis with long‑horizon post‑training. It selects supporting evidence from corporate financial reports, builds executable computation chains, and uses independent LLM review and programmatic checks to ensure evidence consistency and answer correctness. The verified tasks are then used for trajectory collection, supervised fine‑tuning, and reinforcement learning, yielding consistent performance gains over the base model across five financial and office‑task benchmarks.
The paper investigates how large language models develop brand preferences, noting that models often name market‑leading brands like Nike when asked for a running shoe brand but recommend niche brands such as Brooks. Across 102 models and 42 brand categories, the brand chosen for recommendation differs from the named brand in 26% of cases, and preference optimization (DPO) shifts these choices by 11–24 points. The study suggests that pretraining already links recommended brands to advice, supervised fine‑tuning preserves these preferences, and DPO can shift them even when the preferred brand is absent from the preference data, indicating that the judge models used in training play a key role.
The paper introduces RSOD, a framework for creating telecom‑specific language models that blend broad networking knowledge with specialized routing, broadband, and diagnostic capabilities. It generates over 212,000 synthetic training records from networking taxonomies, trains six frozen specialist LoRA teachers, and distills their behavior into a single deployable student adapter, Llama‑CalixCLM v1. The student achieves near‑teacher performance (within 0.05 micro‑score points) while maintaining a validated, auditable training‑to‑deployment path.
Typhoon is a gradient‑based masking strategy for masked language models that estimates token importance using the MLM loss gradient and calibrates a masking distribution to match a target budget. The authors evaluate Typhoon against random and whole‑word masking on GLUE tasks MRPC and CoLA across three BERT‑family models and find no statistically significant advantage over baseline methods when multiple random seeds are considered. The study concludes that gradient‑based task‑adaptive masking is competitive with, but does not outperform, uniform random masking at the evaluated scale.
The paper introduces a channel‑level mask‑guided slimmable image compression technique that allows non‑consecutive channel selection for narrow models, improving performance at low bitrates. After training, a Mask‑to‑Slimmable (M2S) conversion turns the model into a standard slimmable network, enabling efficient inference without architectural changes. Variable‑rate control is achieved through lambda‑ and width‑conditioned modulation, quantization‑step scaling, and a freeze‑then‑fine‑tune training pipeline, yielding superior rate‑distortion performance and significant storage savings across multiple backbones.
TxnQA is a new analytical QA benchmark that tests reasoning and evidence grounding over transaction event sequences. It contains 27 task families with 81 templates and a 5‑scenario fraud‑risk reasoning suite built on synthetic, privacy‑mitigated transaction data. The benchmark evaluates models using format‑following rate, conditional content accuracy, and end‑to‑end correctness, revealing distinct bottlenecks in schema compliance, semantic calculation, and evidence attribution.
The paper introduces CopyGen, a lightweight copy-constrained decoding plugin for frozen large language models that improves exact span reproduction. CopyGen decomposes copying into three decisions—when to copy, how far to copy, and when to stop—using adaptive logit adjustments during verification to emit the longest valid prefix. Across three LLM backbones and four benchmark tasks, it boosts average exact match by 35% relative without finetuning and further improves by 4% with SFT, while also accelerating generation.
The study examines how LoRA adaptation of four Vision Foundation Models—DINOv3, SigLIP2, MerMED-FM, and BiomedCLIP—affects brain MRI tasks. Despite achieving similar segmentation accuracy (mean Dice 78‑81%), the models exhibit distinct internal dynamics, with early‑to‑mid layers being most crucial for segmentation. Restricting LoRA to these layers yields comparable performance while mitigating negative impacts on Alzheimer’s disease classification.
The paper introduces SANA‑SR, an efficient one‑step diffusion restoration framework for real‑world image super‑resolution. It uses a deep compression autoencoder to reduce latent tokens by a 32× ratio, preserving essential structures and textures, and applies a linear‑attention DiT with LoRA fine‑tuning for linear‑complexity token mixing. Experiments show competitive or superior performance on benchmark datasets, and the pruned model runs in 0.019 s with 407.95 G MACs and 344 M parameters, indicating strong potential for mobile deployment.
LightSplit introduces a lightweight orthogonal random projection at the split learning cut layer to reduce communication overhead and limit information exposure. By sending low‑dimensional projections instead of raw activations, the method preserves end‑to‑end differentiability while discarding irreversibly part of the representation, thereby mitigating reconstruction attacks. Experiments on seven benchmarks show up to 32× compression with minimal accuracy loss and strong resistance to state‑of‑the‑art inversion attacks.
arXiv:2610.27192v1 Announce Type: cross Abstract: Multimodal large language models (LLMs) are increasingly used for structured extraction from financial documents, but their behavior under visual amb...
arXiv:2610.28176v1 Announce Type: cross Abstract: Large Language Models (LLMs) often struggle with temporally grounded reasoning because their static parametric memory is misaligned with the dynamic...
arXiv:2610.27029v1 Announce Type: new Abstract: The feature-extraction quality of pre-trained models sets the ceiling of downstream transfer learning, yet few-shot model selection remains difficult:...
FRAGMENT is a two‑stage generative framework that models documents as typed relational graphs, factorizing the joint distribution into structure and content components. The first stage, the Architect, uses a causally masked Transformer to autoregressively generate graph topology and spatial relations, while the second stage, the Builder, employs a GATv2-based graph attention network to add bounding boxes, text, and visual style attributes. The framework provides explicit likelihood models for anomaly scoring and supports prompt‑conditioned, entity‑aware editing via cross‑attention.
arXiv:2610.28072v1 Announce Type: new Abstract: Learning rate schedules are typically fixed before training begins, yet when fine-tuning large language models with low-rank adapters (LoRA), the appro...
arXiv:2610.27444v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) attaches small low-rank matrices to a frozen backbone, and public repositories host many LoRAs per backbone, each for a di...
arXiv:2610.27870v1 Announce Type: cross Abstract: Hamilton-Jacobi (HJ) reachability analysis offers formal safety guarantees for control systems, but training accurate neural certificates remains dif...
arXiv:2502.06970v3 Announce Type: replace Abstract: In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models...