LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
Studying how fine-tuning shapes refusal and noncompliance behaviour requires identifying training examples that refuse, evade or otherwise fail to fulfil the requested task. But existing annotation co...
Users of language-based assistive agents often make ambiguous requests. In response, an assistant can either directly act on its interpretation of the request --- risking misalignment with the user --...
Structured images, such as diagrams, charts, and flowcharts, are inherently symbolic and can be compactly represented in an editable format, yet in practice, they are often rendered as images, and the...
High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to answer a question. Sequential zooming can recover de...
Simulation-based inference is challenging when many heterogeneous observations must be composed, hierarchical latent structure must be preserved, and the simulator is misspecified relative to observed...
Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface: a small amount of harmful data mixed into an othe...
Diffusion and flow policies can model complex behaviors in offline reinforcement learning (RL). However, penalizing their KL divergence from the behavior policy can discourage actions having high crit...
Chinese Semantic Error Correction (CSEC) targets semantic errors in Chinese text, which are typically more subtle and complex than spelling and grammatical errors but remain relatively underexplored....
The paper presents MoCoP v2, an enhanced contrastive pretraining method that aligns small molecule embeddings with deep‑learning‑derived cell morphology profiles. By replacing CellProfiler fingerprints with richer image‑encoded features, the new embeddings better capture how molecules alter cell morphology, leading to improved QSAR, toxicity, ADME, and activity predictions. Performance scales log‑linearly with training data size, indicating further gains with larger datasets.
The paper presents a self‑supervised approach to learning representations of auroral emission spectra using a 1D Vision Transformer trained with a masked autoencoder on 223,000 unlabelled spectra. The pretrained model recovers key emission‑line intensity ratios with high accuracy (R² = 0.91) and, with a single linear probe, matches expert‑designed feature classifiers. When fine‑tuned, it surpasses the previous supervised auroral classifier (macro‑AP 88.5 vs. 77.8) and achieves a 0.870 mAP, outperforming a model trained from scratch by 0.159 using only 10 % of the labels, while attribution reveals reliance on N₂⁺ bands.
CacheReforge is a method for recovering stale key‑value (KV) caches in large language models when lightweight adapters evolve. It represents stale caches as layer‑wise mixed‑version objects and uses adapter anchors, sensitivity calibration, drift accumulation, and restart boundaries to decide between direct reuse, bounded recomputation, or full suffix recovery. Experiments on Qwen2.5 models with continual LoRA updates show a 92.4% reduction in mean KL divergence while only recomputing 5.44% of layers and cutting cache‑maintenance time by 93.2% compared to full prefill.
The paper investigates why per-layer pruning of IoT intrusion detectors can cause severe class-level failures. On the CICIoT2023 dataset, a two-layer convolutional detector pruned at 80% sparsity loses 16 accuracy points and half its macro‑F1, with 17 of 34 classes heavily damaged. The failure is traced to the first layer’s weight starvation, and the authors show that protecting those 192 weights or pruning globally, as well as recomputing normalization statistics, can prevent or repair the collapse.
The doctoral research investigates Audio Emotion Recognition (AER) for individuals with atypical hearing, specifically focusing on auditory hypersensitivity in people with autism. It proposes leveraging affective insights from acoustic traits and aims to generalize emotional responses from limited annotated data. The study begins by fine‑tuning the Contrastive Language‑Audio Pretraining (CLAP) foundation model using low‑rank adaptation (LoRA) on a valence and arousal dataset collected from neurotypical listeners.
BeatGraph is a self‑supervised model that represents infant ECG recordings as graphs of individual heartbeats rather than fixed‑length patches, allowing it to capture the higher heart rates and distinct waveform patterns of infants. The model uses a shared beat encoder, a Transformer for temporal ordering, and graph attention layers to produce a window embedding, which is pretrained on a large unlabeled infant ECG corpus and fine‑tuned for tasks such as sleep‑wake detection, infant‑state classification, activity‑source identification, and affect recognition. BeatGraph achieves state‑of‑the‑art performance on multiple infant‑specific benchmarks and transfers well to pediatric and adult ECG datasets, while also releasing the first public infant ECG corpus collected in diverse home and classroom settings.
The paper introduces TRACE, a fine‑tuning framework for Retrieval‑Augmented Generation (RAG) that addresses conflicts between retrieved knowledge and a model’s internal knowledge. TRACE uses multi‑agent debate traces to identify correct and incorrect candidates and answer‑shift patterns, providing fine‑grained supervision for reliable knowledge‑source selection. It also incorporates an answer‑completeness regularization mechanism to prevent empty, overly short, or prematurely terminated responses, thereby improving robustness against misleading retrieved content and enhancing answer quality.
The paper introduces a two‑stage training framework for neural network observers that guarantees Lyapunov stability for uncertain dynamical systems. First, a point‑guided Lyapunov pre‑training phase quickly achieves high estimation accuracy and local stability over sampled states. Second, an LMI fine‑tuning phase efficiently enforces a strict global Lyapunov stability certificate, yielding provably stable observers that train faster than direct LMI methods and generalize robustly across diverse systems.
arXiv:2605. 30358v2 Announce Type: replace Abstract: Quantum computing remains in the Noisy Intermediate-Scale Quantum (NISQ) era, with performance constrained by noise.
The paper introduces READ, a method for composing low‑rank adapters (LoRA) in large language models. By rewriting each adapter into a balanced canonical form and enforcing a one‑directional coupling, READ allows new skills to read but never write into the output subspaces of existing skills, eliminating interference. Experiments on four benchmark suites and two model families show that READ consistently outperforms existing baselines, improving SuperGLUE scores by over twenty points and domain suite scores by more than seven points.
The paper introduces DUALLM, a dual-method pipeline that uses a Large Language Model and a fine‑tuned small language model to classify Linux kernel security patches with high precision. By analyzing commit titles, messages, diffs, and code context, DUALLM achieves 87.4% accuracy and an F1‑score of 0.875, outperforming existing methods. It successfully identified 111 recent patches addressing out‑of‑bounds or use‑after‑free vulnerabilities, with 90 confirmed true positives and proof‑of‑concept exploits demonstrating the validity of the classifications.
Historical street-view imagery records urban evolution, but uneven coverage leaves substantial gaps in historical records. Generating plausible past appearances requires restoring changed structures w...