LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.
arXiv:2610.22438v1 Announce Type: new Abstract: Studies of what fine-tuning does to a language model often summarise the weight update $\Delta W$ by its rank. We compare the rank measures used for th...
arXiv:2610.22542v1 Announce Type: cross Abstract: While LLMs demonstrate strong capabilities in English Relation Extraction (RE) tasks, a significant gap remains in assessing their RE performance in...
arXiv:2610.22515v1 Announce Type: new Abstract: We describe our submission to Subtask~1b of ImageEval 2026. The task consists of selecting the single visually valid statement from three candidates pa...
arXiv:2610.22545v1 Announce Type: new Abstract: Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires...
arXiv:2610.23587v1 Announce Type: new Abstract: Post-training is widely assumed to reduce the social biases language models inherit from pretraining, yet direct evidence for this assumption remains m...
arXiv:2606.27210v2 Announce Type: replace Abstract: We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce...
The paper investigates the initial step of automated research ideation: generating actionable limitations from literature. By creating the ActLim-Bench dataset of 18,279 evidence‑linked supervision triples, the authors develop a three‑stage post‑training pipeline that improves recall over existing methods. Human evaluations show that the generated limitations are preferred 64.9% of the time and enhance downstream ideation systems.
The paper introduces a model‑agnostic framework that lets frozen large language models (LLMs) and vision‑language models (VLMs) learn from deployment experience without retraining. It does so by adding three external expertise components—a Skill for guiding reasoning and tool use, a Knowledge Memory for storing reliable facts, and a Multimodal Knowledge Base for visual examples—and by using a validation strategy that updates only when new cases improve performance without harming past performance. Experiments on six benchmarks with both open‑weight and closed‑source models show up to a 34.2% performance gain on medical tasks, generalization to unseen cases, and transferability to other models and non‑medical domains.
arXiv:2610.22797v1 Announce Type: cross Abstract: Reasoning rerankers write a rationale before ranking a candidate slate, and existing systems read item metadata to do so, even though generative reco...
arXiv:2608.18097v2 Announce Type: replace Abstract: We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet...
arXiv:2511.11266v5 Announce Type: replace Abstract: Vision-language models have recently emerged as promising planners for autonomous driving, where success hinges on reasoning over spatial structure...
arXiv:2307.10631v2 Announce Type: replace-cross Abstract: The practice of code reuse is crucial in software development for a faster and more efficient development lifecycle. In reality, however, cod...
arXiv:2610.22380v1 Announce Type: new Abstract: Continuous diffusion treats the forward process as a design space; discrete language diffusion often uses absorbing corruption, leaving context fully t...
arXiv:2610.23343v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) has become the standard method for parameter-efficient fine-tuning of large pretrained models, yet theoretical explanations...
arXiv:2610.23205v1 Announce Type: new Abstract: Large Language Models (LLMs) remain unreliable for extremely low-resource languages whose writing systems conflict with high-resource pretraining prior...
The paper introduces ToPA, a parameter‑efficient fine‑tuning method that replaces standard Toeplitz matrices with block‑wise Toeplitz matrices to increase capacity and shorten multiplicative chains, thereby stabilizing training and enhancing expressiveness. The authors theoretically prove that products of block‑wise Toeplitz matrices can approximate arbitrary matrices with shorter chain lengths and empirically demonstrate that ToPA outperforms existing low‑rank and sparse parameterizations across 21 NLP and CV datasets and five model architectures.
Prior-data fitted networks (PFNs) show that pretraining on large populations of synthetic tasks can enable generalization to unseen datasets with little or no downstream fine-tuning. This paradigm is...
CoDA is a framework that lets a simulation data engine and an action policy evolve together using a shared memory of environment construction, policy failures, and learning progress. The memory stores validated diagnoses from a vision‑language critic, which are turned into generation programs that recreate challenging scenarios and training programs that help the policy improve. Using this approach, CoDA boosts out‑of‑distribution success on several manipulation tasks, raising success rates from 5.3 % to 27.6 % and achieving up to 84 % success on LIBERO‑Pro under perturbations.
ZenFlow is an offloaded training framework for large language models that reduces GPU stalls by prioritizing important gradients. It updates critical gradients in-place on the GPU while offloading less important ones to the CPU, overlapping CPU work with GPU computation. The approach leverages spatio‑temporal locality of important gradients to keep identification lightweight, achieving up to 5× speedup, 2× less PCIe traffic, 85% fewer GPU stalls, and lower cloud GPU costs without sacrificing accuracy.
QuasiLeak-Bench is a new benchmark that evaluates large language model (LLM) based privacy detectors on their ability to recover quasi‑evidence groups—sets of weak quasi‑identifiers that jointly reveal private attributes—while respecting sender‑specified disclosure policies. The benchmark uses a reverse‑construction pipeline to generate documents containing verified quasi‑evidence groups and applies automated quality control to produce reliable ground truth at scale. Experiments show that current privacy detectors perform poorly on this task, but a fine‑tuned 4B LLM can outperform the best prompted models, indicating the task is learnable.