The paper introduces Update‑Matched Unlearning (UMU), a method that applies the same simulated relearning update to both a defended model and an undefended reference to better preserve plasticity while resisting relearning of deleted knowledge. Experiments on Llama and Qwen models with TOFU and FaithUn datasets show that UMU reduces harmful and benign relearning by 30.4 and 9.9 points on average and increases new‑task accuracy by 3.5 points, supporting the reference‑timing hypothesis. The study demonstrates that robust unlearning should selectively prevent recovery of deleted knowledge without suppressing the model’s ability to learn new tasks.
By Jie Li, Youming Tao, Bowei He, Meng Ding
The paper investigates whether removing a small set of attention heads in transformer-based language models can improve task performance, a phenomenon termed Capability Enhancement Localization (CEL). Empirical experiments across various models identify sparse head combinations whose ablation boosts performance without fine-tuning. The authors also provide a theoretical explanation using a simplified attention model, showing that training dynamics can concentrate conflicting task features into specific heads, whose removal selectively benefits one task while harming another.
By Anh Tuan Nguyen, Anna Bair, Zico Kolter
The paper investigates how to fine‑tune language models for sparse and continual reinforcement learning by ranking and partitioning attention heads. For sparse learning, heads are ranked by gradient norm to identify a small, effective subset for training. For continual learning, heads are partitioned across tasks to reduce interference and preserve prior knowledge. Empirical results on math, instruction‑following, and coding tasks show that tuning higher‑ranked heads outperforms lower‑ranked ones and matches denser baselines, while head partitioning improves retention without sacrificing adaptation to new tasks.
By Ruizhe Shi, Zaiwen Yang, Qiubo Wang, Banghua Zhu, Maryam Fazel
The paper investigates privacy and integrity risks in federated fine‑tuning of a pretrained MobileNetV4 model on Quick, Draw! data. It shows that a single client update can reveal private labels, that label‑count recovery degrades with more clients, and that backdoor and label‑flipping attacks can succeed under FedAvg but are mitigated by FLTrust. The study also demonstrates a live poisoning session using Indigma’s FedMaestro orchestrator.
By Vasileios Perifanis, Nikolaos Pavlidis
The paper investigates transfer learning in stochastic multi‑armed bandits when historical data come from an unknown adaptive policy. It establishes asymptotic problem‑dependent lower bounds and introduces the KL‑UCB‑Transfer algorithm, which combines online and historical data to achieve logarithmic regret with an improved constant under certain mean‑gap conditions. The study also relaxes the source‑target discrepancy assumption to a one‑sided condition, demonstrating that logarithmic regret can still be achieved under misspecification and identifying regimes of improvement, degradation, or loss of guarantee.
By Adrien Prevost, Timothee Mathieu, Waris Radji, Odalric-Ambrym Maillard
L2GSSL is a self‑supervised learning framework designed for volumetric healthcare data, particularly CT scans. It decomposes pretraining into intra‑slice and inter‑slice representation spaces, using a slice encoder for local, view‑invariant features and a global encoder to aggregate tokens for volumetric context. On the LIDC‑IDRI lung nodule malignancy classification task, L2GSSL achieves high AUROC scores with only 10–25% of labels, outperforming other self‑supervised baselines and approaching the performance of fully supervised models.
By Po-Cheng Cheng, Hung-Hsuan Chen
The paper introduces DLR-Lock, a defense that replaces each pretrained MLP in a language model with a deep low‑rank residual network (DLR‑Net). This substitution increases activation memory linearly with depth during backpropagation and creates architectural mismatches that hinder fine‑tuning. Experiments on large language models show that DLR‑Lock can withstand adaptive attackers while preserving the model’s original performance.
By Keitaro Sakamoto, Pierre Ablin, Federico Danieli, Marco Cuturi
arXiv:2511. 02406v2 Announce Type: replace-cross Abstract: We prove that there exist uniform $(+,\times,/)$-circuits of size $O(n^3)$ to compute the basis generating polynomial of regular matroids on $n$ elements.
By Christoph Hertrich, Stefan Kober, Georg Loho
The paper presents a calibrated evidence inference model fine‑tuned from Qwen3‑4B‑Base using low‑rank adapters and a classification head to classify trial outcomes as significant increase, significant decrease, or no significant difference. Trained on 1,431 open‑access PubMed Central articles, the released model (Apache 2.0) achieves an expected calibration error of 0.017, a log loss lower than Gemma 4 31B‑it in zero‑shot settings, and a macro‑F1 of 0.925 on the test split. Accepting its top‑80 % most confident predictions yields only a 2.6 % error rate, and ablation studies show that article context influences answers while the title alone still provides useful signal.
By Johann Emmanuel Li
MeddeX is a medical report analyzer that fuses a QLoRA‑fine‑tuned LLaMA‑3 (8B) model with Retrieval‑Augmented Generation and a Gemini Vision layer to produce concise, evidence‑backed summaries of medical documents. It processes text‑native PDFs, scanned PDFs, and radiological images via a hybrid extraction pipeline featuring a garbage‑text detection mechanism for corrupted or UTF‑16‑encoded files. The system adds report‑scoped retrieval, an automated longitudinal comparison engine, and a self‑healing backfill pipeline, and is backed by a FastAPI backend, Supabase vector store, and Next.js frontend. Evaluation shows significant gains over the base LLaMA‑3‑Med42‑8B model across ROUGE, BLEU, and BERTScore metrics, underscoring the value of domain‑specific fine‑tuning.
By Rajdeep Pandhare, Rohit Kundaikar, Yakub Kojar, Shreyansh Tiwari
The paper introduces Mentalized Adversarial Self-Play (MASP), a framework for training proactive dialogue agents by modeling the user as a rationally resisting participant with an evolving mental state. MASP separates teacher and student mentalization, alternates self‑play updates for system and user policies, and treats robustness to rational resistance as a training objective. Experiments on proactive dialogue benchmarks show MASP improves success rate, rationality‑constrained success, reduces average turns, and performs better against hard users compared to SFT and Planner‑only approaches.
By Zanwu Liu, Dan Zhang, Xiaodi Wang, Yongbo Mao, Xiaoyang Guo, Yonglin Wu, Peiyue Jiang, Yangyang Wei, Zecheng Lin
The paper introduces TITAN, a mechanistic framework that uses causal intervention, sparse feature decomposition, and targeted preference optimization to discover and fine‑tune specific translation circuits in multilingual decoder‑only large language models. By restricting updates to less than 5% of the parameters, TITAN achieves significant BLEU score improvements—33.75% for Hindi‑to‑Urdu and 45.86% for Hindi‑to‑Sindhi—while human evaluations confirm gains in adequacy and fluency. The study also reveals that multilingual translation operates through sparse, structured circuits over a shared semantic subspace rather than relying on an English‑centric pivot representation.
By Atai Waris Khan, Abdullah Mazhar, Raghu Kumar, Md Shad Akhtar
The paper presents a specialized NMT model for the Masewalkopa variant of Nawatl, a low‑resource language from Mexico. Using a 70,000‑pair multi‑variant foundation, the authors curate 2,665 parallel pairs and augment them with 192 elder‑transcribed sentences via asymmetric back‑translation. A custom finite‑state analyzer reduces unanalyzed forms from 82.0% to 19.9% and supplies functional syntax tags as prefixes, enabling the model to reuse subword embeddings and achieve a dev chrF++ of 58.46 and test chrF++ scores around 58.0. The final LoRA‑v9c‑b model is quantized with CTranslate2 int8, cutting CPU latency from 137.5 to 1.8–2.6 seconds per request for cost‑bounded Cloud Run deployment.
By Luis A. Mercado-Campos, Alexis Palmer
The paper investigates the Mandarin corrective construction "bushi X, ershi Y" (not X, but Y) in large language models, proposing a decision tree that classifies its use as substantive, optional, or defective based on contextual appropriateness. Across five commercial models, the construction appears 2.5 to 9.2 times more frequently than in human text, with 52.8% of annotated instances being optional—correct but unnecessary—while only 1.30% are defective. The study finds that reward models, LLM judges, and fine‑tuning techniques often increase the frequency of this construction, highlighting a gap between correctness and communicative appropriateness in current LLMs.
By Yihang Peng, Yi Jing, Dewen Liu, Weihan Meng, Ying Liu
EVA-Cap is a new framework for audiovisual video captioning that breaks captions into atomic audiovisual events, turning unstructured text supervision into event‑centric alignment. The model is trained with supervised fine‑tuning and Group Relative Policy Optimization on a self‑curated dataset, EVA‑Data, which applies event‑centric quality control. Experiments show EVA‑Cap outperforms prior methods on event‑centric metrics (e.g., +9.0% on ChronusAV) while remaining competitive on general captioning benchmarks (e.g., +3.3% on WorldSense).
By Jiapeng Shi, Qiuxia Hou, Jiaqi Leng, Junke Wang, Yanhao Zhang, Haonan Lu, Ziyi Ye, Zuxuan Wu
The paper shows that large language models can infer ethnicity from names with higher accuracy than Bayesian Improved Surname Geocoding (BISG), achieving up to 84.7% accuracy on balanced samples from Florida and North Carolina voter files. Six models, including Gemini 3 Flash, GPT‑4o, and open‑source alternatives, were tested, with extended reasoning and metadata such as party registration further improving performance. The method also reduces income bias present in BISG and is validated across diverse populations in India, Uganda, Nepal, Armenia, Chile, and Costa Rica, with fine‑tuned small transformer models surpassing BISG while allowing local deployment at no cost.
By Noah Dasanaike
WaveLLM introduces a wavelet-domain entropy model that leverages large language models for lossless image compression. By decomposing images into multi-level wavelet subbands, it transforms heterogeneous prediction tasks into structured, statistically homogeneous subproblems, enabling more effective autoregressive probability estimation. Using a dedicated numerical interface and task-adaptive prompting, WaveLLM, built on Llama‑3.2‑3B with LoRA, achieves state‑of‑the‑art bpsp results on Kodak, CLIC.pro, and DIV2K, outperforming both PixelCNN‑based codecs and recent LLM‑based compressors.
By Yue Yu, Bowei Kang, Changsheng Gao, Dong Liu, Xiaoyan Sun
Gen4P is a unified framework that transforms a pretrained video generator into a feed‑forward 4D perceiver, enabling recovery of 4D geometry and motion from monocular video without iterative denoising. It processes clean video latents in a single encoder pass and uses a lightweight cross‑attention decoder to predict requested 4D attributes, supporting flexible spatial sampling and continuous‑time queries for temporal flexibility. By incorporating a frozen pretrained video‑depth LoRA as a geometry prior, Gen4P improves performance on 3D point tracking, video depth, and camera pose estimation, achieving state‑of‑the‑art or competitive results on these tasks and demonstrating robust tracking of complex motion in in‑the‑wild videos.
MLExpert is a 35B Mixture-of-Experts agentic model designed to autonomously conduct long‑horizon machine learning engineering (MLE) tasks. It uses a post‑training approach that combines hierarchical supervised fine‑tuning with Windowed Interaction RL to learn from intermediate outcomes in complex experimentation workflows. On the MLE‑Bench Lite benchmark, MLExpert raises the average medal rate from 37.9% to 72.7%, matching the performance of larger frontier models and demonstrating transferable optimization capabilities to other benchmarks such as NatureBench Lite.
The paper distinguishes between active facts, which help a language model generalize beyond mere recall, and inert facts, which do not. By relabeling answers to a known relation (e.g., Paris → Europe becomes Paris → crab) and fine‑tuning on a subset of subjects, the authors test whether the model can transfer the new label to unseen subjects sharing the same original answer. Across eleven models and 70 relations, the linear readability of the original relation in pretrained representations strongly predicts this generalization, with a Pearson correlation of 0.87–0.92, even after controlling for various factors.