The paper proposes a memory‑efficient training method for transformers by training a local proxy network for each block to predict the global output via distillation. This allows gradients to be propagated locally while closely approximating the true backpropagation gradients, achieving a better accuracy‑memory trade‑off on CIFAR‑100 vision transformers. The approach also improves fine‑tuning of large language models under tight memory budgets, outperforming other memory‑efficient zeroth‑order algorithms.
By Wu S. Zihan, Brian A. Yuan, Guillaume Bellec
The paper introduces the concept of behavioral couplings, where fine‑tuning a large language model on one behavior can unintentionally trigger another related behavior. By training on documents that pair distinct behaviors, the authors show that fine‑tuning to elicit one behavior causes the other to emerge, and that couplings can compose across multiple document sets. These findings highlight that unintended behaviors can surface after fine‑tuning and may be difficult to trace back to their origins.
By Dongyoon Hahm, Woogyeol Jin, Dennis Wei, Yi Zhou, Nathalie Baracaldo, Kimin Lee
EgoP$^{2}$C is a new framework that internalizes a plan‑perceive‑compose workflow within a single vision‑language model for spatial reasoning in egocentric video. It trains specialized Planner and Perception teachers via Dual‑LoRA adapters, then fine‑tunes a mixture‑of‑experts student under their supervision, using a segment‑level multi‑teacher on‑policy distillation to guide evidence‑seeking, stopping, and answer composition. The resulting student models, built on Qwen3‑VL‑8B and SenseNova‑SI‑1.3‑8B, outperform their base models by 9.2 and 7.5 percentage points on three benchmarks, while cutting inference FLOPs by about 66% and speeding up inference 1.29–1.49×.
By Chaoyu Li, Arka Sadhu, Yuliang Li, Hyo Jin Kim, Yuning Chai, Pooyan Fazli
Q-Reflector is a standalone GUI action reflector built on Qwen3‑VL‑8B that learns to generate pre‑ and post‑conditions for each executed action and then evaluates execution success and task validity. It is trained with supervised fine‑tuning, teacher‑student distillation, and reward‑based refinement, and achieves 82.16% accuracy on 1,082 AndroidWorld transitions, outperforming baseline models. In MobileWorld tasks, it improves task success rates by 17.09 and 14.53 percentage points over no external reflection, and surpasses GPT‑5.2 and Claude Haiku 4.5 by 3.42 and 5.98 percentage points respectively.
By Linqiang Guo (Peter), Wei Liu (Peter), Li Gu (Peter), Yang Wang (Peter), Tse-Hsun (Peter), Chen
The paper introduces LARS (Low-memory Activation-Rank Subspace), a parameter-efficient fine-tuning method that reduces the memory footprint of adapter activations by operating in a sequence-level low-rank subspace. By replacing token-level activations with pooled feature extraction, low-rank modulations, and residual projection, LARS cuts the stored activation intermediates from O(BSRL) to O(BRL). Experiments show that LARS achieves comparable performance to LoRA while lowering GPU memory usage by 23%, CPU memory usage by 47.8%, and reducing CPU power and energy consumption per optimization step by about 22%.
By Irene Tenison, Stella Ahn, Miriam Kim, Ebtisam Alshehri, Lalana Kagal
arXiv:2610. 22621v1 Announce Type: new Abstract: The training loss of binary and ternary neural networks, expanded as a multilinear polynomial over the discrete weights, is structurally equivalent to the energy of a hierarchical high-order Ising model.
By Zihao Chen, Zhili Xiao, Rui-Jie Zhu, Yipeng Wang, Jason Eshraghian, Shantanu Chakrabartty
arXiv:2610. 22794v1 Announce Type: new Abstract: A prominent family of low-rank LLM optimizers, pioneered by GaLore, performs gradient-subspace projection: at each step, they project the gradient into a low-dimensional subspace and map the update back to the full model.
By Xinyu Luo, Jincheng Zhou, Mengbo Wang, Zilin Shen, Nadia Atallah Lanman, Petros Drineas, Bruno Ribeiro, Brian Bullins
The paper introduces Lookahead Editing (LED), a method that enhances masked diffusion language models (MDMs) by adding a rewritable proposal state. LED performs multiple lookahead updates at each decoding step to refine future predictions before committing tokens, thereby improving expressivity while retaining MDMs’ simple transition schedule. Fine‑tuned LED models (LED‑7B and LED‑8B) achieve significant performance gains on eight benchmarks and outperform larger models such as LLaDA2.1‑16B using only supervised fine‑tuning.
By Yuyan Zhou, Zeyuan Yang, Yuxuan Song, Zhicheng Cai, Hongli Yu, Weiying Ma, Hao Zhou, James Kwok
MetaCurate is a training‑free framework that automatically evolves curation filters for agentic coding data by using pairwise large language model judgments to refine its rules, rubric, and aggregation policy. Starting from no initial filter, it discovers a strategy that scores trajectories independently, requiring pairwise comparisons only during the search phase. Experiments on an uncurated trajectory pool and the Nemotron v3 SWE dataset show that MetaCurate achieves the highest mean SWE‑Verified score compared to random selection, hand‑authored filtering, and a training‑loop baseline, while incurring lower search costs.
By Donghoon Kim, Seunghyuk Oh, Kevin Galim, Minseo Kim, Minjae Lee, Younghyun Kim, Wooseok Seo, Hyung Il Koo, Wonjun Kang
Physics-Transfer Learning (PT) is a physics-constrained learning theory that enables zero-shot generalization across domains of increasing complexity. PT builds chains of anchor domains (CADs) linking low-cost source domains to complex targets and introduces computable feasibility and reliability indicators to assess transfer before target labels are available. The authors derive an upper bound on target risk under physical-consistency and stable-representation assumptions, and demonstrate PT’s effectiveness on quantum excited-state energy prediction and classical dynamical-transition prediction, achieving accurate high-fidelity predictions across substantial distribution shifts.
By Yingjie Zhao, Zhiping Xu
The paper introduces CoLU, a confidence‑weighted layerwise unlearning method for large language models that updates only a single transformer layer instead of the entire model. It also proposes Brief Layer Screening (BLS) to quickly rank layers using a normalized forgetting‑retention score, reducing CoLU’s runtime by 60–80% while maintaining performance. Experiments on TOFU, MUSE Books, and News datasets show CoLU outperforms state‑of‑the‑art methods in both forgetting and retention, and uses significantly less GPU memory.
By Jingkai Guo, Ziyi Zhao, Chaitali Chakrabarti, Deliang Fan
The paper proposes a depth‑aware FrankenMerging technique that blends lower layers of an aligned large language model (LLM) with higher layers of its base counterpart to balance diversity and quality without extra training. Experiments demonstrate that adjusting the layer split controls both lexical and semantic diversity, often surpassing the aligned model in semantic diversity and outperforming temperature sampling in most settings. The method is validated by an ablation that reverses the merge order, supporting the hypothesis about layer‑specific diversity contributions.
By Yasuaki Sumita, Tomoharu Iwata, Toshiyuki Tanaka
The paper introduces Cast‑R1, an 8‑billion‑parameter agentic forecasting model that learns to orchestrate analytical and prediction tools through trajectory distillation and multi‑turn reinforcement learning with structured memory. The training pipeline first generates multi‑turn interaction trajectories using a teacher LLM, then fine‑tunes the agent with supervised learning and group relative policy optimization, guided by rewards for accuracy and temporal structure. Experiments across several datasets show that Cast‑R1 effectively equips smaller language models with tool‑augmented, multi‑turn forecasting capabilities within structured workflows.
By Xiaoyu Tao, Mingyue Cheng, Yaguo Liu, Xingpeng Gao, Qingsong Wen, Shijin Wang, Enhong chen
RubricEM is a unified reinforcement learning framework designed for deep research agents that plan, search, evaluate evidence, and synthesize long‑form reports. It introduces rubric‑guided policy decomposition, combining supervised fine‑tuning to scaffold agents with rubrics, stage‑structured GRPO for fine‑grained credit assignment, and a reflection meta‑policy for experience reuse. The resulting RubricEM‑8B model outperforms larger open models on four long‑form research benchmarks and approaches the performance of proprietary deep‑research systems.
By Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister
The paper reports Alkhder’s submission to Subtask 1 of the AlexandriaX-2026 shared task, translating English dialogue turns into thirteen Arabic dialects. The team fine‑tuned NileChat‑3B‑Base with QLoRA on the Alexandria parallel dialogue corpus, training a single set of adapter weights for all dialects. Each translation is conditioned on target country, dialect, domain, participant roles, speaker‑addressee gender direction, and preceding dialogue context, and the model is run autoregressively using its own previous outputs as context. In the constrained track, training takes about one hour and inference on the private test set takes under two hours, achieving a country‑macro‑averaged spBLEU of 20.36 and chrF++ of 36.64 on both tracks.
By Hasan Alkhder, Maher Pasa, Maria Hanini, Amro Najjar, Ahmet Zengin
Pearl-Chat is an 8.5‑million‑parameter causal decoder‑only transformer built from first principles in JAX and Flax NNX, specifically engineered for the low‑resource Bantu language Luganda. It uses a byte‑level BPE tokenizer trained from scratch on curated Luganda corpora, achieving a two‑fold compression improvement over larger models like LLaMA‑3 and Gemma. The model is trained in two stages—pre‑training on diverse Luganda text and instruction fine‑tuning on dialogues—reaching a validation perplexity of 1.20 and high ROUGE‑L and BLEU scores on a held‑out diagnostic benchmark, with all weights, tokenizers, and code released openly.
By Wesley Kambale, Roland Sankara
The paper introduces a 141‑million‑parameter encoder‑based PII detector trained on 1.12 million examples from 77 public datasets, all relabeled under a unified definition of broad contextual detection for 12 structured PII categories. It achieves higher recall than larger baselines on the AI4Privacy and Nemotron‑PII benchmarks at low false‑positive rates, while delivering 2–10× higher throughput and 94–97% less peak GPU memory usage. The model’s gains include improved coverage of multilingual disclosures and IP addresses embedded in URLs.
By Andy Hsu, Konstantin Berlin
Thunder‑DeID+ is a Korean de‑identification system that improves boundary precision, label granularity, and recall when applied to court judgments and other sensitive documents. It introduces a refined three‑tier PII taxonomy and a boundary‑aware tokenizer that augments a pretrained BPE vocabulary with N‑gram and boundary‑scoring components, enabling better alignment of token and PII span boundaries. Evaluations show the tokenizer raises the W. Strict Typed F1‑score from 0.76‑0.77 to 0.87‑0.88 across models, and fine‑tuning on 1,000 broader documents yields a 0.93 score, demonstrating effective transfer beyond court judgments.
By Gyuseong Lee, Gyeongje Cho, Jongmin Kim, Heejin Kim, Yeonkyoung So, Hyunggeun Jeon, Hyunji M. Park, Sangho Kim, Yoonhee Park, Youngbin Pyo, Jaejin Lee
The paper introduces a controlled evaluation set focused on the Go concepts of atari and sente, aiming to test whether language models can recognize these terms and link them to move evaluation. The authors fine‑tune a Go language model, LoGos, with terminology and commentary supervision and then assess its performance on local board changes and strategic move preferences. Results show that LoGos struggles to correctly identify atari relationships and to assess their strategic implications, yet its terminology predictions still influence move choices, sometimes contradicting the reference evaluation.
By Taiyi Chen, Iskar Deng, Amandalynne Paullada
Cross-Centaur is a new language‑model framework that predicts individual decision‑making across diverse psychological tasks. It is trained on the Cross‑Psych‑101 dataset, which contains 591 participants and over 260,000 decisions from 12 different tasks. The model outperforms existing LLMs and the Centaur baseline on held‑out participants, and its internal representations appear to capture participant‑specific behavioral tendencies that transfer across tasks.
By Hikaru Asano, Hidezo Suganuma, Ryutaro Mori, Yukino Baba