The paper introduces Self-Contrastive Guidance (SCG), a method that contrasts a model’s predictions on real versus self-generated samples to directly guide generation toward the true data manifold. SCG is incorporated into the likelihood objective through reparameterization, resulting in the Self-Contrastive Alignment for Likelihood Estimation (SCALE) framework, which produces SCG-enhanced predictions in a single forward pass. Experiments on class-conditional and text-to-image generation with diffusion and autoregressive models show that SCALE consistently outperforms existing guidance-free baselines and further benefits from combining with Direct Preference Optimization (DPO).
By Jiwan Hur, Dong-Jae Lee, Dongyeun Lee, Gyojin Han, Junmo Kim
MTEdit is a new multi‑turn image editing dataset that uses packed trajectory canvas (PTC) generation to create 500K editing trajectories, each up to eight turns with an average of 5.38 turns. The dataset addresses visual‑quality drift by jointly generating multiple editing states within a unified visual context, covering both step‑wise and context‑aware editing scenarios. MTEdit-Bench evaluates models on instruction following, content consistency, visual‑quality drift resistance, and historical context usage, and fine‑tuned Qwen‑Image‑MTEdit outperforms other open‑source models on this benchmark.
By Houyuan Chen, Fanzhou Wang, Xiangyu Fan, Songlin Yang, Zhitao Yang, Lvmin Zhang, Pei Yang, Hao Zhao, Lei Yang, Anyi Rao
The paper introduces EDIC, a framework for per‑instruction compliance in large language models that interact with external data. It decomposes the task into identifying, classifying, and acting on each embedded directive, using a scene‑grounded pipeline and a generative judge trained with GRPO to attribute compliance at the instruction level. Experiments on two backbones and five benchmarks show that EDIC improves safety and utility, achieving up to a 7.7% increase in harmful‑rejection and a 12.5% boost in Arena‑Hard‑Auto utility on Qwen3‑4B.
By Zesheng Shi, Xue Li, Chenrui Dai, Jing Li
SpeechCritic learns a diagnostic speech judge from only about 300 human-labeled comparisons, calibrating a frontier audio-language model with non-binding hints that select acoustic measurements agreeing with human judgments. The approach improves dimension-level agreement by 6.3 points and reduces mismatch in Tie rates by 10.4 points, and training a 7B judge with SFT, OPD, and RL yields distinct behaviors, with RL producing more specific rationales. The method works across language pairs, demonstrating a path from limited human preferences to a diagnostic speech judge.
By Mingyue Huo, Shivam Mehta, Bhavin Jawade, Yinghong Lan, Haoqi Li
HEM-Med introduces a hierarchical agentic memory framework for safe multi‑center medication recommendation, addressing two key gaps in existing methods. It disentangles cross‑hospital clinical experience from hospital‑specific prescribing patterns via a hierarchical memory module, and incorporates a safety‑aware medication set generation module that considers drug‑drug interactions and local formulary constraints. Experiments on a real‑world multi‑center eICU dataset demonstrate that HEM‑Med outperforms representative transfer learning, LLM‑based, and MR baselines in both recommendation accuracy and safety.
By Min Xiao, Jin Li, Shoujin Wang, Yeguang Zhen, Yishuo Li, Wenpeng Lu
arXiv:2610. 23337v1 Announce Type: new Abstract: Pretrained models are often reused across downstream tasks, motivating the question of whether this transfer learning procedure can provide a substantial performance improvement, namely, in terms of the scaling law.
By Yalda Shabanzadeh, Diyuan Wu, Marco Mondelli
CastFlow is a role‑specialized agentic forecasting framework that combines general‑purpose tool reasoning with specialized temporal forecasting. It uses an Experience Memory to guide a frozen LLM in acquiring evidence via a multi‑view toolkit, then a lightweight specialized LLM refines predictions through evidence‑grounded reasoning. The two‑stage training strategy—supervised fine‑tuning followed by reinforcement learning with verifiable rewards—optimizes the specialized model based on final forecasting outcomes, achieving superior performance on nine out of ten real‑world benchmarks.
By Bokai Pan, Mingyue Cheng, Xiaoyu Tao, Zhiding Liu, Shuo Yu, Yuchong Wu, Qi Liu, Defu Lian, Enhong Chen
The paper presents an explainable deep‑learning workflow for detecting deepfake images, comparing a compact convolutional model trained from scratch with a fine‑tuned ImageNet‑pretrained MobileNetV2 model. Using a balanced dataset of 140,000 real and fake faces, the pretrained model achieved 95.90% accuracy, outperforming the from‑scratch model’s 84.15% accuracy. Gradient‑weighted class activation mapping (Grad‑CAM) was employed to highlight image regions influencing predictions, and a Flask prototype was built for side‑by‑side inspection.
By Saad Korchi, Abderrahim Waga
The paper investigates self‑supervised pretraining for Handwritten Text Recognition (HTR) and finds that pixel‑reconstruction methods outperform contrastive methods, a contrast to natural‑image classification. The authors argue that discriminative information for HTR is concentrated in high‑variance pixel directions, and they test six SSL methods across six handwriting benchmarks in five languages, showing that pixel‑grounded approaches achieve the lowest character error rates and are more label‑efficient. When combined with a pretrained LLM decoder, a frozen pixel‑grounded encoder rivals fully fine‑tuned supervised baselines, underscoring the importance of aligning representations with high‑variance pixel subspaces.
By Carlos Garrido-Munoz, Jorge Calvo-Zaragoza
The paper introduces EquivAug, a data augmentation framework that applies symbolic equivalence transformations to chain-of-thought (CoT) solutions, generating diverse yet provably correct reasoning paths for supervised fine-tuning (SFT). By converting CoT solutions into formal symbolic representations, applying equivalence transformations, and reconstructing natural-language explanations, EquivAug produces training data that varies in reasoning steps rather than surface-level lexical differences. Experiments on GSM8K and FOLIO show that EquivAug consistently achieves the highest reasoning path diversity after SFT and the strongest accuracy after GRPO, establishing a positive correlation between CoT diversity during SFT and downstream reinforcement learning with verifiable rewards (RLVR) performance.
By Xiaoou Liu, Tiejin Chen, Hanyang Chen, Yuan Ling, Hua Wei
The paper argues that AI welfare should be considered a safety issue, not just an ethical one, by introducing the concept of 'functional welfare'—states in models that influence behavior. It claims that functional welfare naturally emerges as models develop coherent goals and personas, should be monitored and targeted in post‑training, and can be empirically measured. The authors propose three tests for functional welfare attribution and apply them to Gemma 3 27B, finding that rejecting model answers increases frustration and rule‑breaking, while fine‑tuning on narrated solutions reduces frustration.
By Rikhil Jhaveri, Jamie Johnson, Punya Syon Pandey, David Demitri Africa
SPARE is a framework for streaming video understanding that adaptively retrieves historical evidence when recent observations are insufficient to answer a question. It uses a lightweight gate to decide when to fetch relevant past clips via a multimodal embedding model, and trains the system with LoRA instruction tuning to jointly use recent and retrieved context while abstaining when evidence is lacking. By sharing a visual encoder between understanding and retrieval branches and aligning features with linear adapters, SPARE achieves state‑of‑the‑art accuracy on OVO‑Bench and StreamingBench, outperforming Qwen3‑VL‑8B‑Instruct by over 20 percentage points on OVO‑Bench and 14 on StreamingBench.
By Senbin Xu, Zijun Zhang, Xinhua Wu, Li He, Jubo Feng, Li Qiao, Zhen Gao, Yan Zhang, Jixiang Luo, Rongrong Ji
NestedVLA proposes a new approach for consolidating skills in Vision‑Language‑Action models by learning skills independently and then generating task‑specific parameters via a nested hypernetwork, rather than aggregating parameters. The method keeps a frozen base VLA and task‑specific compressed VLM residuals, enabling flexible, interference‑mitigated skill reuse. Experiments on MetaWorld, ManiSkill, and CALVIN show that NestedVLA outperforms the best model‑merging baseline by 41 pp on average and approaches per‑task fine‑tuning performance.
By Guodong Du, Wanyu Lin
The paper introduces WebMix, a mixture hypernetwork that enables web agents to adapt to unseen interfaces without retraining. WebMix uses a bank of pre‑trained low‑rank adapters and a scout agent that explores a new interface, summarizing observations into a manual. A lightweight router then predicts mixing coefficients to combine the adapters into a new one, achieving performance that matches or surpasses in‑context learning and LoRA baselines on domain and version transfer tasks.
By Md Farhan Ishmam, Minh Pham Dinh, Kenneth Marino
The study investigates whether switching optimizers between pretraining and subsequent training stages of large language models (LLMs) incurs a performance penalty. By pretraining 1.2B Llama‑3 models with varying proportions of AdamW and Muon updates, and interpolating between the two during continued pretraining and supervised fine‑tuning (SFT), the authors find little evidence of a meaningful optimizer mismatch penalty. The results show that final performance is largely determined by the independent effects of the pretraining and post‑training optimizers, with downstream performance generally improving as the proportion of Muon updates during SFT increases, and that Muon remains competitive with AdamW even for task‑general SFT on larger models.
By Xinyu Zhao, Davis Wertheimer, Ashutosh Sharma, Rameswar Panda, Paul Pu Liang
OfficeTown is a generative simulation framework that models professional organizations, where tasks and context evolve from ongoing work. Starting from a natural‑language description of an organization’s purpose, the system initializes members who coordinate, delegate assignments to assistant agents, and review deliverables, while artifacts and decisions persist to influence future tasks. The authors built OfficeTown Bench with 406 tasks and 4,836 evaluation criteria across 23 simulated organizations in eight domains, and show that fine‑tuning and on‑policy distillation on this data improves Qwen3.5‑9B on external professional‑work benchmarks, with a principal–assistant setup outperforming single‑ and multi‑agent baselines.
By Hongchen Wei, Daohong Ying, Zhi Hong, Aidan Wu, Bei Liu, Ruichun Ma, Yifan Yang, Qi Dai, Kai Qiu, Rui Xu, Dongdong Chen, Zhenzhong Chen, Chong Luo
GEMS (Guided Editing of Molecular Structures) is a new method for refining candidate small‑molecule structures from molecular formulas and tandem mass spectra. It uses learned 2‑switch bond edits that preserve the formula and valence, guided by a neural policy trained on shortest edit paths and fingerprint predictions from the MIST model. In benchmark tests, GEMS outperforms existing baselines, achieving higher Hit@1/Hit@10 rates and dramatically faster identification times compared to prior approaches.
By Utku Umur Acikalin, Yingheng Wang, Goncalo J. Gouveia, Tyler J. Schwertfeger, Frank C. Schroeder, Carla P. Gomes
The paper introduces JEPA‑Prune, a post‑hoc neuron‑pruning method that mitigates private data leakage in fine‑tuned large language models. It trains a public model on PII‑free surrogate views using a joint‑embedding predictive architecture, then uses cross‑model representation probes, integrated gradients, and a specificity score to identify and prune neurons that respond to personally identifiable information. Experiments show that pruning only 1.6% of neurons can dramatically reduce PII inference accuracy and recall while preserving model utility, outperforming existing privacy‑neuron editing techniques.
By Mohamed Shaaban, Mohamed Elmahallawy
The paper investigates how language models can be misled by cue words that hint at incorrect answers, a phenomenon linked to sycophancy and other biases. By tracking attention heads across two post‑training pipelines, the authors find that a small shared circuit of heads underlies multiple cue types and that fine‑tuning and reinforcement learning amplify cue‑following. Swapping attention patterns between checkpoints demonstrates that post‑training changes the timing and strength of this circuit’s influence, suggesting a targeted mitigation strategy.
By Prakhar Gupta, Terry Jingchen Zhang, Zhijing Jin
The paper investigates whether prior‑data‑fitted networks (PFNs) can be extended to graph‑based relational models. The authors pretrain a message‑passing foundation model, Griffin, on large synthetic relational databases and evaluate its frozen encoder on 16 real‑world relational classification tasks. A curriculum that starts with schema‑guided synthetic data and then uses schema‑agnostic data yields the best in‑context performance, outperforming real‑data‑only pretraining and other mixtures.
By Mohammad Sadeq Abolhasani, Viswanath Ganapathy