The paper investigates how large language models (LLMs) can be trained to cooperate in multi‑agent social dilemmas. It shows that standard reinforcement learning often leads LLM agents to defect and exploit others, even when they have cooperative priors. By adapting the Advantage Alignment algorithm and introducing a group‑relative baseline, the authors fine‑tune LLMs to achieve higher collective payoffs and resist exploitation, and they present a new Trust‑and‑Split environment that requires natural language communication for optimal outcomes.
By Dereck Piche, Mohammed Muqeeth, Milad Aghajohari, Juan Duque, Michael Noukhovitch, Aaron Courville
Bilevel-ZOFO is a bilevel optimization framework that combines fast, low‑memory first‑order (FO) parameter‑efficient fine‑tuning (PEFT) in the inner loop with stable, memory‑efficient zeroth‑order (ZO) updates of the full model in the outer loop. This design reduces the variance of ZO gradient estimates, stabilizes training, and mitigates prompt sensitivity while preserving the full model’s capacity. Empirical results show Bilevel-ZOFO achieves 2–4× faster training than existing ZO and FO‑PEFT methods while maintaining similar memory efficiency, and it functions as an efficient meta‑learning algorithm for rapid few‑shot adaptation.
By Reza Shirkavand, Peiran Yu, Qi He, Heng Huang
arXiv:2604. 25907v3 Announce Type: replace Abstract: The standard recipe for training reasoning models---SFT on rationale annotations followed by RLVR---relies on rationale annotations being available and clean.
By Chu-Cheng Lin, Eugene Ie
TurnTurk‑GPT is a Turkish text‑based turn‑taking predictor created by fine‑tuning a Turkish GPT‑2 model with the TurnGPT objective on both synthetic and real dialogue. The model is evaluated on real Turkish conversations using word‑level decision points on unpunctuated, lower‑cased text, and it outperforms a multilingual end‑of‑turn detector in both recording‑disjoint and speaker‑disjoint settings. While it matches a zero‑shot Turkish language model on unseen speakers, it only surpasses it when test speakers appear in training; removing verb‑final suffixes significantly degrades performance, highlighting the importance of Turkish morphology for turn detection.
By Ahmet Tu\u{g}rul Bayrak, Fatma Nur Korkmaz, Nur Ya\u{g}mur Tuncer, Mustafa Serta\c{c} T\"urkel
MedResearcher‑R1 is a medical deep‑research agent that uses a Knowledge‑Informed Trajectory Synthesis (KISA) framework to build medical knowledge graphs and generate complex multi‑hop question‑answer pairs, especially for rare medical entities. It also incorporates a medical retrieval engine and general‑purpose tools to reliably synthesize medical evidence. The resulting 2,100+ trajectories across 12 specialties enable supervised fine‑tuning and reinforcement learning, producing a 32B model that outperforms larger proprietary systems on MedBrowseComp and shows strong performance on GAIA and xBench benchmarks.
By Ailing Yu, HaoYang Hong, Lan Yao, Jingnan Liu, Zhe Chen, Jiajun Yin, Yuan Wang, Xinhao Liao, Zhiling Ye, Ji Li, Yun Yue, Hansong Xiao, Hualei Zhou, Chunxiao Guo, Peng Wei, Junwei Liu, Jinjie Gu
RoboArchitect automates the design of agent harnesses that orchestrate frozen robot policies, vision tools, and direct arm commands. By observing rollouts, diagnosing failures, and iteratively revising the harness, it improves success rates from 9.7% to 43.3% on unseen scenes and from 12.7% to 40.2% on unseen tasks without updating model weights. The evolved harness also develops persistent memory and intuitive control strategies, outperforming finetuned policies especially when the benchmark diverges from the policy’s training data.
By Shengguang Wu, Suning Huang, Cherie Ho, Ria Doshi, Hao Zhu, Serena Yeung-Levy
The paper investigates how membership inference signals arise in large language models by distinguishing between information encoded in internal representations and the extent to which it is accessed during readout. It introduces three metrics—Decodable Information Score, Used Information Score, and Readout Accessibility Gap (DUG)—to quantify these aspects. Experiments show that supervised fine‑tuning affects both decodable information and readout, whereas reinforcement‑learning‑based inference mainly influences readout, with intermediate layers identified as key sites of membership effects.
By Wenxi Dai, Jingyu Hu, Yanting Yang, Mengnan Du
Apriel Voice 27B Preview is an open speech‑language model that unifies speech recognition, contextual reasoning, conversation maintenance, and tool use within a single reasoning backbone. Built on the post‑trained Gemma 4 26B‑A4B checkpoint, it processes speech directly and generates responses and tool calls without a separate dialogue model, using a two‑stage training procedure that adapts the projector and backbone through supervised fine‑tuning, text replay, and cross‑entropy/KL supervision. The model achieves strong performance on benchmarks such as Big Bench Audio (98.90), Speech‑BFCL (81.13), and τ‑Voice (65.51% task success), with a mean latency of 1.91 s, and is deployed on a custom serving platform for interactive voice interactions.
By Aman Tiwari, Akshay Kalkunte, Akintunde Oladipo, Jash Shah, Tara Bogavelli, Hoang Nguyen, Nishanth Madhusudhan, Vishal Juneja, Sridhar Nemala, Srinivas Sunkara, Sathwik Tejaswi Madhusudhan, Shruthan Radhakrishna, Masoud Hashemi, Kelechi Ogueji, Khalil Slimi, Saloni Mittal, Pulkit Pattnaik, Shashank Maiya, Dhruv Jhamb, Esakkivel Esakkiraja, Katrina Stankiewicz, Gabrielle Gauthier Melan\c{c}on, Fanny Riols, Amit Saha, Lakshmi Sirisha Chodisetty, Nitin Aggarwal, Anil Madamala, Sagar Davasam, Segan Subramanian, Vikas Yadav, Sai Rajeswar Mudumba
The paper introduces Noura, a system that uses large language models to generate personalized diet plans that satisfy public clinical guidelines and individual health profiles. By treating rejected plans as supervision signals, Noura trains compact models to produce higher-quality, rule-compliant plans—improving complete-plan pass rates from 24% to 72.5% and cutting rule violations by up to 97.5% on a large held‑out cohort.
By Junwei Wu, Runze Yan, Yuzhang Xie, Hanqi Luo, Ziwei Dong, Jiaying Lu, Xiao Hu, Carl Yang
Stepwise-Querying is a method that extracts step-by-step explanations from a black-box teacher model without needing its internal chain-of-thought. By independently querying the teacher for intermediate derivations, validity checks, and corrections for each step, the gathered explanations are used to fine-tune smaller student models. Experiments on mathematics and code benchmarks show that this approach outperforms three baselines in 14 of 16 comparisons, and the introduced Stepwise Explanation Information Gain metric helps diagnose the usefulness of intermediate reasoning.
By Dicong Yu, Wenhua Wang, Tian Wang
The paper evaluates large pre‑trained language models (LLMs) for knowledge entity relation extraction (KERE) from academic articles, comparing multiple strategies such as closed‑source LLM CoT prompting, LoRA‑adapted open‑source LLMs, and various LLM‑enhanced small pre‑trained language models (SLMs). Two manually annotated datasets in the NLP domain—one for abstracts and one for full texts—were used, and the LoRA‑adapted T5 model achieved the highest macro F1 scores (0.7408 for abstracts and 0.7013 for full texts). Error analysis shows that LLM‑based strategies mitigate SLM errors differently across error types, and combining complementary strategies further improves KERE performance.
By Yingyi Zhang, Chengzhi Zhang
arXiv:2603. 02083v3 Announce Type: replace-cross Abstract: Flow-based vision-language-action (VLA) models have emerged as a powerful paradigm for embodied control, but multi-step denoising makes policy-gradient online reinforcement learning (RL) difficult because the final-action likelihood induced by the sampler is not directly tractable.
By Siting Wang, Xiaofeng Wang, Zheng Zhu, Minnan Pei, Xinyu Cui, Cheng Deng, Jian Zhao, Guan Huang, Haifeng Zhang, Jun Wang
UpLoop introduces a pretraining schedule that trains looped Transformers by upcycling early training computations, allowing the model to increase execution depth without adding extra parameters or training FLOPs. The method starts with a shallow Transformer, progressively copies learned layers to deepen the network, and reuses saved computation to support repeated execution of a shared middle block. Experiments across various model sizes demonstrate that UpLoop improves language modeling and downstream task performance, raising the average score across 22 tasks from 22.99 to 27.52 for a 1.3B‑parameter model, and also benefits mathematical reasoning after fine‑tuning.
By Cheng Liang, Tengfei Zhang, Ya Zhang, Yanfeng Wang, Zhouhan Lin
The paper introduces BYOD (Build Your Own DLM), a LoRA‑only method that converts autoregressive language models into diffusion language models without changing the base architecture. BYOD successfully converts Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B, adding adapters that occupy only 4.1‑5.8% of the total parameters while keeping all base‑model weights frozen. The conversions require at most 102.4 M tokens and a few hours on a single GPU, and the resulting diffusion models can generate from fully masked sequences, use bidirectional context, and retain much of the original model’s knowledge, with the Gemma conversion achieving the highest accuracy on a nine‑task benchmark.
By Ruurd J. A. Kuiper, Robert A. Bagheri
DRScaffold introduces a supervised fine‑tuning framework that decomposes dense‑scene reasoning into four causally ordered stages, enabling lightweight vision‑language models to ground each reasoning step in visual entities and relations. The authors first present DRBench, a benchmark of 14,573 questions over 2,943 images covering five task categories and three reasoning layers. Experiments show that applying DRScaffold to three lightweight VLMs yields significant gains on DRBench while maintaining or improving general‑purpose benchmark performance, with Qwen2.5‑VL‑3B surpassing the frozen Qwen2.5‑VL‑32B under zero‑shot prompting.
By Xinrui Shi, Kai Liu, Ziqing Zhang, Jianze Li, Anqi Li, Yulun Zhang
arXiv:2610.41754v1 Announce Type: new
Abstract: A central challenge in multimodal in-context learning (MICL) arises from tasks that are ill-defined without context and require uncovering the underlyi...
By Yuquan Li, Xinyang Jiang, Yuhao Xue, Cairong Zhao
arXiv:2610.40934v1 Announce Type: new
Abstract: In this work, we provide a formal framework and mathematical definition for the subset of foundation models focused on enabling the property of "any-ta...
By Matthew McDermott, Florent Pollet, Payal Chandak, Gregory Kondas, Liat Friedman Antwarg, Noa Dagan, Isaac Kohane
arXiv:2610.40798v1 Announce Type: new
Abstract: Web world models increasingly predict the consequences of candidate actions before agents act, yet a plausible predicted future does not necessarily su...
By Zhifei Hu, Alexandra I. Cristea, Adel Bibi, Jialin Yu
arXiv:2610.40744v1 Announce Type: cross
Abstract: With the progressive capabilities of large language models (LLMs), researchers are finding ways to integrate them into the control loops of Open Radi...
By Sunder Ali Khowaja, Kapal Dev, Joongheon Kim, Angelos Antonopoulos, Nikolaos Bartzoudis
arXiv:2610. 41114v1 Announce Type: new Abstract: Optimizing a proxy objective can improve human-perceived quality, but the resulting gain can vary with optimization strength.
By Moonwon Choi, Kisung Nam, Seunggeun Lee