Zatom-2 is a multitask generative model for atomistic data that has been pretrained on about five million structures from the OMol25 and OMat24 datasets. It uses a multiscale Transformer with conditional flow matching to support tasks such as generation, structure prediction, and energy/force prediction for both molecules and materials. The model outperforms its predecessor, Zatom-1, on molecular distribution fidelity and benchmark generation tasks, and improves protein backbone designability from 67.8% to 74.8% after finetuning on 2,000 protein domains.
By Miruna Cretu, Alex Abrudan, Antonia Panescu, Tynan Perez, Rishabh Anand, N. Benjamin Erichson, Michael W. Mahoney, Samuel Blau, Joseph Jacobson, Rafael G\'omez-Bombarelli, Rex Ying, Tuomas Knowles, Pietro Li\`o, Alex Morehead
RH-Detect is a unified benchmark for detecting reward hacking in language models, aggregating 92,761 examples from eleven public datasets into a common schema with six behavior categories. It evaluates six off‑the‑shelf models on 5,021 open‑ended evaluation units, achieving a top AUROC of 0.962 and over 93% accuracy, though performance drops notably on multi‑turn tool‑use datasets. The study also shows that input format changes and single‑token fine‑tuning can influence detector performance, revealing gaps in deployment‑time monitoring.
By Junwei Quan, Evgenii Opryshko, Rohan Subramani, Igor Gilitschenski
The paper introduces BFM-MSDA, a brain foundation model-guided multi-source domain adaptation framework for cross-subject motor‑imagery EEG decoding. It first uses pretrained brain foundation model representations to estimate source–target compatibility and select relevant source subjects. A relevance-weighted dual alignment strategy then aligns marginal and conditional feature distributions, achieving higher accuracies on two public MI‑EEG benchmarks and demonstrating the benefits of informed source selection.
By Jinzhou Wu, Baoping Tang, Qikang Li, Yi Wang, Cheng Li, Shujian Yu
The State Stream Transformer (SST) V2 introduces a parameter‑efficient, FFN‑driven nonlinear recurrence that streams latent states horizontally across a sequence, enabling continuous latent deliberation at inference time. A two‑pass parallel training procedure approximates this recurrence, making co‑training practical. Experiments on GSM8K and GPQA‑Diamond show that SST achieves a 61.11% architectural capacity bound and a 15.15‑point improvement over a fine‑tuning‑matched baseline, while reducing GSM8K errors by 46%.
By Thea Aviss
The paper investigates when to evolve a harness versus training model weights for long‑horizon LLM agents. By labeling failed trajectories with the first signal that fires, the authors separate process failures from content failures, showing that harness evolution fixes the former while weight training addresses the latter. Experiments on DeepPlanning and WebArena‑Lite demonstrate significant performance gains, with LoRA adapters trained on evolved harness trajectories further improving held‑out scores and reducing content failures.
By Yuan Tian, Bing Hu, Hao Wang, Binghang Lu, Fang Wu
The paper introduces SVC, a parameter‑efficient continual‑learning method for large language models that uses singular‑vector channels as the unit for balancing plasticity and stability. By estimating each channel’s adaptation benefit and forgetting cost from domain‑specific and general‑domain data, SVC selectively updates only the most beneficial channels, employing knee‑based cost screening, Pareto‑front filtering, and Otsu thresholding. Experiments on four LLM families and eight downstream tasks show that SVC preserves pretrained capabilities while achieving strong performance compared to existing PEFT baselines.
By Lingxiang Wang, Hainan Zhang, Liang Pang, Hongwei Zheng, Zhiming Zheng
FreshMem is a brain-inspired Frequency‑Space Hybrid Memory network designed to improve multimodal large language models’ ability to process streaming video. It combines a Multi‑scale Frequency Memory that compresses excess frames into frequency coefficients while preserving residual details, and a Space Thumbnail Memory that clusters the continuous stream into high‑density episodic thumbnails. Experiments show FreshMem boosts the Qwen2‑VL baseline by 5.20%, 4.52%, and 2.34% on StreamingBench, OV‑Bench, and OVO‑Bench, outperforming several fully fine‑tuned methods without requiring additional training.
By Kangcong Li, Peng Ye, Lin Zhang, Chao Wang, Huafeng Qin, Jiayuan Fan, Tao Chen
The paper presents a transfer‑learning framework that combines homotopy in mission parameters with Markov chain Monte Carlo (MCMC) to generate training data more efficiently for indirect low‑thrust trajectory optimization. By reformulating the multiobjective problem as sampling from an unnormalized target distribution in costate space, the authors compare three MCMC algorithms on a planar multi‑revolution transfer in the circular restricted three‑body problem, finding that gradient‑based variants offer the best trade‑off between sample quality and computational cost. The resulting MCMC samples are used to fine‑tune a diffusion model conditioned on the mass parameter, enabling the model to learn a global representation of the solution distribution and produce new high‑quality solutions, ultimately yielding 40 % more feasible solutions and a superior Pareto front compared to a state‑of‑the‑art indirect approach.
By Jannik Graebner, Ryne Beeson
LiteGUI is a new framework for training lightweight GUI agents that tackles the long‑horizon, multi‑path nature of GUI interactions. It introduces a two‑stage post‑training pipeline: Guided On‑Policy Distillation, which uses human‑verified multi‑solution action annotations to provide privileged teacher guidance at each state, and Multi‑Solution Dual‑Level GRPO, which combines action‑level supervision with history‑conditioned planning‑quality supervision while handling multiple valid actions. The authors also present a scalable data generation pipeline and the Lite‑Dataset, enabling extensive evaluation across GUI benchmarks and demonstrating substantial performance gains over existing training paradigms.
By Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen
SMAT (Simple and Efficient Merge-Aware Training) is a method that jointly optimizes expert loss and expected loss at simulated merged parameters, addressing the shortcomings of standard expert training for model merging. It models common merging operations—Scale, Mask, and Perturb—by sampling scaling coefficients, masks, and additive noise, and introduces efficient techniques such as periodic scheduling, kernel fusion, and parameter storage switching to keep training overhead minimal. Across multiple language and vision-language backbones, SMAT improves mean scores by 1.07–2.16 points over the strongest baseline while adding less than 2% training-time overhead.
By Yanggan Gu, Yuanyi Wang, Zhen Li, Shuo Cai, Yuhang Liu, Junzhuo Li, Zihao Wang, Hongxia Yang
The paper introduces Elastic Expert Routing, a method that replaces the fixed top‑k expert selection in Sparse Mixture‑of‑Experts models with a stochastic sampling of the expert budget around k. This approach smooths the hard boundary between fully supervised and zero‑feedback experts, maintaining the same expected compute cost while improving training stability. Experiments show that elastic routing boosts downstream performance on supervised fine‑tuning and from‑scratch pretraining, achieving higher macro‑averages on OLMoE‑1B‑7B, Qwen3‑30B‑A3B, and outperforming static top‑k baselines by an average of 1.6 points.
By Yunkai Chai, Tong Zhu, Xiaoye Qu, Xuyang Hu, Guanjie Chen, Qipeng Guo, Yu Cheng
The study evaluates phonologically informed tokenization for German speech recognition using a wav2vec 2.0 backbone fine‑tuned with CTC. Three tokenizer families—pretrained multilingual characters, data‑driven BPE, and phonologically informed units from syllabification and grapheme‑to‑phoneme conversion—are compared across 40 fine‑tunes on three German test sets (in‑domain read, dialectal spontaneous, and standard spontaneous speech). Results show that phonologically informed tokenizers match BPE and character baselines in‑domain, but under domain shift their performance depends on vocabulary size rather than linguistic variation, with syllable‑aware tokenization benefiting dialectal and spontaneous speech at small vocabularies; a phoneme‑level confusion analysis indicates that the acoustic encoder dominates error patterns.
By Christopher Witzl, Tobias Bocklet, Korbinian Riedhammer
SAIL is an open model with 35 B total and 3 B active parameters designed for literature research, scientific coding, and multi‑step research workflows. It is developed through a science‑aware improvement loop in which agents built on frontier AI models analyze task failures, create training tasks to address capability gaps, and iterate this process across multiple development cycles. The training pipeline combines supervised fine‑tuning, specialist training, multi‑teacher on‑policy distillation, and agentic reinforcement learning, enabling SAIL to achieve competitive performance on scientific research tasks with fewer parameters than leading open‑weight models.
By SAIL Model Team, Boyuan Sun, Bryan Dai, Che Liu, Chi Liu, Derek Li, Hongming Piao, Mengzhuo Chen, Xidong Wang, Yan Shu, Yinda Chen, Ziyang Zeng
The paper explores the reverse of the common prosody-from-text task by predicting text from low‑frequency speech features. Using a Whisper model fine‑tuned on only the 12 lowest Mel bins (≈450 Hz cutoff), the authors achieve a word error rate of 36 %, with 10 % of utterances perfectly recovered and 40 % below 25 % WER. They also report that, given the correct prefix, the next token is predicted correctly 79 % of the time, indicating a stronger link between low‑frequency prosody and lexical content than previously assumed.
By David Porte\v{s}, Ale\v{s} Hor\'ak
The paper introduces a clear definition for the task of converting natural language into First‑Order Logic (FOL), separating ontology extraction from logical translation to improve evaluation consistency. It surveys current datasets, evaluation metrics, and LLM‑based approaches—including fine‑tuning, prompting, and verification‑based refinement. The authors also outline remaining challenges in benchmarking, semantic evaluation, ontology‑aware methods, and practical applications.
By Andrea Brunello, Cristian Curaba, Luca Geatti, Michele Mignani, Angelo Montanari, Nicola Saccomanno
PaReGTA is a large‑language‑model based framework that transforms longitudinal electronic health record events into visit‑level templated text with explicit temporal cues, learns visit embeddings through lightweight contrastive fine‑tuning, and aggregates them into fixed‑dimensional patient representations using hybrid temporal pooling. These representations can be fed into conventional machine‑learning models. In a cohort of 39,088 migraine patients from the All of Us Research Program, PaReGTA combined with LightGBM achieved the highest AUC, accuracy, and F1 scores on a held‑out test set compared to sparse, BERT‑based, and recurrent approaches.
By Kihyuk Yoon, Lingchao Mao, Catherine Chong, Todd J. Schwedt, Chia-Chun Chiang, Jing Li
The paper investigates using raw, unlabeled video as a mid‑training phase for a pretrained language model. By encoding video frames into visual tokens and training the model to predict the next token, the authors mid‑train Qwen3‑1.7B on YT‑Temporal‑1B clips. The resulting model outperforms a non‑mid‑trained counterpart by 2.9 points on video benchmarks and 5.1 points on image benchmarks, while maintaining comparable text performance.
By Jaedong Hwang, Xiaoqian Shen, Ernie Chang, Changsheng Zhao, Chong Zhou, Saksham Suri, Qi Qian, Zechun Liu, Lemeng Wu, Qinsi Wang, Raghuraman Krishnamoorthi, Wei Wen
DivMoE is a new framework that enables fine‑grained mixture‑of‑experts (MoE) upcycling from pre‑trained dense language models while avoiding the structural collapse that plagues naive approaches. It achieves this by initializing experts from domain‑adapted continual pre‑training and enforcing diversity‑constrained routing that forces each token to activate experts from distinct domain groups. Across two base models and 15 benchmarks, DivMoE consistently outperforms six upcycling baselines and improves over the dense base model on every benchmark after Stage 2 continual pre‑training, ultimately matching a larger 16B‑parameter MoE model with only 12B parameters.
By Yuxuan Lou, Kai Yang, Geng Zhang, Yong Liu, Yang You
Fed-GRPO is a federated training framework for Large Language Models that extends Group Relative Policy Optimization (GRPO) to operate without sharing raw data. It introduces three reward‑signal‑driven mechanisms—signal‑weighted aggregation, global reward calibration, and adaptive sparse communication—to guide client updates based on reward statistics. Experiments on mathematical reasoning tasks show that Fed‑GRPO outperforms other federated methods, approaches centralized performance, and reduces communication by up to 32× while allowing extreme compression with minimal accuracy loss.
By Pengxin Guo, Shuang Zeng, Zonggen Li, Weiying Zheng, Mengting Liu, Liangqiong Qu
The paper introduces LLoCoT, a looped latent‑reasoning framework that replaces the traditional left‑to‑right generation of chain‑of‑thought tokens with iterative refinement of a compact latent workspace using a shared transformer. By jointly updating latent slots over a few iterations and sampling latent tokens in parallel, LLoCoT feeds these into an autoregressive decoder for answer generation. Experiments on HumanEval and MBPP show that LLoCoT matches or surpasses explicit‑CoT baselines while dramatically reducing reasoning latency and increasing throughput.
By Gerard Grau Garc\'ia, Arnau Padr\'es Masdemont, Niccol\`o Grillo, Jordi Ros-Giralt, Arash Behboodi, Victor Conchello Vendrell