STAPLE is a method that integrates static sparse memory into pretrained language models by adding token‑indexed embeddings as a residual pathway while preserving the original feed‑forward network. On the OLMo‑2 1B and 7B models, STAPLE outperforms dense training and other static sparse baselines, improving benchmark scores by up to 4.27 points and achieving faster decoding with less GPU memory and energy usage. The approach demonstrates that large pretrained models can be expanded efficiently without costly sparse pretraining from scratch.
By Ranajoy Sadhukhan, Euijin Hong, Sheng Cao, Zechun Liu, Shuowei Jin, Sha Li, Changlong Yu, Xin Liu, Qin Lu, Zhenyu Shi, Priyanka Nigam, Beidi Chen
The paper investigates two key design levers for Continuous Diffusion Language Models (CDLMs): self‑conditioning and embedding geometry. It shows that self‑conditioning mitigates a train/inference mismatch and can be implemented efficiently via a two‑level interpolant called DoubleCat, which significantly boosts TinyGSM accuracy. By adding discrete reveals to DoubleCat, the authors create SemiCat, achieving discrete‑diffusion‑level performance and outperforming discrete diffusion on several likelihood‑based benchmarks at 1.7B parameters. The study also finds that cross‑entropy training drives embeddings toward a geometry with decoding error rates comparable to one‑hot representations, and briefly explores additional options such as hybrid corruption processes and asynchronous noise schedules.
By Oscar Davis, Daan Roos, Alice Bizeul, Amitis Shidani, Victor Turrisi, Louis B\'ethune, Pierre Ablin, Marco Cuturi
The paper proposes Predictive-State Flows (PSF), a new architecture for continuous diffusion language models that separates the diffusion prior’s role to generate high‑level predictive states and the decoder’s role to resolve token‑level choices. PSF partitions sequences into blocks, uses a rectified‑flow prior to produce predictive states, and then a block‑causal decoder generates all blocks in parallel, reducing token‑level sequential depth. Experiments on OpenWebText show that a 197M‑parameter PSF model with eight predictive states achieves a generative perplexity of 22.8 and a MAUVE@512 of 0.93 while using far fewer FLOPs than a comparable ELF‑B model.
By Jinjin He, Xinsong Feng, Lucheng Fu, Sinan Wang, Bo Zhu, Jiayun Wang
The paper introduces TAP-JEPA, a joint‑embedding predictive architecture trained on over 113,000 CT volumes, which achieves performance comparable to leading encoders in scan‑level classification. It demonstrates a predictability bias: while anatomy is largely predictable from context, pathology is not, leading to significant degradation in pathology detection when predictions are applied. The study audits the model’s predictive pathway, showing that pathology representations shift toward control regions and that removing pathology tokens minimally affects downstream decisions, underscoring that contextual predictability alone is insufficient for clinical relevance.
By Fabian Drexel, Bernhard Kainz, Jakob N. Kather, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem, Cosmin I. Bercea
The paper investigates how well two-stage structured Hadamard rotations approximate uniform random rotations in high-dimensional spaces. It establishes nonasymptotic upper bounds of order √{log d/d} for one-dimensional marginals, while showing a persistent Wasserstein-1 discrepancy for the full vector that does not vanish as dimension grows. These results highlight a separation between marginal and joint approximation quality, though they do not directly translate into algorithmic performance guarantees.
By Tomer Zilca, Gal Mendelson
KV-Repurpose is an open‑source, vLLM‑based system that enables segment‑wise KV‑cache reuse beyond exact prompt prefixes in large language model serving. It integrates cache‑aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring to evaluate non‑prefix KV‑cache reuse. Experiments show that naive segment reuse maintains output quality for single‑document QA and summarization, while significantly improving cache hit rates and reducing time‑to‑first‑token compared to prefix caching, especially under Zipfian RAG‑style workloads.
By Taiga Ikeda, Daisuke Miyashita, Jun Deguchi
The paper introduces REFLECT, a role-aware transformer that integrates Semantic Role Labeling (SRL) configurations into self‑attention to better detect sentiment, emotion, politeness, and formality (SEPF) in dialogue. By fusing role-aware representations with contextual embeddings through a learned gate, REFLECT achieves significant improvements over BERT baselines on emotion, politeness, and formality tasks, especially for low‑frequency and complex cases, while sentiment gains are minimal. The approach also enhances interpretability and generalizes to real‑world mental‑health data.
By Sangpil Youm, Annabel Stocks Natalias, Hayden Heninger, Leah Jones, Xiaoyang Jin, Xiaoya Zhang, Bonnie Dorr
The paper investigates how vision‑language models (VLMs) resolve conflicts between image features using a Stroop‑style benchmark. By prompting contrastive and generative VLMs on images where text and color disagree, the authors find a consistent bias toward textual cues, with models preferring to respond based on text rather than color. Analysis of the embedding space reveals that textual signals are represented more strongly and separably than color signals, explaining why steering models toward text is easier than toward color.
By Nurb\"uke Teker, Rui Xiao, Zeynep Akata, Shuchen Wu
The paper investigates what information is captured by egocentric video embeddings from models such as VideoMAEv2, CLIP, DINOv2, and VC-1, finding that the leading principal components largely reflect where the head‑mounted camera is looking. It also explores how to combine these video embeddings with body‑worn inertial sensor data for human activity recognition, showing that propagating inertial predictions over a video‑based graph can significantly improve classification performance (macro‑F1 from 0.648 to 0.852). The study demonstrates that video and inertial modalities play distinct roles—video identifies similar clips within a set, while inertial data labels the activities—yet fusing them directly in a single classifier or adding inertial edges to the graph does not yield better results.
By Kansuke Habano
Chronos Policy introduces explicit physical time into robot control by conditioning on observation capture times and desired action execution times. It explores two diffusion transformer adaptations: timestamp embeddings and time‑aware rotary positional encodings (RoPE), finding that RoPE offers greater robustness. Training with temporally randomized observation sampling exposes the policy to varied sensing and execution schedules, enabling learning from asynchronous, multi‑rate data without a fixed temporal grid.
By Changhao Wang, Jaden Clark, Shuran Song
The paper investigates how Markov chains converge within a geometry defined by Hilbertian embeddings of positive definite kernels, also known as Maximum Mean Discrepancy (MMD). By assuming a contraction in this kernel-induced geometry and a standard spectral gap, the authors derive non-asymptotic, high‑probability bounds on the empirical occupation measure’s convergence to the invariant measure. These bounds separate the influence of the spectral gap, which sets the effective sample size, from the kernel contraction, which appears only as a linear correction. The results are applied to one‑step prediction of dynamical systems from a single trajectory, yielding fast learning rates for a nonlinear least‑squares estimator that illustrate the interaction between kernel‑induced mixing and classical source and capacity conditions.
By Oleksii Kachaiev, Luc Brogat-Motte, Silvia Villa, Lorenzo Rosasco
Lagrangian Neural Networks (LNNs) can learn irreducible three‑body interactions from trajectory data by parameterizing only the potential while prescribing kinetic energy. Using the Axilrod‑Teller triple‑dipole interaction as a test case, the authors show that a signal‑aware sampling strategy allows the LNN to recover the correct radial scaling, angular dependence, and sign reversal from accelerations alone. Symbolic regression on the trained potential reproduces the exact Axilrod‑Teller expression, demonstrating that LNNs can uncover closed‑form many‑body laws without prior functional assumptions.
By Sena Kalabalik, Sukru Caglar, Abdullah Umut Hamzaogullari, Arkadas Ozakin
AgentTune is a source‑available Python library that offers a unified training lifecycle for tool‑using large language model agents. It introduces a two‑tier EventLog representation to preserve agent trajectories and enable consistent training, evaluation, and auditing. The framework supplies a shared runtime, reinforcement learning for tool‑calling, agentic distillation, retrieval‑augmented generation, decision workflows, evaluation utilities, and a path toward self‑healing agents, all exposed through composable interfaces for flexible combination of strategies and methods.
By R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
The study evaluates how well different medical foundation models agree on image embeddings by measuring mutual k‑nearest‑neighbor overlap across 750,000 images from 14 datasets and 18 public models. Results show that label‑supervised models converge the least, sharing only 6–10% of nearest neighbors, whereas models trained with self‑distillation, masked image modeling, or contrastive pretraining share up to 80%. The authors suggest that developers can select pretraining objectives to control convergence and recommend testing unlabeled images for nearest‑neighbor consistency before replacing a model.
By Soroosh Tayebi Arasteh, Sebastian Ziegelmayer, Mahshad Lotfinia, Lisa Adams, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn
The paper introduces Shared Canonical Component Removal (SCCR), an audit framework that quantifies the predictive gap between raw-input black-box models and expert-feature-based models. SCCR selectively removes only the directions that are strongly shared between learned raw and expert embeddings, unlike standard residualization or concept erasure. Experiments on synthetic data, coronary artery disease detection, antidepressant treatment dropout prediction, and skin lesion classification demonstrate that SCCR reveals where expert views fall short and recovers residual predictive signal while minimizing redundancy with the expert view.
By Jose Roberto Tello Ayala, Akl C. Fahed, Roy Perlis, Finale Doshi-Velez
Neurograf v1 is an experimental end‑to‑end Romanian‑first language‑model pipeline that includes a 635 GB Romanian corpus (RoSfera), an 80k byte‑level BPE tokenizer, and five decoder‑only models ranging from 160 M to 6 B parameters trained on 131 B tokens. The pipeline also produces three instruction‑tuned assistants and two contrastively trained embedding models, all trained on ELI‑NP/IFIN‑HH infrastructure. Evaluation shows consistent scaling of validation loss and improved Romanian factual and cultural knowledge with larger models, achieving 69.89 % accuracy on the 1,000‑question WWTBM benchmark for the 6 B model.
"whyItMatters":"The study demonstrates that it is feasible to train Romanian‑first language models from scratch, highlighting practical challenges such as diacritic preservation and tokenization efficiency while providing a unified training pipeline for base, instruction‑following, and embedding models."
By Ionut Adrian Rotaru
The paper introduces MVNN, a measure‑valued neural network designed to learn interaction forces in systems governed by McKean‑Vlasov dynamics directly from particle trajectory data. MVNN extends conventional neural networks to operate on probability measures by learning cylindrical features through an embedding network that maps distributions to scalable vector representations. The authors prove theoretical properties such as well‑posedness, propagation‑of‑chaos, universal approximation, and provide quantitative rates under low‑dimensional measure‑dependence, while demonstrating the method’s accuracy and strong out‑of‑distribution generalization on various first‑ and second‑order dynamical systems.
By Liyao Lyu, Xinyue Yu, Hayden Schaeffer
PlacePulse is a structural geospatial foundation model that transforms 572 attributes—expanded to 2,288 through H3 neighborhood aggregates—into 256‑dimensional unit‑norm embeddings for over a million H3 resolution‑7 cells across the United States. The model achieves high intrinsic reconstruction quality (mean $R^2 = 0.853$) and strong spatial organization (Moran's $I = 0.93$), while outperforming a dimensionality‑matched baseline on 74 held‑out targets from 12 external sources and competing with Google’s Population Dynamics Foundation Model on various health, economic, election, mobility, and social‑vulnerability outcomes. These results demonstrate that static structural composition can serve as a distinct and complementary modality for geospatial foundation models.
By Luc\'ia Garc\'ia-Duarte, Miguel \'Alvarez, Diego Vicente, Javier P\'erez Trufero, Javier de la Torre
The paper introduces JEPA-kMMVAE, a neural network framework for predicting channel state information (CSI) in time‑varying multi‑antenna wireless channels using noisy historical observations. It combines a joint‑embedding predictive architecture with a k‑memory Markov variational autoencoder to generate temporally structured latent representations, enabling both direct prediction and residual corrections relative to the latest latent state and noisy CSI. Simulation results indicate that the residual configuration outperforms baseline methods at moderate and high input CSI signal‑to‑noise ratios, while sequence baselines remain more accurate under stronger noise.
By Efstathios Chatziloizos, Konstantinos Vandikas, Aneta Vulgarakis Feljan, Zheng Chen, Nikolaos Pappas
Vision2Code is a new benchmark for image-to-code generation that covers 15 domains and 539 test examples without relying on reference source code. It evaluates generated programs by executing them and comparing rendered outputs to the original images using domain‑specific rubrics, yielding scores that correlate better with human judgments than generic metrics. The study shows domain variability in model performance and demonstrates that the scoring method can improve fine‑tuning of models like Qwen3.5‑9B and Phi‑4‑MM via rejection sampling.
By Ajay Vikram Periasami, Junlin Wang, Bhuwan Dhingra