The paper demonstrates that greedy decoding from large language models is not precision‑invariant: the same model, prompt, and decoding algorithm can produce different outputs when run in BF16 versus FP16 on identical hardware. Across six models (1.1B–7B parameters, four families, and 12B) and three benchmarks, 49–100 % of prompts diverge, with a single token flip often cascading into trajectory‑level divergence. The authors develop an empirical error‑propagation analysis that identifies the top‑two logit margin at the LM head as the key factor, and they propose a low‑overhead intervention—selective FP32 LM head recomputation—that improves exact agreement by 22–36 percentage points with less than 4 % latency overhead.
"whyItMatters":"The findings reveal that precision choices can fundamentally alter model outputs, challenging the assumption of deterministic greedy decoding and highlighting the need for precision‑aware inference strategies."
By Gaoyuan Du, Anam Nawaz Khan, Rex Zhou, Xiaoyang Liu, Deepayan Chakrabarti, Fnu Suya, Xueping Li
ProteinJEPA introduces a joint‑embedding predictive architecture that supplements masked language modeling (MLM) with a cosine loss to predict latent representations of a teacher model. On 19 protein tasks, MLM+JEPA outperforms compute‑matched and step‑matched MLM‑only training across 78 and 76 of 114 comparisons, achieving notable gains on structure‑ and homology‑sensitive tasks such as SCOPe‑40 retrieval and remote homology. Ablation studies show the cosine loss is superior to mean squared error and that latent prediction complements rather than replaces MLM.
By Dan Ofer, Dafna Shahaf, Michal Linial
The paper evaluates nine on‑device named‑entity recognition models ranging from classical taggers to large language models, measuring not only accuracy but also latency and output validity. Using a silver‑gold benchmark derived from an LLM judge panel and a human‑validated corpus, the study shows that encoder‑based models achieve comparable accuracy to a 4 B instruct LLM while being much smaller, faster, and producing no malformed output. Confidence calibration of GLiNER is analyzed, revealing over‑confidence but improved reliability after temperature scaling and thresholding.
By Vinay Kumar Chaganti
LILA (Latent-Informed Layer Analysis) introduces a calibration‑free method for structured pruning of large language models by scoring neuron importance using the Kolmogorov–Smirnov distance between singular value distributions of full and neuron‑ablated feed‑forward network weight matrices. The approach requires no training, calibration data, or auxiliary networks, and outperforms existing methods such as PruneNet and SliceGPT on LLaMA‑2‑7B and Phi‑2 at various sparsity levels. After a single epoch of LoRA fine‑tuning, LILA matches heavily calibrated baselines, and a Neural Tangent Kernel analysis provides theoretical support for its spectral importance criterion. Additionally, LILA can dynamically allocate sparsity budgets, achieving state‑of‑the‑art generative preservation and revealing architectural bottlenecks at higher compression.
By Sankar Behera, Dhruv Singh, Anshika Agnihotri, Raj Kumar Choudhary, Satyadev Ahlawat, Yamuna Prasad
The study investigates whether self‑supervised pretraining improves molecular graph neural networks by adapting the LeJEPA architecture to molecular graphs. While pretraining enhances learned representations and a frozen probe outperforms random initialization on tasks such as ogbg‑molhiv, it does not consistently boost finetuning performance across different data splits. Combining pretrained embeddings with 1024‑bit Morgan fingerprints yields modest gains, indicating that pretraining provides complementary information best exploited at the feature level.
By Micha{\l} Kulczykowski, Rafa{\l} {\L}ab\k{e}dzki
arXiv:2607. 15525v1 Announce Type: cross Abstract: Kolmogorov--Arnold Networks (KANs) replace fixed node activations with learned one-dimensional edge functions, offering an explicit interface for interpretation and a possible alternative to transformer feed-forward networks.
By Felippe Alves, Renato Vicente
arXiv:2605. 28149v2 Announce Type: replace Abstract: Sparse Autoencoders (SAEs) extract interpretable features from Large Language Model activations, but standard variants enforce non-negative latents, so a bidirectional semantic axis (e.
By Bartosz Wieciech, Zmnako Awrahman, Marcin Czelej, Victor Hugo Jaramillo Velasquez, Wioletta Stobieniecka
The paper introduces Murmur2Vec, a lightweight, alignment‑free embedding that uses k‑mer counts hashed with MurmurHash to create a compact representation for biological sequences. It provides a full theoretical analysis, including bias/variance formulas, a Johnson–Lindenstrauss‑style concentration bound, and an excess‑risk bound that clarifies the trade‑off between hash‑table size and classifier performance. Empirically, Murmur2Vec matches or surpasses a fine‑tuned 650M‑parameter ESM‑2 protein language model across several classification tasks, including SARS‑CoV‑2 spike lineage and HIV‑1 Env subtype identification.
By Sarwan Ali, Taslim Murad, Imdadullah Khan, Safi Faizullah
arXiv:2606. 08365v1 Announce Type: cross Abstract: Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features.
By Evan Duan
The paper proposes a flexible four‑coefficient parameterization for per‑token gating in on‑policy knowledge distillation, unifying existing methods such as EOPD and ToDi as special cases. Experiments on TweetEval with a Qwen3 teacher‑student pair show that the full family of gating configurations outperforms single‑channel baselines in most cells, and dynamic gating beats static baselines in a majority of isolated comparisons. The authors present the framework mainly as a shared coordinate system for comparing gating designs rather than as definitive statistical evidence.
By Suwan Wu, Yumeng Lin, Pengcheng Yuan, Xiaolong Jiang
The paper introduces Omega‑N, a set of ten interpretable node‑level structural descriptors derived from localizing four factors of a composite structural index. By correcting the ill‑conditioned localization with a configuration‑null excess and a multi‑scale personalized‑PageRank neighbourhood, Omega‑N achieves competitive or superior performance in six in‑domain node‑classification tasks compared to a recursive feature engine that uses up to 252 features. In drug‑target prioritisation on protein interaction networks, Omega‑N improves AUPRC by 0.073 to 0.144 over a centrality baseline and remains robust across independent datasets and bias controls, though it offers no benefit when combined with Node2Vec.
whyItMatters":"The study demonstrates that a compact, interpretable set of structural features can match or exceed more complex feature sets in practical graph‑based prediction tasks, particularly in biomedical network analysis."
By Alberto Acedo
The paper compares five machine unlearning (MU) methods—NegGrad, Fine‑Tuning (FT), Random Labeling (RL), SalUn, and MUNBa—on noisy‑label correction across CIFAR‑10, CIFAR‑100, and Food‑101N. Results show that the best MU strategy depends on the noise type: FT works well for most closed‑set noise, RL and SalUn are robust and nearly match retraining accuracy under instance‑dependent noise, while MUNBa excels only under extreme symmetric noise. In open‑set noise, retraining on the cleaned data actually hurts performance, indicating that approximating retraining is not suitable in that regime, yet all MU methods still achieve near‑retraining accuracy on Food‑101N with much lower runtime.
By Jo\~ao L. P. Santana, Filipe R. Cordeiro