The paper presents a new analysis of Oja's algorithm for streaming principal component analysis (PCA) that works without any eigengap assumptions, achieving near‑optimal rates and matching lower bounds. It extends the results to a Rayleigh quotient notion of approximate PCA, resolving an open question, and applies the findings to provide gap‑free differentially private PCA guarantees for sub‑Gaussian data. The analysis relies solely on a second‑moment bound of stochastic updates, avoiding the almost‑sure bounds used in previous work.
By Anming Gu, Syamantak Kumar, Kevin Tian, Chutong Yang
The paper introduces the Proximity Trap, a problem where long-context language models fail to attend to distant evidence because of competition from abundant, irrelevant proximal background. To mitigate this, the authors propose LYRA, a t‑distributed directional matching mechanism that reshapes the retrieval distribution to focus more on task‑relevant evidence while preserving positional information. Experiments on LongBench‑v2, RULER, and LongBench show consistent performance gains, and a new benchmark, ProxBench, is presented to evaluate distant evidence usage under varying proximal interference.
By Xiaoyu Yang, Jie Lu, Wei Duan, En Yu
The paper investigates how different 4‑bit quantization techniques affect privacy when fine‑tuned small language models are deployed. It finds that methods using a calibration corpus, such as Activation‑aware Weight Quantization (AWQ) and Gradient‑based Post‑Training Quantization (GPTQ), prevent the reproduction of planted private records, whereas a calibration‑free format (GGUF Q4_K_M) leaks 5.3% of them. Across models ranging from 0.5 to 7 billion parameters, AWQ consistently leaks the least while maintaining minimal accuracy loss, indicating that the choice of 4‑bit method is a privacy decision as well as a performance one.
By Cristhian Kapelinski, Diego Kreutz
SSP-Bench is a dynamic benchmarking framework designed to evaluate large language models on safety, security, and privacy (SSP) by generating evaluation instances on demand while maintaining domain consistency. It ensures label validity through external sources, enforces scope with service-specific validation, and calibrates difficulty using a multi-model steering panel, framing benchmark construction as a multi-objective optimization over difficulty, separability, novelty, and diversity. Across 24 models and four SSP services, SSP-Bench exposes systematic failures of static benchmarks, such as near-zero correlation in safety rankings, strong safety–over-refusal coupling, and hidden within-family regressions.
By Fatih Deniz, Yazan Boshmaf, Issa Khalil
Matryoshka Attribution (MAttr) is a mask‑learning method that identifies nested subsets of a language model’s internal components by minimizing downstream loss. It uses a differentiable sigmoid top‑k operator and randomizes sparsity during training to produce an attribution ordering of components. MAttr tops the Mechanistic Interpretability Benchmark leaderboard and can be applied via reinforcement learning to pinpoint weight changes that control behaviors such as refusal in Llama 3.1 8B Instruct, where restoring just 1% of weights removes refusals while preserving capabilities.
By Aryaman Arora, Kirill Acharya, Nathan Hu, Yanzhe Zhang, Noah Goodman, Dan Jurafsky, Christopher Potts
The paper studies federated learning where honest clients have heterogeneous data-generating models and adversarial clients can exacerbate this heterogeneity by sending arbitrary updates. It derives new bounds on gradient heterogeneity for linear and nonlinear regression, separating effects from honest clients’ model differences, label noise, and initialization. The authors show that for any (f,κ)-robust aggregator with κ = O(f/n) (where f is the number of adversarial clients and n the total number of clients, with f/n < 1/2), convergence is guaranteed after an explicit sample burn‑in period.
By Leonardo F. Toso, James Anderson, Nirupam Gupta, Rafael Pinot
The paper introduces a lightweight evolutionary heuristic scheduler for Diffusion Large Language Models (dLLMs) that optimizes denoising trajectories using CMA-ES. It addresses failure modes of existing confidence-based schedulers—EOS Overflow and Proximal Bias—by integrating multiple heuristic features with a contextual mean-field embedding, requiring only 393 trainable parameters. Evaluations on LLaDA and Dream across reasoning and planning benchmarks show consistent outperformance over strong baselines and recent state‑of‑the‑art methods.
By Zijian Zhao, Dian Jin, Xialiang Tong, Sen Li, Mingxuan Yuan
The paper introduces MGRL-RSCC, a multi‑granularity reward reinforcement learning framework for Remote Sensing Change Captioning (RSCC). It uses a CNN with hierarchical self‑attention to extract visual features, a Transformer decoder for visual‑to‑linguistic translation, and a dual‑decoding strategy combined with token‑level supervised learning and self‑critical reinforcement learning. Three reward functions—linguistic fluency, change state consistency, and structural‑semantic relevance—are employed to improve caption quality and reduce exposure bias and conservative generation.
By Futian Wang, Mengqi Wang, Xiao Wang, Wentao Wu, Haowen Wang, Zhicheng Zhao, Jin Tang
arXiv:2609. 26290v1 Announce Type: cross Abstract: Causal tabular foundation models amortize effect estimation across synthetic mechanisms, but latent-effect supervision rewards posterior shrinkage instead of directly encoding the repeated-sample response needed in a fixed deployment population.
By Zhiheng Zhang
HYDRA is a proactive Android malware drift adaptation framework that learns drift‑invariant representations from hierarchically structured data. It combines fine‑grained Control Flow Graphs and coarse‑grained Function Call Graphs to model applications, then applies a cross‑domain contrastive learning objective to align historical and new data distributions. Experiments on large‑scale, time‑ordered malware datasets show HYDRA achieves lower false negative and false positive rates than state‑of‑the‑art baselines while needing up to 87.5% fewer labeled samples.
By Han Chen, Hanchen Wang, Hongmei Chen, Lu Qin, Wenjie Zhang, Ying Zhang
The paper investigates preference elicitation under the Bradley‑Terry‑Luce model, focusing on estimating an unknown partworth vector from pairwise queries that satisfy a joint identifiability condition. It derives minimax lower bounds and shows that the canonical maximum likelihood estimator (MLE) exists, is unique, and achieves near‑optimal error rates once the sample size exceeds a design‑dependent threshold, without requiring compactness constraints or external regularizers. The analysis decomposes the estimation error into a linear stochastic term, a second‑order bias, and a higher‑order remainder, providing a unified non‑asymptotic theory for parametric utility elicitation.
By Yicheng Li, Huifu Xu
The paper introduces Gated Token Recurrence (GTR), a softmax‑free recurrent vision backbone that replaces global softmax attention with gated linear attention, alternating scan directions, and enhanced SwiGLU blocks. GTR is distilled from a DINOv3 teacher using only final‑layer patch‑token alignment, and achieves strong performance on COCO object detection (58.9 box AP) with very low latency (1.908 ms on an RTX 4090). The backbone also transfers to multiple dense prediction tasks and runs efficiently on edge hardware via a specialized CUDA operator and TensorRT deployment.
By Zhe Feng, Longfei Liu, Wei Liu, Kai Chen, Jiangjiang Kong, Wei Zhou, Yifeng Qian, Dexiong Chen, Xuanlong Yu, Xi Shen
MMAP is a Multimodal Missing‑Aware Alignment Pretraining method designed to learn image‑tabular representations from incomplete data. It uses a sigmoid contrastive learning image encoder with generative reconstruction, a tabular encoder based on a foundation model, and a missing token generator to handle missing modalities. The approach is evaluated on longitudinal Alzheimer’s tasks—predicting disease stage conversion and amyloid status—and outperforms both multimodal and unimodal baselines.
By Fiona Kekwick, Matthew Baugh, Bernhard Kainz, Paul M. Matthews, Wenjia Bai
FedNIA is a defense framework for federated learning that identifies and excludes malicious clients without needing a central test dataset. It works by injecting random noise inputs and analyzing layerwise activation patterns with an autoencoder to detect abnormal behaviors caused by data poisoning. The method can counter various attack types—including sample poisoning, label flipping, and backdoors—even when multiple attackers collaborate, and shows strong performance on non‑iid federated datasets.
By Ehsan Hallaji, Roozbeh Razavi-Far, Mehrdad Saif
PRISM‑FCP is a federated conformal prediction framework that achieves Byzantine robustness while reducing communication costs. It does so by partially sharing model updates—transmitting only a subset of parameters per round—to dampen the influence of poisoned clients during training, and by filtering out suspected Byzantine clients during calibration using histogram‑based techniques. Experiments on synthetic data and UCI datasets show that PRISM‑FCP maintains near‑nominal coverage and offers favorable trade‑offs between communication overhead and predictive performance.
By Ehsan Lari, Reza Arablouei, Stefan Werner
arXiv:2410. 14788v4 Announce Type: replace-cross Abstract: Neural operator (NO) architectures learn nonlinear maps between infinite-dimensional function spaces and are widely used to accelerate simulation and enable data-driven model discovery.
By Takashi Furuya, Anastasis Kratsios
FMMD is a multimodal, multidisciplinary dataset of open peer reviews from F1000Research that pairs manuscript-level visual and structural data with version‑specific reviewer reports and editorial decisions. It addresses key gaps in existing datasets by preserving precise alignment between review comments and the exact manuscript version, and by including a wide range of scientific disciplines beyond computer science. The dataset supports tasks such as visual‑semantic consistency classification, figure‑related review comment generation, and editorial decision prediction, providing a comprehensive empirical resource for multimodal automated scholarly paper review research.
By Zhenzhen Zhuang, Yuqing Fu, Jing Zhu, Zhangping Zhou, Jialiang Lin
The paper introduces eXplaining to Learn (eX2L), an interpretable framework that regularizes a classifier by penalizing similarity between Grad‑CAM maps of the main label classifier and a confounder classifier. This approach decorrelates confounding features from latent representations during training. On the Spawrious Many‑to‑Many Hard Challenge benchmark, eX2L outperforms the current state‑of‑the‑art by 5.49% in average accuracy and 10.90% in worst‑group accuracy, while also demonstrating functional domain invariance through explicit label‑nuisance decoupling.
By Paulo Mario P. Medina, Jose Marie Antonio Mi\~noza, Sebastian C. Iba\~nez
The paper investigates how unbiased stochastic gradients affect stability in one‑layer linear self‑attention models used for in‑context linear regression. It shows that while full‑batch training preserves an elliptic stability region, mini‑batch sampling introduces batch‑dependent boundaries that drift outward, leading to an exact one‑step crossing criterion and adaptive concentration bounds. The authors also derive drift identities, prove almost‑sure escape in a balanced switching model, and analyze mode leakage and effective context length for softmax attention, corroborated by experiments on causal Transformers.
By Krishnakumar Balasubramanian
The paper introduces a four‑stage framework—SFT, PG‑CoT, Dynamic, and K‑RL—to improve large language models for Traditional Chinese Medicine prescription generation. It addresses three key gaps: lack of auditable reasoning (SR Gap), failure to adjust prescriptions over time (LA Gap), and non‑enforcement of absolute contraindication rules (SC Gap). Experiments on 12 fine‑tuned models and 6 zero‑shot baselines show that the framework, particularly a 7B Mistral model, outperforms zero‑shot GPT‑5 on all three TCM evaluation metrics.
By Zheng Chen, ZhiCheng Du, Haoxuan Li, Peiwu Qin