The paper introduces chained linear minimization oracles (chained LMOs) as a unified framework for optimizers that compose multiple matrix normalizations, addressing fragmentation in the field. It shows that while many such chains can diverge on smooth convex objectives, chaining can still improve performance under anisotropic embeddings, as demonstrated by the new TensorChain optimizer. TensorChain, which stacks compatible weight matrices across layers and normalizes a 3‑D tensor, achieves a 9.6% token efficiency gain over Muon in Qwen3 0.6B and 1.7B pretraining at matched validation loss.
By Sungyoon Kim, Kaan Ozkara, Youngsuk Park
The paper presents a detailed study of feature representations in Vision Transformers (ViTs), revealing that feature collapse at initialization causes redundancy and proposing a reduction scheme to address it. It demonstrates that entropy and minimum eigenvalue metrics reliably predict generalization, and that token‑space features and linear submodule outputs are particularly informative. The authors develop a proxy that improves ranking of ViT architectures by 18‑48% over previous baselines, enabling the selection of models with higher accuracy at comparable or lower computational cost.
By Kim-Cuc Nguyen, Ngai-Man Cheung
arXiv:2610. 11976v1 Announce Type: cross Abstract: We detail scalable methods for approximating the quadratic entropy $p^T d p$ for arbitrary distributions $p$ and common distances $d$ of negative type.
By Steve Huntsman
PaReGTA is a large‑language‑model based framework that transforms longitudinal electronic health record events into visit‑level templated text with explicit temporal cues, learns visit embeddings through lightweight contrastive fine‑tuning, and aggregates them into fixed‑dimensional patient representations using hybrid temporal pooling. These representations can be fed into conventional machine‑learning models. In a cohort of 39,088 migraine patients from the All of Us Research Program, PaReGTA combined with LightGBM achieved the highest AUC, accuracy, and F1 scores on a held‑out test set compared to sparse, BERT‑based, and recurrent approaches.
By Kihyuk Yoon, Lingchao Mao, Catherine Chong, Todd J. Schwedt, Chia-Chun Chiang, Jing Li
Large language models can reveal memorized training sequences when prompted with a prefix, but real-world systems often use additional context such as instructions or retrieved documents. This study measures how such contextual conditioning affects memorization by comparing suffix probabilities with empty prompts versus retrieved contexts across three instruction-tuned models. The results show that context does not simply erase memorization; instead, it preserves a core set of robustly extractable samples while also shifting the boundary of marginal cases, sometimes suppressing and sometimes enabling new exposures.
By Ali Satvaty, Narjes Sharafi, Jirui Qi, Suzan Verberne, Fatih Turkmen
RFChipAgent is a novel multi‑agent flow that uses large language model agents to automate end‑to‑end analog/RF circuit design under human supervision. The system comprises four key components: a retrieval‑augmented generation subsystem that pulls design knowledge from documentation, agents that handle topology selection and schematic/testbench assembly, a hybrid sizing engine that blends TPE and CMA‑ES optimization with simulator feedback, and a trust‑scored simulation database that informs future optimization runs. Validation on GF22FDSOI 60 GHz wideband mm‑wave low‑noise amplifiers demonstrates automated topology generation, specification‑driven exploration, and simulator‑guided optimization, yielding significant reductions in design effort while preserving sign‑off quality.
By Awani Khodkumbhe, Yunfei Feng, Raj Rangarajan, Kevin Wang, Kamal Sahota
The paper surveys how large language models (LLMs) are integrated into software systems under various labels such as chatbot, copilot, retrieval‑augmented generation (RAG), workflow, coding agent, and AI agent. It identifies seven recurring architectural forms—LLM chats, custom agents, RAG, AI‑enhanced workflows, copilots, coding agents, and agentic RAG—and describes each along four structural dimensions: architectural pattern, execution control, user intervention point, and tool usage. The study uses a corpus of 22 systems from research and vendor documentation to illustrate these forms and their limits.
By Irene Weber (University of Applied Sciences Kempten, Germany)
The paper investigates how much original audio information remains in frozen representations from various pretrained audio encoders by attempting to reconstruct five‑second, 44.1‑kHz stereo music clips. Using a shared Stable Audio Open latent diffusion decoder, the authors reconstruct audio from embeddings produced by supervised classifiers (VGGish, ConvNeXt), an audio‑text contrastive model (CLAP), and a waveform‑reconstruction model (EnCodec). Results on the Million Song Dataset reveal that reconstructability varies across encoder families and that embeddings exposing finer temporal or spectral detail enable better recovery of source content, even for compressed, task‑oriented embeddings.
By Marios Glytsos, Brian McFee
APEX is a new image quality assessment framework that eliminates assumptions by using the Sliced Wasserstein Distance as a similarity measure. It is embedding‑agnostic, scalable to high‑dimensional spaces, and employs open‑vocabulary models CLIP and DINOv2 for feature extraction. Benchmarks show that APEX is more robust to visual degradations and remains stable across different datasets, including out‑of‑domain ones.
By Caterina Gallegati, Monica Bianchini, Franco Scarselli, Vittorio Murino, Barbara Toniella Corradini
The paper introduces D‑SLR, a closed‑form decomposition that combines a low‑rank approximation with a disjoint row‑sparse component, offering a drop‑in replacement for truncated SVD. Unlike previous joint methods, D‑SLR restricts each row to either be stored verbatim or approximated by the low‑rank fit, achieving the joint optimum without extra parameters and without iterative solvers or regularization tuning. The authors provide an assumption‑free, a‑posteriori error bound for every shape, and demonstrate performance gains on synthetic data, LLM embedding tables, network traffic, and hyperspectral images.
By Vincent Szolnoky
CoPoE is a new multimodal fusion framework for Alzheimer’s disease diagnosis that maps diverse clinical, imaging, genomic, and biomarker data into a structured latent space divided into four biological and clinical axes: genetic Risk, molecular Pathology, Neurodegeneration, and clinical Stage. Each available modality contributes a diagonal Gaussian expert, and a masked Product-of-Experts architecture fuses only the observed modalities, avoiding synthetic imputation. In extensive experiments on the ADNI dataset, CoPoE outperforms standard missing-modality fusion baselines, achieving higher AUROC, better calibration metrics, and retaining pathology signals even when certain modalities are missing.
By Chihun An, Ikbeom Jang
The paper introduces MC-TRCM, a Modality‑Conditioned Temporal Recursive Context Model that treats each feature source from mobile and wearable mental‑health datasets as a separate token and explicitly models missingness. By encoding observed values, summarizing missingness, and using learned absence tokens, the model refines predictions recursively and is conditioned on dataset and task embeddings. Evaluated on six endpoints from DepreST‑CAT and PSYCHE‑D, MC‑TRCM outperforms tabular baselines on PHQ‑9 and GAD‑7 severity prediction and matches or slightly exceeds reference performance on classification tasks, with ablations confirming the importance of feature‑wise modulation, absence tokens, missingness projections, and recursive refinement.
By Wentao Wang, Lifeng Han, Zining Ren, Hengyu Zhong, Guangyu Zou
Puffin is a probabilistic framework that disaggregates coarse spatial totals into high-resolution estimates by leveraging satellite embeddings as covariates. Unlike deterministic methods, it learns a probability distribution for each fine-scale subregion and trains using an aggregation-aware likelihood, ensuring that predictions sum to the observed regional totals. The approach provides calibrated uncertainty and does not require fine-resolution labels, and it has been evaluated on German and US census, employment, and election data.
By Chaitanya Jobanputra, Sebastian Vollmer, Gerrit Gro{\ss}mann
LIME is a new recommendation system architecture that tackles the high computational cost of transformers by introducing low‑rank link embeddings and a linear attention mechanism called LIME‑XOR. These innovations decouple user and candidate interactions, allowing pre‑computation of attention weights and reducing inference complexity to be nearly independent of candidate set size and linear in user sequence length. Experiments on public and industrial datasets demonstrate that LIME matches transformer performance while achieving a tenfold speedup on large candidate sets or long sequences, and it has been shown to improve user engagement on a major recommendation platform.
By Yang Liu, Yunjiang Jiang, Ayush Agarwal, Yihan Wu, Haoran Liu, Bi Xue
The paper introduces Local Prototype Reconstruction (LPR), a lightweight regularizer that encourages bridge embeddings in speech‑to‑LLM systems to stay close to the frozen LLM’s input‑embedding neighbourhoods. By measuring lexical manifold compatibility with a fixed, head‑free, timestamp‑free diagnostic, the authors show that next‑word prediction and sentence‑level contrastive pretraining do not fully capture token‑level lexical compatibility. Experiments on multilingual ASR and speech translation demonstrate that LPR improves transfer, especially for translation and low‑resource adaptation, and that the diagnostic correlates with downstream gains across objectives.
By Xinnian Zhao, Chia-Hua Wu, Pu Wang, Hugo Van Hamme
SignRAG is a unified framework for gloss‑free sign language translation that combines hierarchical pretraining, target‑domain retrieval augmentation, and retrieval‑aware reinforcement fine‑tuning. It first learns linguistically grounded sign representations, aligns a sign encoder with a decoder‑only large language model, and then uses a retrieval gallery to provide instance‑specific translation cues. Retrieval Utility‑Guided Reinforcement Fine‑Tuning (RUG‑RFT) balances translation quality and retrieval usefulness, leading to state‑of‑the‑art results, including the first gloss‑free approach to outperform gloss‑supervised methods on CSL‑Daily.
By Zhi Rao, Yucheng Zhou, Qianran Sun, Yiqing Huang, Longcan Yuan, Jiayi Hou, Chengwen Yao, Lin Cheng, Donghui Sun, Xiaoxin Chen, Jun Wan
The paper introduces the AutoRegressive Retriever (ARR), a multimodal retrieval model that iteratively refines a query embedding by selecting informative items and incorporating their content. ARR alternates between retrieval and query update, using stepwise contrastive supervision and reinforcement learning to optimize item selection and final ranking. Experiments show ARR outperforms baselines on in‑domain and zero‑shot benchmarks, with feedback improving both inference performance and initial query embeddings.
By Jianfei Zhao, Yifan Wang, Feng Zhang, Xin Sun, Chong Feng, Zhixing Tan, Yang Luo, Boyuan Pan, Xu Kai, Yao Hu
The paper evaluates how vision-language models (VLMs) perform in image classification tasks, comparing language-guided and vision-only approaches across various dual-encoder VLMs such as SigLIP 2 and RADIOv2.5 on ImageNet-1k and its label-corrected variant. It identifies key factors influencing accuracy—including prompt design, class diversity, k‑NN neighbors, and reference set size—and demonstrates that language and vision provide complementary strengths. A simple, learning‑free fusion method based on per‑class precision is proposed to further improve classification performance.
By Illia Volkov, Nikita Kisel, Klara Janouskova, Jiri Matas
The paper introduces MatLat, a material latent space designed for generating high‑quality physically‑based rendering (PBR) textures on 3D meshes. It fine‑tunes a pretrained VAE to incorporate new material channels while preserving latent distribution, and adds a locality‑preserving regularization that aligns cropped latent patches with image regions. Experiments and ablation studies show that these components together improve PBR texture fidelity and outperform prior baselines.
By Kyeongmin Yeo, Yunhong Min, Jaihoon Kim, Minhyuk Sung
RIT‑RAG is a retrieval‑augmented generation framework that combines content retrieval with structural navigation of documents. It builds a tree for each document from its table of contents or sitemap, then uses retrieved chunk positions to induce manageable sub‑trees that an LLM agent can navigate, read selectively, and reformulate queries. Experiments on financial, scientific, and customer‑support benchmarks show RIT‑RAG outperforms vanilla, graph‑based, and agentic baselines, achieving up to 11.4‑point accuracy gains on the new EntQABench of 2.84 million technical‑documentation webpages.
By Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu