The paper proposes a sketch-based visual grounding framework that converts text embeddings into topographic sketches to enhance visualization, explainability, and information retrieval. It evaluates the method on the MS-COCO dataset, finding that while SBERT outperforms the sketch-based approach in top‑1 retrieval, the two methods achieve comparable top‑5 results, and a cascaded combination improves performance further. The study suggests that sketch-based representations could be valuable for retrieval tasks if trained directly on text.
Amplitude embedding lets a variational quantum model read many classical features with few qubits, but the model output is then only a quadratic form in the normalized input. Data re-uploading, which...
The paper presents a model‑centric evaluation framework for process discovery that uses node embeddings and Procrustes alignment to compare Petri net models across different datasets. By transforming Petri nets into graph representations, the method computes a bounded cosine‑distance metric that unifies fitness and precision assessments and remains robust to noise. Experiments on real and synthetic logs demonstrate that the embedding‑based indicator outperforms traditional conformance metrics and is sensitive to model degradation while providing activity‑level deviation localization.
The paper introduces the Foundation Event Predictor (FEP), a scalable in‑context architecture that performs next‑event prediction across diverse event logs using learned, event‑level nonparametric inference. FEP employs causal self‑attention to encode context and target histories, cross‑attention for relevance‑weighted aggregation, and permutation‑robust embedding learning to decouple event IDs from semantics. Trained on 696 real‑world datasets and evaluated on 50 unseen ones, FEP generalizes effectively to new systems without parameter adaptation, often matching or surpassing models trained separately on each target dataset.
The paper introduces a three‑stage generative AI pipeline to identify and fill semantic gaps in AI test datasets without requiring labeled data. First, vision‑language models zero‑shot label semantic factors and clustering highlights those most impacting model behavior. Second, diffusion‑based image‑to‑image models generate new images that preserve scene content while adding underrepresented semantics. Third, statistical checks confirm improved coverage before the synthetic images augment the test set. The method is validated on image classification and object detection tasks, showing that enriched test sets better differentiate models with or without exposure to missing semantics, leading to conclusions that align more closely with an ideally covered dataset.
By Connor Hashemi, Bharadwaj Ravichandran, Connor Anderson, Shitij Govil, Brandon RichardWebster, Brian Hu, Keith Fieldhouse
The paper evaluates unsupervised embedding geometries for patent claim classification, comparing hyperbolic and Euclidean Variational Autoencoders trained on 45,000 US patents. Hyperbolic embeddings retrieve cross-category prior art 52% more often than Euclidean embeddings (18.76% vs. 12.34% of top‑10 results across 3,000 queries). The study also links classification breadth and patents per subgroup to model performance, offering a practical signal for when the method is most useful.
By Sujith Subramanian, P N Bhargav Teja, Susen Sarkar, Rajasekhara Reddy Duvvuru Muni
The paper investigates anomaly detection in high-energy physics by learning low-dimensional representations of collision events. Using supervised and self-supervised neural embeddings on a large simulated dataset, the authors find that supervised objectives generally outperform self-supervised ones and improve signal efficiency compared to traditional kinematic-feature autoencoders. They also propose a local diagnostic pipeline that models the Standard Model latent space with a Gaussian Mixture Model, profiles components via high-level observables, and ranks these observables by their local discriminating power, thereby linking anomaly scores to specific kinematic features.
By Donatella Genovese, Philip Ploner, Simone Scardapane, Stefano Giagu, Thea Klaeboe Aarrestad
The paper presents iRAG, a human‑in‑the‑loop decision‑support system that blends Retrieval‑Augmented Generation with Hybrid Decision‑Making. Unlike traditional RAG models that rely on static knowledge bases, iRAG starts with an empty KB and incrementally builds it from human decisions, storing past decisions as retrievable precedents. The system progresses from silent observation to evidence‑grounded contestation and ultimately to autonomous handling with abstention, achieving low error rates even under knowledge drift and demonstrating a stability–plasticity trade‑off.
By Federico Mazzoni, Andrea Failla
The paper introduces REDaCT, an inference-time unlearning framework for large reasoning models that removes forget-set information from both the reasoning trace and the final answer without causing reasoning collapse. REDaCT uses a binary leak detector to spot sentences that leak forget-set data and an embedding optimization module that injects a corrective embedding at the first detected leak, steering subsequent reasoning toward non-leaking sentences. Experiments demonstrate that REDaCT preserves coherent chain-of-thought reasoning, outperforms existing unlearning methods, and achieves perfect answer forgetting with near-perfect chain-of-thought forgetting.
By Joshua Maccoby, Wendy Hui Wang
PRIME introduces an index-side optimization framework for structured document retrieval, comprising a Semantic Restructuring track that converts long documents into dense, structured sections and a Metadata Synthesis & Hierarchical Retrieval track that extracts semantic anchors and heading structures for efficient coarse-to-fine retrieval. The approach reduces embedding computation by 45.8× and index vector count by 22.4× while maintaining retrieval quality comparable to structure-aware chunking. In full deployment, Semantic Restructuring improves Answer‑F1 by 13.7%, and the study identifies qualitative regimes that influence when index-side optimization is beneficial or detrimental, noting that strong rerankers neutralize these effects.
By Cheuk Hei Chung, Li Liu
The paper introduces the Controllable Retrieval Model (CRM), a retrieval framework that incorporates watch time as a controllable condition in recommendation systems. CRM learns to predict the next item based on observed watch time during training and can steer retrieval at inference by supplying a desired watch time without retraining. Implemented as both a DNN two‑tower model and a Decision Transformer, CRM allows item embeddings to remain precomputed for efficient ANN search and has been deployed in Kuaishou’s short‑video recommendation system, yielding measurable improvements in video watch time.
By Chi Liu, Jiangxia Cao, Rui Huang, Kuo Cai, Weifeng Ding, Qiang Luo, Kun Gai, Guorui Zhou
The paper introduces a dictionary learning framework for probability measure-valued data by embedding measures into Hilbert spaces using features derived from optimal transport geometry. It learns data‑adapted dictionaries through sparse autoencoder parameterizations, yielding fast, scalable representations of probability measures as sparse finite‑dimensional codes. The authors demonstrate the method on hyperspectral images, geometric point clouds, and document embeddings from large language models, and provide generalization bounds and covering number results for optimal transport and entropic transport map dictionaries.
By Brendan Mallery, Shuchin Aeron, James Murphy
The study investigates shortcut learning in geospatial foundation model embeddings for cross-disaster damage assessment. By holding out entire disasters during evaluation, the authors demonstrate a significant drop in mean balanced accuracy across multiple datasets, revealing that models rely on event or scene characteristics rather than transferable evidence. They further show that simple feature-level transformations, such as supervised contrastive learning or PCA, can improve generalization to unseen disasters.
By Ali Shibli, Andrea Nascetti, Yifang Ban
The paper introduces zip2zip++, a method for inference‑time token compression that reduces token counts using a dynamic Lempel‑Ziv‑Welch (LZW) codec while preserving language model quality. It identifies four problems with prior LZW‑based compression—embedding conflict, improper RoPE application, harmful digit merges, and segmentation ambiguity—and proposes solutions: separate input/output embeddings, size‑aware RoPE, and a selective‑merge LZW codec. Experiments on four models show that these changes halve the quality gap in perplexity and reasoning accuracy compared to the uncompressed baseline, achieving similar compression ratios.
By Saibo Geng, Andrea Gentilini, Xinxian Ma, Nathan Ranchin, Yunzhen Yao, Robert West
The paper addresses the need for transparency and verifiability in AI systems used in critical fields like law. It critiques current LLM-based Retrieval Augmented Generation (RAG) methods for their black-box nature and susceptibility to hallucinations, especially as source volumes grow. By employing structure-driven parsing to build a Knowledge Graph for German legal code, the authors compare various KG-informed chunking and retrieval strategies to improve the accuracy of retrieved citations.
By Ibrahim Siddig, Sviatoslav Tugeev, Munir Georges
VaME (Variational Multimodal Embeddings) introduces a framework that treats latent reasoning as a learnable distribution over trajectories, enabling autoregressive exploration in latent space guided by answer reconstruction. By augmenting embedding-token readout with a latent-fused embedding and optimizing over stochastic variational trajectories via reinforcement learning with a Semantic Decoding Reward, VaME achieves superior performance on the MMEB-V2 benchmark and reasoning-intensive tasks like MRMR. The method also delivers at least a 4.25× inference speedup compared to deterministic latent autoregressive baselines.
By Peixi Wu, Mingzhou Jiang, Feipeng Ma, Biao Yang, Yunhao Zhou, Wei Yuan, Bosong Chai, Huizu Lin, Jie Chen, Zhangchi Hu, Fan Yang, Wenwu Ou, Hebei Li, Xiaoyan Sun
Kronecker Product Attention (KPA) is introduced as a memory‑efficient method for compressing the key‑value cache in autoregressive models. By representing key and value activations as sums of Kronecker products of compact, token‑dependent matrix factors, KPA generalizes Tensor Product Attention and can capture richer block‑structured patterns. Empirical results show that KPA matches or surpasses existing techniques such as MHA, MQA, GQA, and MLA across multiple model scales and benchmarks, achieving up to 10× compression while preserving relative‑position information through a structured rotary embedding.
By Bahareh Nikpour, Hossein Rajabzadeh, Maryam Dialameh, Vahid Partovi Nia, Parsa Omidi
The study examines how 105 language models choose a single word from a large set of valid options, finding that 46% of the time they all pick the same word, "serendipity." Using 96 prompts that ask for one-word answers across various categories, the researchers measure convergence and each model’s share of the consensus. They observe that post‑training and persona‑tuned models diverge more, while heavily post‑trained assistants are more conformist, and that the field’s answer choices are more concentrated than human norms in most categories.
By Tapan Parikh
STAPLE is a method that integrates static sparse memory into pretrained language models by adding token‑indexed embeddings as a residual pathway while preserving the original feed‑forward network. On the OLMo‑2 1B and 7B models, STAPLE outperforms dense training and other static sparse baselines, improving benchmark scores by up to 4.27 points and achieving faster decoding with less GPU memory and energy usage. The approach demonstrates that large pretrained models can be expanded efficiently without costly sparse pretraining from scratch.
By Ranajoy Sadhukhan, Euijin Hong, Sheng Cao, Zechun Liu, Shuowei Jin, Sha Li, Changlong Yu, Xin Liu, Qin Lu, Zhenyu Shi, Priyanka Nigam, Beidi Chen
The paper investigates two key design levers for Continuous Diffusion Language Models (CDLMs): self‑conditioning and embedding geometry. It shows that self‑conditioning mitigates a train/inference mismatch and can be implemented efficiently via a two‑level interpolant called DoubleCat, which significantly boosts TinyGSM accuracy. By adding discrete reveals to DoubleCat, the authors create SemiCat, achieving discrete‑diffusion‑level performance and outperforming discrete diffusion on several likelihood‑based benchmarks at 1.7B parameters. The study also finds that cross‑entropy training drives embeddings toward a geometry with decoding error rates comparable to one‑hot representations, and briefly explores additional options such as hybrid corruption processes and asynchronous noise schedules.
By Oscar Davis, Daan Roos, Alice Bizeul, Amitis Shidani, Victor Turrisi, Louis B\'ethune, Pierre Ablin, Marco Cuturi