Retrieval-augmented generation

Retrieval pipelines, vector search, chunking and reranking: how models are grounded in a corpus instead of their weights.

6,423 stories · RSS feed

arXiv Machine Learning
Aug 20

Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systems

The paper proposes a new dimension‑reduction strategy for transfer‑operator models of dynamical systems that focuses on scoring the σ‑algebra generated by coordinates rather than the operator’s spectral span. By using a χ²‑divergence criterion between embedded present and future states, the method guarantees that twice the intrinsic system dimension suffices to capture the full operator spectrum, even for systems with weakly interacting components that would otherwise require exponentially many modes. Experiments on benchmark systems show that this algebraic approach recovers masked components missed by rank‑based methods and enables accurate prediction of those components from few labels.

By Mark Kozdoba, Shie Mannor
arXiv AI
Aug 20

Forgetting, plasticity, and co-observation: a third facet of continual learning

The paper argues that catastrophic forgetting and loss of plasticity alone cannot explain why naive sequential training underperforms offline joint training. It introduces data co-observation as a third factor, showing that observing training data together consistently improves performance across supervised and self-supervised settings. The study also reinterprets common continual learning methods, suggesting that memory replay’s success stems from restoring co-observation benefits rather than merely mitigating forgetting.

By Timm Hess, Abhishek Jha, Gido M. van de Ven, Tinne Tuytelaars
arXiv AI
Aug 20

rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation

The paper introduces rEDMRec, a method that compresses a large language model’s reasoning about user preferences and item comparisons into a compact, editable memory. This memory, organized into four channels—long‑term preference, short‑term context, item perception, and counterfactual hard‑negative comparisons—can be updated by an LLM controller and queried by a lightweight student LLM for ranking, eliminating the need to re‑run the expensive teacher model for each request. Experiments on ML‑1M, Amazon Beauty, and Steam datasets show that rEDMRec consistently outperforms zero‑shot, few‑shot, RAG, and GraphRAG baselines, achieving up to a 13.3% improvement in HR@1 on ML‑1M.

By Minh Hoang Nguyen, Tung Le, Huy Tien Nguyen
arXiv AI
Aug 20

PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

PGFS++ is a synthesis‑aware reinforcement learning framework that improves molecular properties while ensuring the resulting molecules can be synthesized and remain structurally similar to the input. It builds on PGFS+ by using trainable embedding lookup tables for reaction templates and second reactants, a more effective scoring function, and a refined RL algorithm. Experiments demonstrate that PGFS++ enhances target properties and preserves high output diversity, overcoming the reward‑hacking failure mode seen in earlier versions.

By Boqiao Zhang, Godbless James, Sai Krishna Gottipati, Andrew Fitzgibbon
arXiv Machine Learning
Aug 20

Coordination on a Budget: Federated Active Learning with Few Labels

The paper introduces a federated active learning (FAL) approach that tackles data privacy and label scarcity by coordinating query selection across clients. In low-budget scenarios, it finds that homogeneous (IID) data actually requires stronger coordination to avoid redundant queries, while heterogeneous data naturally yields diversity—a reversal of the usual federated learning narrative. The authors propose a new framework that aligns client data in a shared embedding space via federated representation learning, enabling globally coordinated active selection while keeping annotations local, and demonstrate that this method outperforms existing FAL methods even with larger annotation budgets.

By Liam Mohr, Daphna Weinshall
arXiv AI
Aug 20

Syntactic Simplification of OWL Class Expressions

The paper introduces Class Expression Simplifier (CES), an algorithm that syntactically simplifies OWL class expressions in Description Logics. CES preserves formal semantics by applying rewriting rules to remove redundancies and produce simpler, equivalent expressions, resulting in more compact and human-readable representations. Experiments on two medium-sized ontologies show that CES improves reasoning efficiency and reduces verbosity, and the tool is available as an open-source Python package within OWLAPY.

By Alkid Baci, N'Dah Jean Kouagou, Caglar Demir, Axel-Cyrille Ngonga Ngomo
arXiv Machine Learning
Aug 20

Monroe: A Molecular Foundation Model for In-Context Probabilistic Inference

Monroe is a new molecular foundation model that improves upon existing models by pre‑training on over 81 million molecules from the PM6 quantum chemistry dataset, enhancing stereochemistry representation, and introducing novel training losses such as conformer denoising and embedding decorrelation. It also incorporates a prior‑data‑fitted model (TabPFN) for downstream in‑context prediction and demonstrates superior performance on Polaris benchmarks and activity cliff tests. Ablation studies show that the PFN‑based downstream approach can upgrade other models, producing state‑of‑the‑art variants MiniMol_PFN and CheMeleon_PFN.

By Blazej Banaszewski, Andrew W. Fitzgibbon
arXiv Machine Learning
Aug 20

Fair Multi-View Determinantal Coresets via Adaptive NEPv

The paper introduces a method for selecting a small, diverse subset from a large pool by addressing multiple, potentially conflicting notions of diversity. It formulates a fair multi‑view determinant selection problem that maximizes the weakest per‑view log determinant of a size‑k subset, smooths and relaxes the objective to the Stiefel manifold, and derives an adaptive self‑consistent‑field solver with damping and level shifting. The solver operates using only feature‑map products for each view and includes a rounding step via leverage‑score screening followed by fair local refinement.

By Richard Yi Da Xu
arXiv AI
Aug 20

Composed Historical Image Retrieval by Modeling Temporal Representations

The paper introduces Temporally Decomposable Image Representations (TDIR), a learning algorithm that separates historical photographs into distinct date and content components using orthogonal subspaces. It establishes theoretical conditions for this decomposition, quantifies errors when conditions are partially met, and demonstrates that temporal and categorical subspaces naturally become orthogonal during joint optimization. TDIR supports transitive operations, allowing temporal information from one image to be transferred to another without label supervision, and is validated on composed image retrieval tasks that require both object content and time period specification.

By Adri\`a Molina Rodr\'iguez, Oriol Ramos Terrades, Josep Llad\'os Canet
arXiv Machine Learning
Aug 20

Enhancing Distance-Based Graph Autoencoders with Structural Penalties for Dynamic Graph Embedding

The paper introduces three distance‑based graph autoencoder variants that add structural penalties to the reconstruction loss. All models use a two‑layer Graph Convolutional Network encoder and a Euclidean‑distance decoder, with two node‑level regularizers: a hub penalty based on degree centrality and a penalty based on Natural Community Local Intrinsic Dimensionality (NC‑LID). Experiments on multiple dynamic graph datasets show that incorporating NC‑LID regularization consistently improves reconstruction performance compared to baselines without structural regularization and to the hub‑aware variant.

By Aleksandar Tom\v{c}i\'c, Milo\v{s} Savi\'c, Milo\v{s} Radovanovi\'c
arXiv AI
Aug 20

Nine Emotion Centroids: A Label-Free Valence Axis That Transfers Across Four Modalities

The paper demonstrates that a single internal direction in modern language models—called the valence axis (V-axis)—captures how positive or negative a sentence feels. By using only nine emotion category names and 50 short narrative paragraphs per emotion, the authors identify this axis via principal component analysis of frozen encoder embeddings, achieving 93% of supervised performance on SST‑2 and strong correlations with human valence ratings across images, audio, and brain recordings. The method transfers across modalities without target‑modality labels, but works only for continuous attributes and is specific to certain model families.

By Yousef Radwan
arXiv AI
Aug 20

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

The paper investigates the vulnerability of Gaussian‑noised text embeddings to inversion attacks. It identifies a "Double Noise Trap" that hampers standard generative methods and introduces DAEI, a denoising‑aware pipeline that significantly outperforms existing baselines in reconstructing original text. Experiments show DAEI improves BLEU by 154% and token‑level metrics by 32–60%.

By Yubo Wang, Shujie Cui, James Bailey, Hongzhi Yin, Wenyu Liang, Min Tang, Shiyue Qin, Weiqing Wang
arXiv Machine Learning
Aug 20

GraphK: Variable-Size Graph Generation with Efficient Edge Construction

GraphK introduces an encoder‑sampler‑decoder framework that generates variable‑size graphs efficiently. It learns permutation‑invariant latent representations and samples new node embeddings via maximum likelihood, enabling both upscaling and downscaling of graph size. Edge construction uses KDTree‑based top‑k neighbor search in latent space, reducing computational cost while capturing graph properties.

By Resul Tugay, Eren Olu\u{g}, Elif Ak, Sule Gunduz Oguducu
Hugging Face Trending Papers
Aug 20

Enforcing LLM Safety through DMD-based Classification of Prompt-Response Embedding Dynamics

The paper extends a dynamical systems approach to classify unsafe outputs from large language models (LLMs) by projecting prompts and responses into high‑dimensional embeddings and fitting separate Koopman-based predictive models for safe and unsafe regimes. A differential residual score compares prediction errors from these models to classify new outputs. Experiments on three safety benchmarks show that including prompt embeddings improves detection of interaction‑dependent violations, especially with causal decoders like Llama‑3, while response‑only violations benefit more from dense semantic embeddings.

Hugging Face Trending Papers
Aug 20

CVSD-Reg: Cross-Modal Visual Semantic Prior Distillation for Robust LiDAR Registration

Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations.

Hugging Face Trending Papers
Aug 19

Comment-level Topic Drift Analysis in the Reddit Corpus

The paper introduces a new method that uses embedding-based dynamic topic modeling to detect and measure topic drift at the comment level in a large dataset. By applying pretrained language models to generate contextualized embeddings for 12.7 billion Reddit comments from 2006 to 2022, the authors identify evolving topic clusters over time using unsupervised techniques. Their approach includes scalable modifications to existing methods and a null model comparison test, revealing that politically and socially contentious topics show significant directional drift, while areas like music and sports remain relatively stable.

Hugging Face Trending Papers
Aug 19

PGFS++: Molecular Property Improvement under Synthesis and Diversity Constraints

PGFS++ is a synthesis‑aware reinforcement learning framework that improves molecular properties such as drug‑likeness or binding affinity while ensuring the resulting molecules can be synthesized and remain structurally similar to the input. It builds on PGFS+ by using trainable embedding lookup tables for reaction templates and second reactants, a more effective scoring function, and a refined RL algorithm. The method addresses a reward‑hacking failure mode by treating each input molecule as the start of a forward‑synthesis trajectory, applying learned reaction templates with in‑stock building blocks, and producing diverse, high‑quality outputs with explicit synthesis routes.

Towards Data Science
Aug 19

Kimi K3’s 1M Token Context Window vs. RAG: Cost, Latency and Answer Quality

The article presents a controlled comparison between a top‑5 Retrieval‑Augmented Generation (RAG) pipeline and a single 127,000‑token prompt using the same 12 questions, system prompt, and model. Both approaches were graded blind on correctness, completeness, and grounding. The study evaluates cost, latency, and answer quality for each method.

By Sarah Schürch
Hugging Face Trending Papers
Aug 19

Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systems

The paper addresses the limitations of spectral dimension reduction for dynamical systems composed of weakly interacting components, where standard rank‑based methods either require exponentially many modes or omit entire components (a phenomenon termed linear masking). It proposes scoring the σ‑algebra generated by coordinates instead of individual modes, using a χ²‑divergence criterion that guarantees an embedding with twice the intrinsic dimension captures the full operator spectrum. Experiments on benchmark systems show that this algebraic approach recovers masked components and enables accurate prediction from few labels, outperforming traditional rank‑based and VAMP methods.