arXiv Machine Learning
Jun 30
Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders
arXiv:2606. 29888v1 Announce Type: new Abstract: Vision-language models map images and text into a joint embedding space.
Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders
arXiv:2607. 00023v1 Announce Type: cross Abstract: Dense sentence embeddings are fundamental to modern Retrieval-Augmented Generation (RAG) systems but suffer from a lack of interpretability due to feature superposition.
Related stories
arXiv Machine Learning
Sep 10
Disentangling Steering Vectors
arXiv:2609.07037v1 Announce Type: new Abstract: Activation steering has emerged as a lightweight, inference-time approach to control the behavior of Large Language Models (LLMs). However, traditional...
arXiv Computation and Language
Sep 24
EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering
arXiv:2603.09222v2 Announce Type: replace Abstract: Efficient context compression is critical for retrieval-augmented question answering in resource-constrained settings, where long retrieved context...
arXiv AI
Jun 18
Rescaling MLM-Head for Neural Sparse Retrieval
arXiv:2606. 18811v1 Announce Type: cross Abstract: Learned sparse retrieval (LSR) models such as SPLADE have traditionally used BERT-style masked language models as backbone encoders.
arXiv Machine Learning
Jul 28
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
arXiv:2607. 23052v1 Announce Type: cross Abstract: Dual-encoder vision-language models (VLMs) expose a similarity interface that enables zero-shot retrieval but fails compositional constraints: queries like "umbrella and no person" retrieve images containing both, even when concept detection is reliable.
arXiv Machine Learning
Sep 22
Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation
arXiv:2609.22162v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves the factuality of large language models (LLMs) and vision-language models (VLMs) by grounding generatio...
arXiv AI
Jul 24
Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit
arXiv:2512. 10092v2 Announce Type: replace Abstract: Analyzing large-scale text corpora is a core challenge in machine learning, crucial for tasks like identifying undesirable model behaviors or biases in training data.
arXiv Machine Learning
1d ago
Retrieval from Within: An Intrinsic Capability of Attention-Based Models
The paper introduces INTRA, an attention-based encoder-decoder framework that retrieves directly from its own internal representations instead of using an external retriever. By having decoder attention query pre-encoded evidence chunks, INTRA unifies retrieval and generation, eliminating the typical mismatch seen in retrieval-augmented generation pipelines. Experiments on question-answering benchmarks show that INTRA outperforms strong engineered retrieval pipelines in both evidence recall and overall answer quality.
arXiv Machine Learning
Jun 30
Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
arXiv:2507. 23220v2 Announce Type: replace-cross Abstract: Traditional topic models are effective at uncovering latent themes in large text collections.
Hugging Face Trending Papers
Aug 13
SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale.
arXiv AI
Jul 24
KeySI: An Interaction Framework for Tuning Text Embeddings Based on Human Feedback
arXiv:2607. 20556v1 Announce Type: new Abstract: In large-scale text analysis tasks, pre-trained language models are often used to embed text corpora for downstream analysis.
arXiv AI
Sep 25
R-DEIM Net: An Efficient Rationale-Augmented Dual-Expert Interaction Model for Paraphrase Detection
R-DEIM Net is a 76‑million‑parameter dual‑expert model designed for paraphrase detection that balances accuracy with computational efficiency. It combines an Interaction Expert, which captures token‑level similarity via multi‑scale 2D convolutions and attention, with a Reasoning Expert that generates human‑readable rationales using a Flan‑T5‑small decoder. On the Quora Question Pairs dataset, the model attains 90.07% accuracy and 90.16% F1‑score, matching strong transformer baselines while producing auxiliary rationales.