arXiv Computation and Language
TaxoSAE: Rethinking Sparse Autoencoders through Taxonomic Logic
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Sep 22
Guiding the coarse levels of semantic IDs makes the fine levels learnable
arXiv:2609.22227v1 Announce Type: cross Abstract: Generative retrieval represents each item by a short Semantic ID and casts recommendation as autoregressive generation of that sequence. Because the...
arXiv AI
Jun 24
SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization
arXiv:2606. 24259v1 Announce Type: cross Abstract: Fine-tuned encoders deployed across heterogeneous NLP tasks face three compounding problems: mismatched inductive biases, class-imbalance corruption of feature statistics, and no mechanism to condition attention on external lexical knowledge.
arXiv Machine Learning
Oct 15
Sparse Autoencoders Are a Shared Coordinate System for Symbolic Matching
The paper introduces SAE‑IDF, a data‑driven approach that uses sparse autoencoders to encode symbols into interpretable sparse codes, re‑weights dimensions by rarity, and matches units across heterogeneous symbolic systems. On 12 tasks spanning six symbolic system types, SAE‑IDF outperforms dense LLM baselines (Gemma‑2‑9B and Llama‑3.1‑8B) by up to 45 points, consistently across layers and configurations. Each sparse dimension carries a human‑readable meaning, allowing matches to decompose into named concepts, thereby enabling a scalable, universal symbolic backbone.
arXiv Machine Learning
Oct 16
AlignSAE: Concept-Aligned Sparse Autoencoders
AlignSAE is a method that aligns Sparse Autoencoder (SAE) features with a predefined ontology using a two‑stage curriculum: an initial unsupervised pre‑train followed by supervised post‑train that binds specific concepts to dedicated latent slots. This approach preserves general reconstruction capacity while creating an interpretable interface for inspecting and controlling individual concepts. Experiments on GPT‑2 show that AlignSAE enables precise causal interventions—such as reliable concept swaps with 94.8% success in a 1‑hop QA task—and achieves 9–∞× better disentanglement than standard SAEs, as measured by Mutual Information Gap, while supporting multi‑hop reasoning and probing generalization dynamics.
arXiv AI
Sep 2
From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification
arXiv:2609.01564v1 Announce Type: cross Abstract: Large language models (LLMs) struggle to classify text into taxonomies with many semantically similar labels, as the distinctions are domain-specific...
arXiv Computation and Language
Oct 12
Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion
The paper introduces SAFE‑QE, a training‑free, retriever‑agnostic method for query expansion that mitigates hallucinated terms from large language models. It uses a subsequence‑association filter to score candidate terms based on their reproducibility across randomised query contexts, down‑weighting terms that lack corpus support. SAFE‑QE then selects a broad yet consistent set of expansions via constrained coverage maximisation, and defines a new metric, the Hallucinated Expansion Term Rate, to quantify faithfulness. The approach matches state‑of‑the‑art ranking performance while injecting far fewer unsupported terms on TREC and BEIR benchmarks.