arXiv AI
Jul 7
Unsupervised Features Mining via Activation Geometry
arXiv:2607. 04222v1 Announce Type: new Abstract: Interpretability methods aim to reveal the features represented inside large language models (LLMs).
J-Miner: Recovering Executable Decision Knowledge from Language-Model Classifiers
J-Miner extracts executable decision knowledge from fine‑tuned language‑model classifiers by aggregating internal signals aligned with named concepts and learning rules over them. The mined rules reproduce up to 98.3% of the original classifier’s decisions and outperform compact rules based on input words by 6.0–29.5 percentage points in behavioral fidelity. Moreover, these rules can be transferred to lightweight student models that use only about 1/24 of the parameters yet retain 99.8% of the source classifier’s accuracy.
Related stories
arXiv AI
Jun 8
Small Language Model Agents Enable Efficient and High-Quality Knowledge Mining
arXiv:2510. 01427v3 Announce Type: replace Abstract: At the core of Deep Research is knowledge mining, the task of extracting structured information from massive unstructured text in response to user instructions.
arXiv Machine Learning
Sep 21
Exemplar Partitioning for Mechanistic Interpretability
The paper introduces Exemplar Partitioning (EP), an unsupervised technique that constructs interpretable feature dictionaries from large language model activations by clustering streamed activations into Voronoi regions defined by exemplars and their averages. EP allows comparison of dictionaries across layers, checkpoints, and architectures, and demonstrates utility in interpreting model behavior, tracking training dynamics, detecting hidden concepts, and enabling targeted interventions. Experiments on Gemma‑2‑2B and Llama‑3.1‑8B show EP can reveal how instruction tuning reorganizes harmful prompt activations, facilitate interventions that alter model responses, and achieve high concept‑detection performance while requiring far fewer construction tokens than comparable methods.
arXiv Machine Learning
Sep 22
Guiding the coarse levels of semantic IDs makes the fine levels learnable
arXiv:2609.22227v1 Announce Type: cross Abstract: Generative retrieval represents each item by a short Semantic ID and casts recommendation as autoregressive generation of that sequence. Because the...
arXiv AI
Aug 20
The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations
The paper introduces a lifecycle framework for LLM-as-a-Judge systems used to evaluate recommendation explanations at Netflix. It outlines four phases—Birth, Training, Deployment, and Monitoring—detailing how each stage addresses specific technical and operational challenges. The authors report that after five weeks of A/B testing, judge-aligned explanations increased novel content viewing and successful browse-to-play sessions without quality takedowns.
arXiv AI
Jul 22
SkillSight: Seeing Through Shared Descriptions for Accurate Skill Retrieval
arXiv:2607. 18785v1 Announce Type: new Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.
arXiv AI
Aug 10
Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes
arXiv:2608. 07208v1 Announce Type: cross Abstract: Existing measures of how much a text is about a concept read the surface of the text: dictionary word shares, topic proportions, embedding similarities.
arXiv AI
Jul 31
SkillSight: Calibrating Generic Content Bias for Skill Retrieval
arXiv:2607. 18785v2 Announce Type: replace Abstract: As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution.
arXiv AI
Jun 2
On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
arXiv:2606. 00467v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions.
arXiv AI
Sep 10
Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
The paper introduces XTF, an explainable token‑level noise filtering framework for fine‑tuning large language models. XTF breaks down token contributions into reasoning importance, knowledge novelty, and task relevance, scores them, and masks gradients of noisy tokens to improve fine‑tuning. Experiments on math, code, and medicine tasks across seven LLMs show up to a 13.7% performance boost over standard fine‑tuning.
arXiv AI
Aug 17
The Metacognitive Bottleneck: Japanese Riddles Reveal Fundamental Limits of Machine Insight and Self-Evaluation in Reasoning AI
arXiv:2509. 14704v3 Announce Type: replace Abstract: Benchmark saturation and training-data contamination increasingly obscure whether reported gains in large language models (LLMs) reflect genuine advances in reasoning or familiarity with recurring patterns in benchmark problems.
arXiv Computation and Language
Aug 31
Knowing Before Answering: Decoding Language Models for Reliable RAG
The paper introduces a method for determining whether retrieval-augmented generation (RAG) systems have sufficient, insufficient, or conflicting evidence to answer a question. By training a lightweight linear classifier on hidden activations and attention-derived features from 16 language models, the authors demonstrate that these internal signals reliably predict the adequacy of retrieved documents, outperforming prompting-based baselines and specialized RAG models. Analysis shows that middle-layer hidden states carry the most informative signals for this triage task.