arXiv AI September 28, 2026 By Ziqi Gao, Yaotian Zhu, Qingcheng Zeng, Xu Zhao, Ziqing Wang, Feng Ruan, Kaize Ding

RAPTOR: Ridge-Adaptive Logistic Probes

RAPTOR (Ridge-Adaptive Logistic Probe) is a lightweight, L2‑regularized logistic probe designed to extract accurate, directionally stable concept vectors from frozen large language model (LLM) layer representations. By tuning the ridge penalty on validation data, RAPTOR produces concept vectors from normalized weights that match or surpass strong baselines in accuracy, maintain stability under ablation, and require significantly less training cost. The authors also provide a mechanistic analysis using the Convex Gaussian Min‑max Theorem to explain how penalty strength influences probe accuracy and stability in a high‑dimensional few‑shot setting, with predictions that align with empirical observations on real LLM embeddings.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Jun 10

When is Your LLM Steerable?

Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts.

llms

arXiv Machine Learning

Jun 11

When is Your LLM Steerable?

arXiv:2606. 11599v1 Announce Type: cross Abstract: Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration.

By Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou

llms

arXiv Machine Learning

Jun 9

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

arXiv:2606. 08454v1 Announce Type: new Abstract: Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors.

By Tuc Nguyen, Thai Le

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 16

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.

By Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov

llms fine-tuning benchmarks safety

arXiv Computer Vision

Sep 16

HyCal: A Training-Free Prototype Calibration Method for Cross-Discipline Few-Shot Class-Incremental Learning

arXiv:2604.15678v2 Announce Type: replace Abstract: Pretrained Vision-Language Models (VLMs) like CLIP show promise in continual learning, but existing Few-Shot Class-Incremental Learning (FSCIL) met...

By Eunju Lee, MiHyeon Kim, JuneHyoung Kwon, Yoonji Lee, JiHyun Kim, Soojin Jang, YoungBin Kim

llms rag multimodal benchmarks safety

arXiv Machine Learning

Sep 15

What Does an LLM Learn from Reinforcement Learning? A Mechanistic Interpretability Perspective with Fixed-SAE Track

arXiv:2609.15064v1 Announce Type: new Abstract: Reinforcement learning (RL) is widely utilized in large language model training to improve targeted capabilities, yet how RL reshapes a model remains p...

By Lingheng Du, Yiming Tang, Xufeng Duan, Dianbo Liu

llms reinforcement-learning benchmarks safety

One email a morning, machine-written

Related stories

When is Your LLM Steerable?

When is Your LLM Steerable?

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

HyCal: A Training-Free Prototype Calibration Method for Cross-Discipline Few-Shot Class-Incremental Learning

What Does an LLM Learn from Reinforcement Learning? A Mechanistic Interpretability Perspective with Fixed-SAE Track