← Back to all news
arXiv AI By Zilu Tang, Jongin Kim, Kerem Sahin, David Atkinson, Aaron Mueller, Derry Wijaya

Interventional Training Reduces Causal Bypass in In-Context Introspection

Read the original on arXiv AI →

The paper investigates how language models (LMs) can introspect their internal signals and identifies a problem called causal bypass, where self-reports depend on correlated but causally unrelated signals. By training hundreds of introspectors on low‑level signals across multiple LMs, the authors show that standard supervised fine‑tuning (SFT) often leads to causal bypass. They propose interventional training, which trains models to report on counterfactually perturbed values, and demonstrate that this approach yields more accurate introspection for both endogenous and injected signals, even on out‑of‑distribution data, and that the method generalizes across token positions and scales with more data.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 24

Can LLMs Introspect? A Reality Check

The paper questions whether large language models (LLMs) truly introspect by critiquing recent studies that claim they can detect and report their internal states. It proposes two necessary conditions for genuine introspection: privileged access to internal representations and second‑order computation that distinguishes from first‑order task performance. Re‑examining two existing paradigms, the authors find that apparent introspective abilities can be explained by input‑based classifiers or generic anomaly detection, concluding that current evidence does not support metacognitive monitoring in LLMs.

By Shashwat Singh, Tal Linzen, Shauli Ravfogel
llms
More like this →
arXiv Computation and Language
Sep 2

Can LLMs Reliably Self-Report Adversarial Prefills, and How?

The study investigates whether large language models (LLMs) can reliably detect when their own responses have been manipulated by adversarial prefill attacks. Across ten instruction‑tuned LLMs ranging from 3B to 70B parameters and four safety benchmarks, none consistently recognized compromised outputs, with models claiming intent on prefilled responses at an average of 25.3%. The research identifies that introspective signals mainly arise from safety reasoning and refusal, and that training to improve introspection can paradoxically increase attack success, underscoring the fragility of LLM self‑reporting in safety contexts.

By Quang Minh Nguyen, Uzair Ahmed, Taegyoon Kim
llmsbenchmarkssafety
More like this →