← Back to all news
arXiv AI By Xinyu Fu, Narayan Ramasubbu, Dennis Galletta

Knowing Is Not Enough: Information Retrievability as a Precondition to Effective LLM Oversight

Read the original on arXiv AI →

The paper argues that effective oversight of large language models (LLMs) depends on users’ ability to retrieve relevant information during review. Through two lab-in-the-field experiments with 640 customer‑facing employees, the authors demonstrate that self‑generated explanations and retrieval cues improve error detection and sustain it over repeated LLM use. They propose that information retrievability is a distinct precondition for oversight, and suggest lightweight onboarding explanations and daily cues as practical solutions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 28

Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation

The paper investigates Self‑Generated Text Recognition (SGTR), the ability of large language models (LLMs) to identify their own outputs. By evaluating 13–21 models across 6 experimental designs, it shows that SGTR accuracy varies with evaluation format, conversation structure, and task domain, and that a quality‑heuristic bias dominates results. The study also finds that fine‑tuning for SGTR in one setting can generalize to others and may cause models to prefer their own outputs when judging, highlighting potential safety concerns.

By Jesse St. Amand, Callum Canavan, Sohaib Imran, Joseph Hewson, Aaron Lutz, Shi Feng, Puria Radmard, Lennie Wells
llmsnlpfine-tuningsafety
More like this →
arXiv AI
Jul 1

Shared Lexical Task Representations Explain Behavioral Variability In LLMs

arXiv:2604. 22027v2 Announce Type: replace-cross Abstract: One of the most common complaints about large language models (LLMs) is their prompt sensitivity -- that is, the fact that their ability to perform a task or provide a correct answer to a question can depend unpredictably on the way the question is posed.

By Zhuonan Yang, Jacob Xiaochen Li, Francisco Piedrahita Velez, Eric Todd, David Bau, Michael L. Littman, Stephen H. Bach, Ellie Pavlick
llms
More like this →
arXiv AI
Aug 24

Can LLMs Introspect? A Reality Check

The paper questions whether large language models (LLMs) truly introspect by critiquing recent studies that claim they can detect and report their internal states. It proposes two necessary conditions for genuine introspection: privileged access to internal representations and second‑order computation that distinguishes from first‑order task performance. Re‑examining two existing paradigms, the authors find that apparent introspective abilities can be explained by input‑based classifiers or generic anomaly detection, concluding that current evidence does not support metacognitive monitoring in LLMs.

By Shashwat Singh, Tal Linzen, Shauli Ravfogel
llms
More like this →