arXiv Machine Learning

The Implications of Linguistic Illegibility for LLM Security

The paper introduces the concept of "linguistic illegibility," describing how a large language model’s (LLM) language outputs and extracted linguistic features may not accurately reflect its internal computations. It argues that because LLMs compute primarily in activation spaces, any reliance on linguistic self‑reporting for security—such as chain‑of‑thought monitoring or constitutional self‑critique—cannot be fully reliable. The authors propose taint tracking and other sandboxing techniques that do not depend on the model’s linguistic state as a more robust security foundation.

arXiv AI

Aug 26

Semantic Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens and Steering Vectors

The paper introduces Semantic Overlays, a steering technique that adds non‑textual annotations to a language model’s input by applying learned adapters at specific prefill positions. These overlays create an out‑of‑band channel that encodes span identity and complex semantics, enabling the model to interpret marked text differently—such as rewriting code in a specified language or ignoring executable instructions. Experiments show that Semantic Overlays dramatically reduce prompt‑injection success rates while preserving model utility and readability of marked spans.

By Joshua Penman

llms benchmarks

arXiv AI

Jun 30

Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

arXiv:2605. 00994v2 Announce Type: replace-cross Abstract: Finetuning can significantly modify the behavior of large language models, including introducing harmful or unsafe behaviors.

By Mohammed Abu Baker, Luca Baroni, Dan Wilhelm

llms agents fine-tuning benchmarks

arXiv AI

Jun 29

On the Inseparability of Instructions and Data in Shared-Embedding Sequence Models

arXiv:2606. 27567v1 Announce Type: cross Abstract: Prompt injection is the top security risk for LLM-integrated applications, yet every defense proposed so far has been broken.

By Dewank Pant, Shruti Lohani, Avijit Kumar

llms rag safety

arXiv AI

Aug 25

Walking on the DARKSIDE

arXiv:2608.23370v1 Announce Type: new Abstract: Large Language Models (LLMs) recognise patterns but do not natively track the path of exclusions that a coherent discourse demands. When an input rests...

By Aldo Gangemi, Emanuele Bottazzi

llms safety

arXiv Computation and Language

2d ago

Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion

The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.

By Zhuoang Cai

llms agents benchmarks safety

arXiv AI

Jul 13

The LLMbda Calculus: AI Agents, Conversations, and Information Flow

arXiv:2602. 20064v2 Announce Type: replace-cross Abstract: Large language models are increasingly deployed as agents: they plan, call tools, read untrusted data, and act on the results.

By Zac Garby, Andrew D. Gordon, David Sands

llms agents benchmarks

arXiv AI

Jun 30

Linguistic Firewall: Geometry as Defense in Multi-Agent Systems Routing

arXiv:2606. 30555v1 Announce Type: new Abstract: The rapid integration of Large Language Models (LLMs) has driven the evolution of Multi-Agent Systems (MAS), where specialized agents collaborate to execute complex workflows.

By Dvir Alsheich, Adar Peleg, Ben Hagag, Rom Himelstein, Amit Levi, Avi Mendelson

llms rag agents robotics multimodal

arXiv AI

Jul 1

A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

arXiv:2606. 31639v1 Announce Type: cross Abstract: Large language models are no longer only text generators.

By Seyed Bagher Hashemi Natanzi, Bo Tang

llms agents robotics safety

arXiv AI

Jul 13

Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitors

arXiv:2505. 14300v2 Announce Type: replace Abstract: White-box monitoring is increasingly adopted as an auditing tool as Large Language Models (LLMs) are deployed in daily operations to ensure safe model behavior.

By Maheep Chaudhary, Fazl Barez

llms agents robotics benchmarks

arXiv Machine Learning

Jun 2

Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs

arXiv:2506. 14003v5 Announce Type: replace Abstract: Machine unlearning (MU) for large language models (LLMs), commonly referred to as LLM unlearning, seeks to remove specific undesirable data or knowledge from a trained model, while maintaining its performance on standard tasks.

By Yiwei Chen, Soumyadeep Pal, Yimeng Zhang, Qing Qu, Sijia Liu

llms safety

arXiv AI

Jul 1

The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs

arXiv:2606. 22686v2 Announce Type: replace-cross Abstract: Modern Large Language Models (LLMs) rely on extensive safety alignment, yet the mechanistic basis of refusal remains opaque.

By Shivam Ratnakar, Kartikeya Vats

llms multimodal safety

arXiv AI

3d ago

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

arXiv:2602.13576v2 Announce Type: replace-cross Abstract: Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-langu...

By Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng

llms benchmarks safety

Related stories