arXiv AI September 2, 2026 By Carolina Camassa, Derek Shiller

Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs

The paper investigates how large language models balance instruction-following with pattern completion when the two objectives conflict. By creating dialogues where a user instruction to act in a target way T is opposed by assistant turns that demonstrate a competing pattern P, the authors measure instruction-following rates across 13 models and 16 instructions over up to 50 turns. Results show wide variability (1%–99%) in instruction adherence, with robustness influenced by instruction content, output format, and chain-of-thought reasoning, but overall instruction-following remains brittle under induction pressure.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Sep 23

Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs

arXiv:2609.25602v1 Announce Type: new Abstract: In language models, the choice between believing the prompt and believing the weights is made by a handful of identifiable attention heads. Instruction...

By Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney, Navya Deshmukh, Roushni Sareen, Roshan Kumar Singh

llms fine-tuning benchmarks safety

arXiv AI

Jun 15

Did You Forget What I Asked? Prospective Memory Failures in Large Language Models

arXiv:2603. 23530v2 Announce Type: replace-cross Abstract: Large language models often fail to satisfy formatting instructions when they must simultaneously perform demanding tasks.

By Avni Mittal

llms benchmarks

arXiv AI

Sep 1

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

arXiv:2608.28648v1 Announce Type: new Abstract: We study how instruction-tuned LLMs arbitrate direct conflicts between system and user instructions. We introduce a benchmark of 41 paired constraints...

By Enrique Balp-Straffon, Chih-Hao Hsu, Rushiraj Gadhvi, Sunishchal Dev, Callum Stuart McDougall, Anusha Mujumdar

llms benchmarks

arXiv AI

Sep 12

How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism

Instruction tuning is often thought to give language models a universal ability to follow instructions, but this study shows otherwise. By probing nine tasks across three models, the authors find that general probes reveal selective, not uniform, deficits, cross‑task transfer is weak and skill‑similar, and causal ablation uncovers sparse, asymmetric dependencies. The results suggest instruction following is a coordinated use of diverse linguistic skills rather than a single shared mechanism.

By Elisabetta Rocchetti, Alfio Ferrara

llms fine-tuning benchmarks

arXiv AI

Sep 10

Compliance vs. Sensibility: On the Reasoning Controllability in Large Language Models

arXiv:2604.27251v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) acquire reasoning capabilities through shared inference patterns in pre-training data, which are further elicite...

By Xingwei Tan, Marco Valentino, Mahmud Elahi Akhter, Yuxiang Zhou, Maria Liakata, Nikolaos Aletras

llms

arXiv AI

Jun 9

Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models

arXiv:2606. 07808v1 Announce Type: new Abstract: Reasoning language models deployed in agentic workflows must follow an instruction hierarchy: when instructions from different sources conflict, the model should obey the highest-privilege applicable instruction.

By Sanjay Kariyappa, G. Edward Suh

llms agents benchmarks

One email a morning, machine-written

Related stories

Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs

Did You Forget What I Asked? Prospective Memory Failures in Large Language Models

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism

Compliance vs. Sensibility: On the Reasoning Controllability in Large Language Models

Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models