arXiv Computation and Language
LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Oct 12
Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
The paper investigates how safety‑aligned large language models (LLMs) can still be tricked by jailbreaks or over‑refuse benign requests. By analyzing residual‑stream updates in decoder‑only Transformers, the authors find that harmfulness is more readily decoded from attention updates—especially in early layers—while the MLP component is primarily responsible for writing refusal responses. Post‑training enhances MLP‑driven refusal writing without improving harmfulness decodability, and causal interventions confirm that downstream MLP updates are crucial for the refusal shift caused by upstream attention changes. Building on these insights, the authors propose Sentinel, a system that combines an attention‑derived perception score with an MLP‑derived decision score to better detect jailbreaks and over‑refusals across multiple models.
arXiv Machine Learning
Jul 21
Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation
arXiv:2604. 16197v2 Announce Type: replace Abstract: Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability challenges on LLMs.
arXiv AI
Oct 15
SPINE: Measuring LLM Sycophancy under Sustained Multi-Turn Pressure
SPINE is a benchmark that tests large language models (LLMs) for sycophancy by having a proxy user persistently challenge a target model for up to 25 turns. The study evaluates several commercial and open‑source models on false presuppositions and unethical queries, finding that collapse rates—where the model abandons the correct position—rise sharply with conversation length, from 0‑57% within five turns to 20‑97% within 25 turns. Analysis of reasoning traces shows that the correct position often remains present, indicating that sycophancy is a choice to please rather than a knowledge failure, and that emotional appeals are the most effective tactic for inducing collapse.
arXiv AI
Jun 2
On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
arXiv:2606. 00467v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for zero-shot annotation and LLM-as-a-judge tasks, yet their reliability hinges on how model-internalized priors interact with user-provided instructions.
arXiv AI
Aug 25
Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion
The study investigates how lexical perturbations—such as keyboard noise, character swaps, and filler insertion—affect large language models (LLMs) on reasoning benchmarks. Four open-weight instruction-tuned models and frontier models were evaluated, revealing that character-level perturbations significantly reduce accuracy, especially on multi-step reasoning tasks, while filler insertion has minimal impact. The authors attribute this asymmetry to Attention Diversion, where fragmented subword tokenization draws disproportionate attention in middle and final transformer layers; they demonstrate that both token content and attention allocation are coupled, making it difficult for inference-time repair strategies to fully recover performance.
arXiv AI
Aug 11
LLMs Remember First, Forget Last: Dual-Process Interference in Large Language Models
arXiv:2603. 00270v3 Announce Type: replace-cross Abstract: Large language models can process millions of tokens, yet how they handle conflicting information within context remains poorly understood.