arXiv AI

Interpreting Language Model Hidden States at Scale

arXiv:2608. 10260v1 Announce Type: new Abstract: Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network.

Hugging Face Trending Papers

Aug 10

Interpreting Language Model Hidden States at Scale

Lens methods interpret large language models (LLMs) by mapping intermediate activations to the output vocabulary, revealing how next-token predictions develop through the network. Trained lenses remain expensive: affine-translator parameters grow quadratically with model width, while exact, full-vocabulary Kullback--Leibler (KL) training dominates memory.

llms safety

arXiv AI

Jul 13

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.

By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

llms fine-tuning multimodal safety

arXiv AI

Aug 5

In-Context Collapse in Vision-Language Models and How to Mitigate it?

arXiv:2608. 02830v1 Announce Type: cross Abstract: Many-shot in-context learning (ICL) lets vision-language models (VLMs) adapt from image--label demonstrations without weight updates, and is widely assumed to improve as more demonstrations are supplied.

By Mohammad Rostami

llms computer-vision multimodal benchmarks

arXiv Machine Learning

Jul 21

Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation

arXiv:2604. 16197v2 Announce Type: replace Abstract: Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods suffer from scalability challenges on LLMs.

By Yide Ran, Jianwen Xie, Minghui Wang, Wenjin Zheng, Denghui Zhang, Chuan Li, Zhaozhuo Xu

llms

arXiv AI

Jun 11

ICA Lens: Interpreting Language Models Without Training Another Dictionary

arXiv:2606. 11722v1 Announce Type: cross Abstract: Finding interpretable directions in language-model representations is critical for understanding and controlling model behavior.

By Sida Liu, Feijiang Han

llms safety

arXiv AI

Jun 16

OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

arXiv:2604. 18827v2 Announce Type: replace-cross Abstract: Scaling data and artificial neural networks has transformed AI, driving breakthroughs in language and vision.

By Konstantin F. Willeke, Polina Turishcheva, Alex Gilbert, Goirik Chakrabarty, Hasan A. Bedel, Paul G. Fahey, Yongrong Qiu, Marissa A. Weis, Michaela Vystr\v{c}ilov\'a, Taliah Muhammad, Lydia Ntanavara, Rachel E. Froebe, Kayla Ponder, Zheng Huan Tan, Emin Orhan, Erick Cobos, Sophia Sanborn, Katrin Franke, Fabian H. Sinz, Alexander S. Ecker, Andreas S. Tolias

llms computer-vision multimodal benchmarks

arXiv Machine Learning

Jun 17

Olmo Hybrid: From Theory to Practice and Back

arXiv:2604. 03444v4 Announce Type: replace Abstract: Recent work has demonstrated the potential of non-transformer language models, especially linear recurrent neural networks (RNNs) and hybrid models that mix recurrence and attention.

By William Merrill, Yanhong Li, Tyler Romero, Anej Svete, Caia Costello, Pradeep Dasigi, Dirk Groeneveld, David Heineman, Bailey Kuehl, Nathan Lambert, Chuan Li, Kyle Lo, Saumya Malik, DJ Matusz, Benjamin Minixhofer, Jacob Morrison, Luca Soldaini, Finbarr Timbers, Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi, Ashish Sabharwal

llms

arXiv Machine Learning

Jun 16

Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning

arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.

By Gustav Olaf Yunus Laitinen-Fredriksson Lundstrom-Imanov

llms fine-tuning benchmarks safety

arXiv AI

Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen

llms computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 2

DLLM-JEPA: Joint Embedding Predictive Architectures for Masked Diffusion Language Models

arXiv:2606. 00091v1 Announce Type: cross Abstract: Joint Embedding Predictive Architectures (JEPAs) have reshaped self-supervised representation learning in vision.

By Sangdae Nam

llms rag diffusion fine-tuning benchmarks

arXiv Machine Learning

Aug 7

Cross-Architecture Steering Transfer in Language Models: A Systematic Empirical Study

arXiv:2608. 05164v1 Announce Type: cross Abstract: Independently trained large language models may develop shared internal representations of semantic concepts despite architectural differences -- but whether this geometric similarity has functional consequences for cross-model behavioural control remains untested.

By Ayushi Agarwal

llms fine-tuning safety

arXiv AI

Jul 17

Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect

arXiv:2607. 14111v1 Announce Type: cross Abstract: Can small language models detect and report on perturbations their own internal activations?

By Ely Hahami, Ishaan Sinha, Lavik Jain

llms fine-tuning benchmarks safety

Related stories