← Back to all news
arXiv Machine Learning By Md Abdul Kadir (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany), Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbr\"ucken, Germany), Omair Shahzad Bhatti (German Research Center for Artificial Intelligence), Abdulrahman Mohamed Selim (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany), Daniel Sonntag (German Research Center for Artificial Intelligence, University of Oldenburg, Oldenburg, Germany)

Where Do LLMs Retrieve Network Information? Identifying the Responsible Heads and Neurons

Read the original on arXiv Machine Learning →

The study investigates how large language models retrieve network information, such as domain names and IP addresses, by identifying specific attention heads and neurons responsible for this capability. Using causal ablation and correlational analyses across five LLMs, the authors find that a small subset of heads can achieve near-perfect detection of network relationships, while single neurons show variable performance. The results demonstrate that the granularity of analysis—heads versus individual neurons—significantly affects the ability to localize retrieval mechanisms.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 7

Finding the Heads and the Neurons Responsible for Network Information Retrieval in Language Models

The study investigates whether particular attention heads and individual neurons within those heads in language models are responsible for detecting network infrastructure information—specifically hostnames paired with IP addresses. Using causal ablation and selective testing across five models from three architecture families, the authors find that a small subset of heads reliably identifies such information with near-perfect accuracy. However, the extent to which this responsibility is concentrated in a single neuron varies by model; in some cases a single neuron suffices, while in others the signal is distributed across the head. The findings generalize to an independent reverse‑DNS dataset, though single‑neuron detectors are less robust.

By Abdul Kadir (University of Oldenburg, Oldenburg, Germany, German Research Center for Artificial Intelligence), Md Mohasin Hossain (German Research Center for Artificial Intelligence, Saarland University, Saarbrucken, Germany), Daniel Sonntag (University of Oldenburg, Oldenburg, Germany, German Research Center for Artificial Intelligence)
llmsbenchmarks
More like this →
arXiv Computation and Language
Sep 4

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

The study investigates the consistency and specificity of language model circuits across six tasks and five models, focusing on component-level (attention heads and MLP blocks) and neuron-level circuits. Component-level circuits are highly consistent and causally important but lack task specificity, as ablating a circuit for one task similarly harms performance on other tasks. Neuron-level circuits show higher task specificity but lower consistency, with overlap mainly between closely related tasks. The analysis of Llama‑3.2‑3B reveals that shared components are predominantly MLP blocks, while attention heads act as generic attention‑sink heads.

By Michael Li, Nishant Subramani
llmssafety
More like this →