arXiv AI

Same physical state, different collective dynamics: state encodings select synchronization outcomes in language-model agents

arXiv:2608. 06968v1 Announce Type: cross Abstract: Language-model agents act on state encodings of their environment, yet these are treated as interchangeable interfaces.

arXiv AI
Sep 24

How a shared state is described determines whether AI agents synchronize

The article investigates how the textual description of a shared state influences the collective behavior of language‑model agents. By testing 507,112 responses across different model families on a circular coordination task, the authors show that varying the state description (e.g., numerical summaries vs. histograms) can alter whether agents align, split, or fail to coordinate. The study demonstrates that the way a shared state is described is an integral part of the interaction rule that determines collective order.

By Takahiro Ezaki, Naoto Imura, Katsuhiro Nishinari
arXiv AI
Aug 20

First-Token Broadcasters: Mechanistic Origins of Language Identity and Distributed Robustness in Transformers

The paper introduces Language Identity Head Ablation (LIHA), a causal method that zeroes individual attention heads in transformer models to measure language switch rates across multilingual prompts. Applying LIHA to GPT‑2 reveals a small set of first‑token broadcaster heads—most notably L6H1—that persistently attend to the initial prompt token and propagate language signals throughout generation, with compensatory head recruitment occurring hierarchically in higher layers. A controlled comparison between Qwen2.5‑1.5B‑Base and Qwen2.5‑1.5B‑Instruct shows that instruction tuning concentrates language‑identity influence in early layers, while experiments with Chinese and Russian confirm script‑specific first‑token broadcasting at layer 0.

By Arjun Pillai, Christian Hoang, Anjelo Jann Laroza
arXiv AI
Sep 12

Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cells

The study investigates whether independently trained language‑model societies share a common packet language and how inherited interface states affect learning. A comprehensive audit of 30 pairwise interactions among six restricted societies shows that only one pair is fully interoperable, another is partially compatible, and the remaining 26 pairs fail across all alignment levels. Further experiments reveal that a globally trained communication interface can act as a severe negative‑transfer prior, but inherited interfaces never outperform fresh‑interface controls by the preregistered margin.

By Narcis Marincat
Hugging Face Trending Papers
Sep 10

Portable Semantics, Private Dialects: Reuse and Negative Transfer in Latent Communication Between Language-Model Cells

The study investigates whether independently trained language‑model societies share a common packet language and how inherited interface states affect learning. A comprehensive audit of 30 pairwise interactions among six restricted societies shows that most cross‑initialization pairs fail to interoperate, with only one pair achieving full bidirectional compatibility. Further experiments reveal that reinitializing only the packet reader, writer, and mouth dramatically improves accuracy, while inherited interfaces never outperform fresh ones by the preregistered margin.