arXiv:2608. 20054v1 Announce Type: new Abstract: Multi-module systems often expose every module to the full input.
arXiv:2608.20054v3 Announce Type: replace Abstract: Multi-module neural systems often expose every module to the full input. We test whether a slot-selective evidence-masking regime -- restricting ea...
The study investigates whether independently trained language‑model societies share a common packet language and how inherited interface states affect learning. A comprehensive audit of 30 pairwise interactions among six restricted societies shows that only one pair is fully interoperable, another is partially compatible, and the remaining 26 pairs fail across all alignment levels. Further experiments reveal that a globally trained communication interface can act as a severe negative‑transfer prior, but inherited interfaces never outperform fresh‑interface controls by the preregistered margin.
The study investigates whether restricting a module’s access to information—through evidence masking—enhances a system’s ability to learn compositional tasks. In a preregistered experiment with sixty‑four‑cell systems built on a frozen language‑model backbone, researchers varied evidence masking, ownership markers, and filler replacement across multiple initialization clusters and data orders. Results showed that when markers were available, masking significantly improved accuracy on held‑out two‑ and three‑operation compositions, with all tested pairs meeting performance thresholds and the preregistered behavioral criterion satisfied. The study also explored packet interventions and found predicted intermediate‑value changes, though mediation was not conclusively established. "whyItMatters":"The findings demonstrate a substantial performance benefit from evidence masking in compositional generalization tasks, offering a promising direction for designing more effective learning systems."
The study investigates whether independently trained language‑model societies share a common packet language and how inherited interface states affect learning. A comprehensive audit of 30 pairwise interactions among six restricted societies shows that most cross‑initialization pairs fail to interoperate, with only one pair achieving full bidirectional compatibility. Further experiments reveal that reinitializing only the packet reader, writer, and mouth dramatically improves accuracy, while inherited interfaces never outperform fresh ones by the preregistered margin.
arXiv:2610.15883v1 Announce Type: cross Abstract: Text wrapped in a trigger can be made behaviorally invisible to a language model: omitted from summaries and quotations, ignored in question answerin...
arXiv:2610.19696v1 Announce Type: new Abstract: Multi-agent systems allow language models to solve tasks collaboratively by sharing complementary information. These systems typically communicate thro...
arXiv:2608. 08829v1 Announce Type: cross Abstract: Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task.
arXiv:2608.21766v1 Announce Type: cross Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their beha...
arXiv:2607.27617v2 Announce Type: replace Abstract: Identical language-model answers can arise from hidden states that support different future computations, so current-answer probes do not establish...
The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.
arXiv:2608. 05162v1 Announce Type: cross Abstract: Pooling is a consequential but under-examined design choice in decoder-only concept representation work: practitioners must collapse token-level hidden states into a passage-level vector, yet no shared protocol exists for comparing this choice across concepts, models, and tasks.
The paper investigates whether removing a small set of attention heads in transformer-based language models can improve task performance, a phenomenon termed Capability Enhancement Localization (CEL). Empirical experiments across various models identify sparse head combinations whose ablation boosts performance without fine-tuning. The authors also provide a theoretical explanation using a simplified attention model, showing that training dynamics can concentrate conflicting task features into specific heads, whose removal selectively benefits one task while harming another.