Formal semantics has shown that sentence meanings arise by recursively composing lexical meanings, yet much of the literature on semantic universals models either lexicons with fixed signal structures or holistic composition without interpretable lexical parts. We introduce a framework that integrates this fundamental insight of formal semantics in evolutionary modeling, by allowing lexical meanings and a composition function to co-evolve under pressures for conceptual simplicity and communicative accuracy.
arXiv:2605. 08934v2 Announce Type: replace Abstract: Mechanistic interpretability aims to explain neural model behaviour by reverse-engineering learned computational structure into human-understandable components.
By Ward Gauderis, Thomas Dooms, Steven T. Homer, Kola Ayonrinde, Geraint A. Wiggins
arXiv:2607.29642v2 Announce Type: replace
Abstract: The iterated learning model was introduced to investigate language evolution: the way in which the characteristic properties of human languages hav...
By Fabio De Ponte, Eloise Gaines-White, Conor Houghton, Seth Bullock
arXiv:2607. 18961v1 Announce Type: new Abstract: Large language models (LLMs) generate fluent text by incrementally predicting the next token from a prefix.
By Remo Pareschi
The paper investigates why Transformers struggle more with structural than lexical compositional generalisation. It argues that this disparity stems from low structural type diversity rather than an inherent limitation of Transformers. By creating linguistically diverse variants of the COGS and SLOG datasets, the authors show that type diversity correlates equally with generalisation in both lexical and structural cases, challenging previous explanations of the difficulty.
By Anssi Moisio, Mathias Creutz, Mikko Kurimo
arXiv:2607. 25335v1 Announce Type: cross Abstract: Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes.
By Jianfei Ma, Zhaoxin Feng, Emmanuele Chersoni, Si Chen
The paper reports a Size‑Fidelity Paradox in large‑language‑model (LLM) context compression: larger compressors can produce lower reconstruction error yet yield less faithful reconstructed contexts. Two key mechanisms drive this effect: knowledge overwriting, where larger models replace source facts with their own beliefs, and semantic drift, where they paraphrase or restructure content. Across 27 compressor setups, mid‑sized models often recover contexts more faithfully, and analysis of embedding geometry shows that larger models organize memory into broader semantic subspaces, leading to ambiguous representations that weaken recovery.
By Ruishan Guo, Yibing Liu, Guoxin Ma, Yan Wang, Yueyang Zhang, Long Xia, Kecheng Chen, Zhiyuan Sun, Daiting Shi
arXiv:2609.01491v1 Announce Type: cross
Abstract: The growing rate at which LLM agents interact with one another raises key questions about language evolution in multi-LLM-agent settings, with implic...
By Elias Stengel-Eskin, Newton Sander, Carlos Bonetti, Sasha Boguraev, James Bowler, Hale Sirin, Simon Kirby
arXiv:2610.35111v1 Announce Type: new
Abstract: Large language models operationalize an intuitive, view of meaning as use. We argue that it faces a rule-following challenge. Using Kripke's interpreta...
By Niyar R Barman, Omar Naim, Nicholas Asher
arXiv:2610.08303v1 Announce Type: new
Abstract: Current evaluation of multilingual Large Language Models (LLMs) rests on an implicit Translation-Isomorphism Assumption (TIA): that semantic structures...
By Shu-Kai Hsieh, Da-Chen Lian
The study investigates whether pretrained transformer models encode functional words—such as pronouns and adverbs—in a way that mirrors human usage. By comparing embeddings of nouns with those of their functional counterparts in both isolated and parallel sentences, the authors find that functional words occupy a central yet distinct position in embedding space and that parallel lexicalized and functional sentences reside in different subspaces. Experiments show that only a mixed training set of functional and lexicalized sentences reveals shared syntactic and semantic structure, whereas training on either type alone fails to capture this parallelism.
By Giuseppe Samo, Vivi Nastase, Paola Merlo
arXiv:2603. 20381v2 Announce Type: replace-cross Abstract: Understanding the fundamental mechanisms governing the production of meaning in the processing of natural language is critical for designing safe, thoughtful, engaging, and empowering human-agent interactions.
By Christopher J. Agostino, Quan Le Thien, Nayan D'Souza, Louis van der Elst