CHRONOBERG is a temporally structured corpus of English book texts covering 250 years, curated from Project Gutenberg and enriched with temporal annotations. It enables quantification of lexical semantic change via time‑sensitive Valence‑Arousal‑Dominance analysis and the creation of historically calibrated affective lexicons. Experiments show that language models trained sequentially on CHRONOBERG struggle to encode diachronic shifts, highlighting the need for temporally aware training and evaluation pipelines.
By Niharika Hegde, Subarnaduti Paul, Lars Joel-Frey, Manuel Brack, Kristian Kersting, Martin Mundt, Patrick Schramowski
The study probes Gemma‑2‑9B‑IT with Sparse Autoencoders across English, Hebrew, and Russian to examine how multilingual LLMs handle informal register. By using a dataset of polysemous terms that appear in literal and informal contexts, the authors isolate pragmatic register processing from lexical cues. They discover a small, robust cross‑linguistic core that forms an informal register subspace, which becomes clearer in deeper layers and can causally shift output formality across all tested languages, even transferring zero‑shot to six unseen languages.
By Uri Z. Kialy, Avi Shtarkberg, Ayal Klein
The paper investigates whether different latent language probes—GMM-based representation probes and decoding-based probes—measure the same phenomenon in multilingual language models. Across various model families, training regimes, domains, tasks, checkpoints, and up to 27 languages, the authors find systematic disagreement: representation probes indicate earlier cross‑lingual mixing, while decoding probes reveal sharper, English‑biased language signals. These differences correlate with model multilinguality and training progression but remain relatively stable across domains, suggesting that current probes capture distinct aspects of multilingual processing rather than a single internal lingua franca.
By Deniz Bayazit, Badr AlKhamissi, Antoine Bosselut
arXiv:2609.08609v2 Announce Type: replace
Abstract: Tracking semantic change in low-resource languages across extensive historical timelines presents significant challenges due to data scarcity and t...
By Nevidu Jayatilleke, Nisansa de Silva
The study compares an English-only and a bilingual decoder-only model, each 310 M parameters, trained on eight diverse languages while controlling for English exposure, compute, and document overlap. After aligning on shared English vocabulary, the authors find that token embeddings appear similar, but the deeper hidden states used for prediction diverge across models. This hidden‑state mismatch grows through middle transformer layers and persists despite controls, indicating that contextual processing differs between the models.
"whyItMatters":"The findings show that embedding alignment can conceal significant internal representation differences, which is crucial for any downstream work that assumes aligned multilingual models are interchangeable."
By Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos
arXiv:2608.28924v1 Announce Type: new
Abstract: Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar...
By Sasha Boguraev, Toshiki Nakai, Kyle Mahowald, Julius Steuer
arXiv:2604.08568v3 Announce Type: replace-cross
Abstract: The widespread use of LLM-based writing assistance has raised an interesting question about the homogenization of English. As LLMs tend to re...
By Nabelanita Utami, Ryohei Sasano
arXiv:2609.23178v1 Announce Type: new
Abstract: Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its response...
By Ted Underwood, Ziliang Qiu, Sarah Griebel, Laura K. Nelson, Edwin Roland, Wenyi Shang, Matthew Wilkens
The paper introduces Coupled Usage–Sense Processes (CUSP), a method that models lexical semantic change by coupling contextual distributions through latent usage components and using Markov composition to link adjacent time periods. CUSP quantifies change magnitude and timing, separates variation into component movement and internal reorganization, and attributes changes to specific transported component pairs. The approach is validated on synthetic data, English and German corpora, and a large corpus of US court opinions, providing detailed, text‑grounded insights into how word meanings evolve over time.
By Haruka Ezoe, Ryohei Hisano
arXiv:2608. 19472v1 Announce Type: new Abstract: Lexical semantic change (LSC) is commonly modelled through vector-space representations, but these approaches often provide limited insight into which aspects of usage are changing.
By Bach Phan-Tat, Kris Heylen, Dirk Geeraerts, Stefano De Pascale, Dirk Speelman
The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.
By Muge Zhang, Aaron Jencks, Krishna Badikela, Yulia Tsvetkov, Sachin Kumar
arXiv:2608. 12334v1 Announce Type: cross Abstract: Despite the impressive multilingual capabilities of Large Language Models, the latent dynamics dictating language selection remain poorly understood.
By Arnav Srivastav