Chronoberg is a 250‑year English book corpus from Project Gutenberg, enriched with temporal annotations and affective lexicons (Valence‑Arousal‑Dominance). It allows researchers to track lexical semantic change over time and to evaluate how large language models handle sentiment and discriminatory language across different historical periods. The authors demonstrate that current LLMs struggle with diachronic shifts, underscoring the need for temporally aware training and evaluation pipelines.
By Niharika Hegde, Subarnaduti Paul, Lars-Joel Frey, Manuel Brack, Kristian Kersting, Martin Mundt, Patrick Schramowski
arXiv:2405. 17838v3 Announce Type: replace-cross Abstract: Socio-linguistic indicators of affectively-relevant phenomena, such as emotion or sentiment, are often extracted from text to better understand features of human-computer interactions, including on social media.
By Keith Burghardt, Daniel M. T. Fessler, Chyna Tang, Anne Pisor, Kristina Lerman
arXiv:2606.02991v2 Announce Type: replace-cross
Abstract: While modern language models increasingly rely on ever-larger web corpora, we show that pretraining on historical text (e.g., pre-1913 text)...
By Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber, Yixuan Wang, Junchi Yu, Freda Shi, Philip Torr, Yao Lu
arXiv:2608. 03507v1 Announce Type: cross Abstract: Historical language change affects morphology, syntax, semantics, and pragmatics, yet computational studies typically examine these levels with incompatible representations and therefore cannot determine whether they evolve together across languages.
By Gagan Bhatia, Julian Schlenker, Simone Paolo Ponzetto, Steffen Eger
The paper investigates how seven large language models encode fine‑grained emotions by analyzing their hidden representations on the GoEmotions dataset, which contains 27 emotion categories plus neutral. Using KMeans clustering on the models’ final‑token representations, the authors find that the representations tend to over‑compress, grouping emotions mainly into positive versus negative clusters and recovering coarser emotion families with fewer clusters than the full taxonomy. Despite this compression, linear probes still capture substantial fine‑grained emotional information, indicating that detailed affective signals remain accessible within the models’ internal geometry.
By Laya Iyer, Sanmi Koyejo, Chen Shani
The paper introduces point‑in‑time language models that are trained only on text available up to each calendar date, thereby eliminating lookahead bias that plagues standard large language models. By scaling decoder‑only transformers to 4 billion parameters and training on 1 trillion chronologically filtered tokens from FineWeb, the authors produce monthly checkpoints from 2013 to 2024 that achieve performance close to unconstrained models of similar size on common‑sense reasoning and language understanding tasks. Instruction fine‑tuning with LoRA further enhances downstream usability, and the authors release the full pipeline—including dataset construction, training infrastructure, and evaluation code—for reproducible research.
By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu