Chronoberg is a 250‑year English book corpus from Project Gutenberg, enriched with temporal annotations and affective lexicons (Valence‑Arousal‑Dominance). It allows researchers to track lexical semantic change over time and to evaluate how large language models handle sentiment and discriminatory language across different historical periods. The authors demonstrate that current LLMs struggle with diachronic shifts, underscoring the need for temporally aware training and evaluation pipelines.
By Niharika Hegde, Subarnaduti Paul, Lars-Joel Frey, Manuel Brack, Kristian Kersting, Martin Mundt, Patrick Schramowski
arXiv:2405. 17838v3 Announce Type: replace-cross Abstract: Socio-linguistic indicators of affectively-relevant phenomena, such as emotion or sentiment, are often extracted from text to better understand features of human-computer interactions, including on social media.
By Keith Burghardt, Daniel M. T. Fessler, Chyna Tang, Anne Pisor, Kristina Lerman
arXiv:2606.02991v2 Announce Type: replace-cross
Abstract: While modern language models increasingly rely on ever-larger web corpora, we show that pretraining on historical text (e.g., pre-1913 text)...
By Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber, Yixuan Wang, Junchi Yu, Freda Shi, Philip Torr, Yao Lu
arXiv:2608. 03507v1 Announce Type: cross Abstract: Historical language change affects morphology, syntax, semantics, and pragmatics, yet computational studies typically examine these levels with incompatible representations and therefore cannot determine whether they evolve together across languages.
By Gagan Bhatia, Julian Schlenker, Simone Paolo Ponzetto, Steffen Eger
The paper investigates how seven large language models encode fine‑grained emotions by analyzing their hidden representations on the GoEmotions dataset, which contains 27 emotion categories plus neutral. Using KMeans clustering on the models’ final‑token representations, the authors find that the representations tend to over‑compress, grouping emotions mainly into positive versus negative clusters and recovering coarser emotion families with fewer clusters than the full taxonomy. Despite this compression, linear probes still capture substantial fine‑grained emotional information, indicating that detailed affective signals remain accessible within the models’ internal geometry.
By Laya Iyer, Sanmi Koyejo, Chen Shani
The paper introduces point‑in‑time language models that are trained only on text available up to each calendar date, thereby eliminating lookahead bias that plagues standard large language models. By scaling decoder‑only transformers to 4 billion parameters and training on 1 trillion chronologically filtered tokens from FineWeb, the authors produce monthly checkpoints from 2013 to 2024 that achieve performance close to unconstrained models of similar size on common‑sense reasoning and language understanding tasks. Instruction fine‑tuning with LoRA further enhances downstream usability, and the authors release the full pipeline—including dataset construction, training infrastructure, and evaluation code—for reproducible research.
By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu
arXiv:2607. 11889v1 Announce Type: cross Abstract: Large language models trained on unrestricted internet corpora inevitably embed information from the future, introducing lookahead bias that compromises the validity of backtests and causal inference in finance and the social sciences.
By Bryan Kelly, Semyon Malamud, Johannes Schwab, Teng Andrea Xu
The study examines how emotions are represented across layers of large language models (LLMs) by probing eight 1B–9B open‑weight models on three datasets (Twitter, Reddit, autobiographical narratives). It finds that the optimal probing layer varies systematically with the dataset, moving from near‑input layers to deeper layers, and that targeted forward‑pass interventions on these layers degrade performance more than random interventions. Additionally, the selected layers transfer across datasets and emotion categories, and early‑exit representations from these layers outperform full‑depth exits by an average of 6.9 percentage points.
By Tian Fang, Ga\"el Guibon, Davide Buscaldi
The paper introduces the Concept Field, a local drift field with pointwise uncertainty derived from sentence-embedding deltas in a text corpus. It scores candidate sentence transitions using the mean absolute z-distance (ζ) to quantify groundedness, offering a model‑agnostic, corpus‑attributable metric. The authors implement a Vector Sequence Database to store embeddings and delta metadata, and demonstrate the method’s effectiveness in detecting groundedness in the U.S. Code of Federal Regulations and novelty in Project Gutenberg, achieving strong selective classification performance under a triage policy.
By Nicholas S. Kersting, Vittorio Castelli, Chieh Ting Yeh, Xinzhu Wang, Saad Taame, Khaoula Allak
The paper presents a quantitative method for analyzing historical change in Italian media by reconstructing a diachronic corpus of about 600,000 articles from "La Repubblica" (1985‑2000). Using NLP to examine lexical and semantic patterns, and tools from complex systems, the authors identify shifts in media discourse, such as Italy’s transition from the First to the Second Republic and major international conflicts like the Gulf and Kosovo Wars. They find that war periods correspond to lower semantic entropy, indicating a more focused discourse.
By Dario Zarcone, Salvatore Miccich\`e, David Sanchez
KoNeoBench is a curated dataset designed to evaluate large language models’ understanding of Korean neologisms. It contains 1,785 recently attested Korean words from online news since 2020, each accompanied by usage examples, word‑formation analyses, and dictionary‑style definitions. The authors define four evaluation tasks, report results from recent models and a human baseline, and find that current LLMs struggle with recovering source components, distinguishing semantic categories, and generating accurate definitions.
By Soha Lee, Soojin Lee, Heesung Yang, Hyunju Song, Hyunji Lee, Jinsan An, Jeongwan Shin, Jin Hyun Park, Jun Lee, Hyeyoung Park, Kilim Nam
arXiv:2606.27314v2 Announce Type: replace
Abstract: To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive m...
By Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu