arXiv AI
Aug 7
MameLoshnLM: Yiddish Language Model and Evaluation Benchmark
arXiv:2608. 05850v1 Announce Type: cross Abstract: We present MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish.
Wiktionary as a Crowdsourced Lexicon for English Dialects
Related stories
Hugging Face Trending Papers
Aug 6
MameLoshnLM: Yiddish Language Model and Evaluation Benchmark
We present MameLoshnLM, the first open-source 8B-parameter language model built specifically for Yiddish. Despite Yiddish's rich textual tradition, its limited digital presence and the scarcity of reliable evaluation resources have constrained progress in Yiddish language modeling.
arXiv AI
Jul 24
slang.gr as a Large-Scale Crowdsourced Resource for Non-Standard Greek
arXiv:2607. 21255v1 Announce Type: cross Abstract: Slang is a central component of everyday language, reflecting linguistic creativity, social identity, and cultural change, yet its dy- namic and non-standard nature makes it difficult to model computationally.
arXiv Computation and Language
Sep 22
Large Language Models for Low-Resource Languages: A Conceptual Framework for an Electronic Explanatory Dictionary of the Tajik Language
arXiv:2608.04186v3 Announce Type: replace Abstract: This paper presents a conceptual framework for developing an electronic explanatory dictionary of the Tajik language using large language models (L...
arXiv Computation and Language
2d ago
Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries
The paper tackles the problem of automatically generating dictionary definitions for learner’s dictionaries, focusing on simplicity and clarity. It introduces a new evaluation framework that uses large language models as judges, validated against human annotators with comparable agreement levels. The authors also present an iterative simplification approach that produces definitions scoring highly on their criteria and exhibiting lexical simplicity.
arXiv Computation and Language
Sep 22
Revisiting Lexicon Evaluation in Unsupervised Word Discovery
The paper critiques the normalized edit distance metric used for evaluating lexicons derived from unsupervised word discovery, noting its bias toward large clusters and its failure to account for the distribution of true classes across clusters. It proposes two new metrics—one that weights cluster size when measuring within‑cluster consistency and another that evaluates how true words are spread across clusters—drawing on clustering theory. Experiments on synthetic and real‑world lexicons show that these combined metrics better correlate with ground‑truth distributions and are more robust to evaluation biases.
arXiv AI
Jun 3
Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models
arXiv:2606. 03165v1 Announce Type: cross Abstract: The language used by digital chat assistants such as ChatGPT can diverge from human expectations (misalignment).
arXiv Machine Learning
Jun 9
Community-Specific Slang and Entity Detection via Semantic Shift in Fine-Tuned Language Models
arXiv:2606. 07522v1 Announce Type: cross Abstract: We propose an unsupervised method of resolving slang, unique entities, and folklore from online communities by isolating words in the lexicon that have the highest magnitude of semantic shift.
arXiv Computation and Language
Sep 23
FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing
arXiv:2609.25298v1 Announce Type: new Abstract: Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely...
arXiv AI
Sep 18
KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms
KoNeoBench is a curated dataset designed to evaluate large language models’ understanding of Korean neologisms. It contains 1,785 recently attested Korean words from online news since 2020, each accompanied by usage examples, word‑formation analyses, and dictionary‑style definitions. The authors define four evaluation tasks, report results from recent models and a human baseline, and find that current LLMs struggle with recovering source components, distinguishing semantic categories, and generating accurate definitions.
Hugging Face Trending Papers
Sep 22
From Utterances to Networks: Modelling Slang Adoption and Diffusion Across Subreddits
Adoption and diffusion of neologisms in online communities have received renewed attention in recent years. As internet slang terms such as APT, referring to a K-pop song, and phrases such as Canon Ev...
arXiv AI
Sep 1
Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu Benchmark
arXiv:2608.30065v1 Announce Type: cross Abstract: Large language models (LLMs) demonstrate strong Helpfulness, Harmlessness, and Honesty (3H) alignment in English-centric settings, but these gains tr...