arXiv AI
Mimir: Large-scale Multilingual Concept Modeling
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
3d ago
To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs
The paper proposes a modular tokenizer framework for multilingual large language models, allowing the creation of language‑specific subtokenizers that match monolingual compression quality. It introduces a pretraining strategy that samples these subtokenizers to limit predictions to relevant vocabularies, enabling efficient training and inference. This approach reduces memory usage and speeds up inference without compromising performance.
arXiv AI
Aug 21
When Machines Speak: A Unified Generative Framework for Integrating Machine-Native Symbols into Pretrained Large Language Models
arXiv:2608. 19529v1 Announce Type: cross Abstract: Many real-world AI systems represent entities, behaviors, and structured information using discrete machine-native symbols rather than natural language.
arXiv Machine Learning
Aug 5
Speculative Decoding and the Curse of Multilinguality
arXiv:2605. 30580v2 Announce Type: replace-cross Abstract: Speculative decoding is a popular technique for large language model (LLM) inference, enabling faster generation by drafting multiple tokens with a smaller draft model.
arXiv Machine Learning
Jul 7
TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior
arXiv:2512. 20757v2 Announce Type: replace-cross Abstract: Tokenizers provide the fundamental basis through which text is represented and processed by language models (LMs).
arXiv AI
Aug 18
LLMs Get Smarter from Targeted Synthetic Multilingual Data
arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.
arXiv Machine Learning
Sep 11
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
The paper proposes an information-weighted cross‑entropy loss that rescales token contributions using TF‑IDF statistics, thereby emphasizing semantically informative tokens and down‑weighting ubiquitous ones. Experiments on five decoder‑only language models (1.1B–13B parameters) show consistent reductions in memorized substring length while maintaining perplexity and downstream performance. The method is architecture‑agnostic, adds less than 3% computational overhead, and can be integrated into existing training pipelines.