arXiv AI
MoMaTune: Model, Match, Then Tune Language Models for Numerical Records
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 1
Manac\'a-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation
Manacá-1B is a 1.72‑billion‑parameter, open decoder‑only language model trained from scratch for Brazilian Portuguese, released with a fully containerized, reproducible training pipeline and complete logs. The authors evaluate it against nine open baselines on four Portuguese benchmarks, reporting standard errors and paired significance tests, and find that Manacá-1B outperforms smaller models on LAMBADA‑PT while remaining competitive on commonsense completion. They also uncover a tokenizer‑related evaluation pitfall that can drastically lower accuracy and provide a simple fix, releasing all code, logs, and corrected tokenizer for full reproducibility.
Hugging Face Trending Papers
Aug 31
Manacá-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation
Brazilian Portuguese remains under-served by open language models, and the few that exist are difficult to reproduce and are often compared without measures of uncertainty. We release Manacá-1B, an op...
arXiv AI
Jun 11
Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
arXiv Machine Learning
Jul 7
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.
arXiv Machine Learning
Aug 19
TokEval: A Tokenizer Evaluation Suite
TokEval is a tokenizer evaluation suite that introduces metrics beyond traditional fertility and compression rate, capturing linguistically and structurally meaningful properties such as UTF‑8 character boundary integrity and digit place‑value alignment for mathematics. The authors validate these metrics by pretraining language models with varied tokenizers and measuring downstream performance on bits‑per‑byte and benchmarks covering linguistic understanding, mathematical reasoning, and code generation. Their results show that information‑theoretic metrics predict language modeling performance, while structure‑sensitive metrics correlate with task accuracy, suggesting TokEval can guide tokenizer selection more principledly.
arXiv Machine Learning
Jul 22
A Better Start for Language Models: Domain-Conditional Position Offsets
arXiv:2607. 18302v1 Announce Type: new Abstract: Autoregressive language models are least accurate at the beginning of a sequence, where little context forces reliance on a generic pretraining prior.