arXiv Computation and Language

Training a Language Model End-to-End in Rust: An Experience Report

arXiv Machine Learning
Sep 11

Numbat: Building and Verifying a Self-Contained Machine-Learning Stack

The paper introduces Numbat, a self‑contained machine‑learning stack implemented entirely in Zig with no external runtime dependencies. It covers tensor computation, automatic differentiation, neural‑network modules, mixed precision, multi‑GPU training, data loading, and monitoring, and exposes a stable C ABI with over 1,400 entry points and bindings for six languages. The authors verify the stack against a reference implementation at multiple levels, uncovering ten silent recipe divergences, and demonstrate its practical capability by training a 25.9M‑parameter YOLOv8m detector on COCO 2017, achieving a competitive mAP score and matching single‑GPU performance.

By Thang Tran (CloudKites AI Lab, New South Wales, Australia), Lan Dang (Monash Business School, Monash University, Victoria, Australia)
arXiv AI
Aug 20

Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

The paper introduces Latent Space Refusal Anchoring (LSR‑Anchoring), a training‑free technique that extracts a refusal direction from English prompts and applies it to the residual stream of instruction‑tuned models at inference time. The primary variant, Mean‑Activation Steering (MAS), works across several architectures (Llama‑3‑8B, Llama‑3.1‑70B, Mistral‑7B‑Instruct, Qwen2.5‑7B), restoring safety for low‑resource African languages with minimal performance loss, while a refined SAE‑Derived Steering (SDS) further reduces KL divergence without degrading legitimate prompt performance. The method shows positive transfer for Yoruba, Igbo, Igala, and Hausa, but fails for Arabic, suggesting a geometric mismatch rather than a data scarcity issue.

By Godwin Abuh Faruna
arXiv Machine Learning
Aug 19

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

The paper evaluates how three large mixture‑of‑experts models (Alibaba, OpenAI, NVIDIA) can be fine‑tuned to reason in a low‑resource language, specifically Greek. Accuracy metrics show little change, but the authors uncover significant qualitative improvements: after supervised fine‑tuning, models reason in Greek on ~98% of items, with better grammaticality and retained general ability. Reinforcement learning with pre‑registered rewards further eliminates reasoning‑channel leaks and format skips, while the Greek‑reasoning habit remains robust to an accuracy‑only gradient.

By Ayoub Kirouane, Christos Petrocheilos
arXiv Machine Learning
Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y
arXiv Computation and Language
4d ago

ufakzeka-1: Building and Evaluating a 151M-Parameter Turkish Language Model from Scratch

The paper introduces ufakzeka-1, a 151‑million‑parameter Turkish language model trained from scratch on 13.5 B tokens. It details the tokenizer, a three‑stage pretraining schedule, post‑training data augmentation, and a comprehensive evaluation suite that includes release gates, a rule‑checked conversation sweep, and hand tests—all with prompts excluded from training data. The authors report three key findings about safety gate performance, training‑seed variance, and data‑round effects, and they release the model weights, data recipe, evaluation code, and spend ledger under Apache‑2.0.

By Sait Furkan Teke (ufak AI)
arXiv Machine Learning
Aug 27

Amplifying, Not Learning: The Price of Out-of-Distribution Generalization in AI-Text Detection

The paper shows that AI‑text detectors, rather than learning a clear AI‑versus‑human boundary, amplify an inherited predictability axis that already exists in language models. This amplification causes detectors to over‑flag fluent, formal human writing while missing high‑temperature AI outputs, and the bias persists across languages, code, and detector architectures. A training‑free operator can relocate the bias but cannot erase it, underscoring that the unfairness is a structural cost of out‑of‑distribution generalization.

By Alexander Smirnov
arXiv Computation and Language
Sep 1

Manac\'a-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation

Manacá-1B is a 1.72‑billion‑parameter, open decoder‑only language model trained from scratch for Brazilian Portuguese, released with a fully containerized, reproducible training pipeline and complete logs. The authors evaluate it against nine open baselines on four Portuguese benchmarks, reporting standard errors and paired significance tests, and find that Manacá-1B outperforms smaller models on LAMBADA‑PT while remaining competitive on commonsense completion. They also uncover a tokenizer‑related evaluation pitfall that can drastically lower accuracy and provide a simple fix, releasing all code, logs, and corrected tokenizer for full reproducibility.

By Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andre Machado Porto