The paper reports on adapting the Qwen3 language model to Tajik using small LoRA adapters across sizes from 0.6B to 8B. While the adapters consistently lowered Tajik text loss by 0.60 to 1.45 nats per token, they did not improve Tajik reading comprehension on the Belebele benchmark; in some cases accuracy even dropped. A more extensive 8B training pipeline, including 375 million tokens of raw Tajik text and supervised fine‑tuning, produced a new model (e3) that outperformed Qwen3‑8B on Belebele by 11.4 points, though it slightly degraded English MMLU performance.
By Tohir Saidzoda, Madina Abdusalomzoda, Nasimjon Rajabov, Akhrorkhon Ikromov, Komron Giyosiev, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
Saidzoda‑Tajik‑E5‑Large is a 560 M‑parameter Tajik text embedder fine‑tuned from multilingual‑e5‑large‑instruct. The authors generated 235,140 question‑passage pairs from Tajik laws, tax documents, Wikipedia and web text, trained for one epoch with hard negatives, and mixed in English and Russian pairs to preserve multilingual performance. On unseen Tajik law and tax passages the model retrieves the correct answer in the top ten for 93.6 % and 94.5 % of questions, respectively, outperforming the base model and matching larger models on similar tasks.
By Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
The study evaluates the effectiveness of small, fine‑tuned open large language models (LLMs) for predicting bug severity in Eclipse bug reports, comparing them against traditional TF‑IDF classifiers and a fine‑tuned CodeBERT encoder. Using a balanced sample of 20,000 reports, eleven model configurations—including zero‑shot and QLoRA‑fine‑tuned Llama 3 8B and Qwen 2.5 7B—were tested on 2,000 held‑out reports, revealing that fine‑tuned LLMs achieve macro‑F1 scores around 0.52, comparable to CodeBERT, while zero‑shot LLMs lag behind TF‑IDF baselines. The analysis also highlights class‑specific performance differences and the impact of duplicate reports on model evaluation.
By Asset Kanatov, Talgar Bayan, Askar Boranbayev
arXiv:2609.16145v1 Announce Type: new
Abstract: We study a practical question: can a small correction module fix errors in a frozen language model's outputs without degrading its base capabilities? W...
By Gautam Kishore
The paper introduces SafeStart, a technique for Mixture-of-Experts (MoE) language models that mitigates the safety degradation caused by active expert reduction during serving. By running all experts for the prompt and the first eight generated tokens before switching to a reduced set, SafeStart largely restores the model’s refusal to harmful requests while preserving most of the speedup achieved by expert reduction. Experiments on several MoE models show that SafeStart maintains high offline decode speedups (≈85% of the original) and limits the increase in harmful responses to within a few percentage points, even under jailbreak prompts.
By Erfan Baghaei Potraghloo, Saeid Shokoufa, Seyedarmin Azizi, Arya Fayyazi, Souvik Kundu, Mehdi Kamal, Massoud Pedram
arXiv:2610.18420v1 Announce Type: cross
Abstract: We trained Agaram 2.5B, a 2.5-billion-parameter Tamil-English language model, from scratch on about 50 billion tokens (58% Tamil, the rest English, m...
By Ramchand Kumaresan, Madhusudhanan G, Kaushik Kezendirane