← Back to all news
arXiv Computation and Language By Tohir Saidzoda, Madina Abdusalomzoda, Nasimjon Rajabov, Akhrorkhon Ikromov, Komron Giyosiev, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva

Adapting Qwen3 to Tajik from 0.6B to 8B: Lower Text Loss Did Not Mean Better Understanding

Read the original on arXiv Computation and Language →

The paper reports on adapting the Qwen3 language model to Tajik using small LoRA adapters across sizes from 0.6B to 8B. While the adapters consistently lowered Tajik text loss by 0.60 to 1.45 nats per token, they did not improve Tajik reading comprehension on the Belebele benchmark; in some cases accuracy even dropped. A more extensive 8B training pipeline, including 375 million tokens of raw Tajik text and supervised fine‑tuning, produced a new model (e3) that outperformed Qwen3‑8B on Belebele by 11.4 points, though it slightly degraded English MMLU performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Computation and Language
2d ago

Saidzoda-Tajik-E5-Large: Fine-Tuning a 560M Embedder for Tajik Retrieval

Saidzoda‑Tajik‑E5‑Large is a 560 M‑parameter Tajik text embedder fine‑tuned from multilingual‑e5‑large‑instruct. The authors generated 235,140 question‑passage pairs from Tajik laws, tax documents, Wikipedia and web text, trained for one epoch with hard negatives, and mixed in English and Russian pairs to preserve multilingual performance. On unseen Tajik law and tax passages the model retrieves the correct answer in the top ten for 93.6 % and 94.5 % of questions, respectively, outperforming the base model and matching larger models on similar tasks.

By Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
llmsragfine-tuning
More like this →
arXiv Computation and Language
4d ago

Saidzoda-Tajik-LFM-2.6B: Adapting a 2.6B Hybrid Model to Tajik Tool Calling, and What Continued Pretraining Takes Away

Saidzoda‑Tajik‑LFM‑2.6B is a 2.6‑billion‑parameter hybrid model adapted for Tajik tool calling. The adaptation uses two stages: LoRA continued pretraining on 1.49 million cleaned Tajik web documents followed by one epoch of supervised fine‑tuning on machine‑translated Tajik tool calls, English tool calls, and Tajik instructions. On a 174‑item Tajik tool‑calling test set, strict full‑call accuracy rises from 20.69 % for the base model to 64.14 %, while English performance remains unchanged; however, continued pretraining removes the native tool‑call format and part of the skill, and the model shows limited general assistant capabilities.

By Tohir Saidzoda, Nekruz Maxudov, Bedil Karimov, Abubakr Abdullozoda, Amin Jalilov, Jonibek Kishvarov, Rahmonali Sharipov, Mariyamkhon Shodyeva
agentsfine-tuningbenchmarks
More like this →