TelecomGPT‑R1‑9B is an open‑source large language model designed specifically for telecom reasoning tasks. It was trained on a 67,427‑example supervised fine‑tuning corpus that covers protocol, knowledge, modeling, and fault reasoning, and further refined with a two‑stage post‑training process involving low‑rank adaptation and policy optimization. The model tops the GSMA open telco leaderboard and matches state‑of‑the‑art closed‑source reasoners across seven public telecom benchmarks.
By Bohao Wang, Chenwei Wu, Haoyu Li, Hang Zou, Yu Tian, Lina Bariah, Li Wei, Chongwen Huang, Yongliang Shen, Zhaoyang Zhang, Merouane Debbah
TelecomGPT‑R1 is an open‑source family of unified telecom reasoning models that address the limitations of existing telecom‑specific and general‑purpose LLMs. It is built on a four‑axis framework—protocol, knowledge, modeling, and fault—and trained on a curated corpus of 104,880 verified question‑answer pairs with chain‑of‑thought reasoning. After supervised fine‑tuning, dynamic sampling policy optimization with task‑routed rubric rewards is used to stabilize reinforcement learning across heterogeneous telecom tasks, achieving an 89.64% mean score on seven GSMA Open Telco Leaderboard benchmarks, surpassing leading proprietary models.
By Bohao Wang, Chenwei Wu, Hang Zou, Yu Tian, Lina Bariah, Li Wei, Chongwen Huang, Yongliang Shen, Zhaoyang Zhang, Merouane Debbah
arXiv:2608. 15436v1 Announce Type: new Abstract: Frontier AI models have advanced rapidly, but they still struggle with telecom-specific tasks.
By Farbod Tavakkoli, Roderic Paulk, Jorden Terrazas, Kenneth Church, Mark Austin, Louis Powell, Gregory Diamos, Lina Bariah, Syed Ali Raza Zaidi, Maryam Hafeez, Ali Maatouk, Imtiaz Karim
TeleTables is a benchmark that evaluates large language models on interpreting telecom tables from 3GPP specifications. It contains 2,220 tables in four formats and 500 human‑verified multiple‑choice questions that range from simple retrieval to multi‑step reasoning. Tests on 20 open‑weight LLMs show that closed‑book performance is limited by domain knowledge, while providing the table as context yields high accuracy that still drops with deeper reasoning, evidence scope, and structural complexity.
By Anas Ezzakri, Nicola Piovesan, Mohamed Sana, Antonio De Domenico, Fadhel Ayed, Haozhe Zhang
The paper evaluates lightweight, edge‑deployable large language models—Claude‑Haiku‑4.5, GPT‑5.4‑Mini, and Gemini‑3.1‑Flash‑Lite—on free‑text 5G domain knowledge and fault‑analysis tasks using three benchmarks (TeleQNA ORAN FT, 5G‑Faults FT, TeleInter FT). All models achieve at least 90% accuracy on fault diagnosis, but zero‑shot recall of 3GPP and O‑RAN specifications remains below 60%. Multi‑judge scoring yields a mean inter‑judge agreement of at least 0.90, and Gemini‑3.1‑Flash‑Lite emerges as the most efficient model for production telecom deployments.
By Rishiraj Sengupta, Sotiris Chatzimiltis, Mohammad Shojafar, Xiatian Zhu
The paper introduces CRAFT, a data‑centric fine‑tuning approach that aligns small language models (SLMs) for pre‑hoc reasoning in AI‑native 6G radio access networks (RAN). By automatically generating verified (input, trace, label) triplets and fine‑tuning with low‑rank adaptation, CRAFT achieves high accuracy and F1 scores on TRACTOR and IC xApp datasets while avoiding parse failures that plague RL methods like GRPO. It also reduces energy consumption by 59% compared to GRPO baselines, offering a more sustainable path to auditable AI in 6G RAN.
By Pranshav Gajjar, Vijay K Shah
The next generation of mobile networks is envisioned as fully AI-native, with AI-RAN architectures embedding small language models (SLMs) to perform reasoning over real-time telemetry. The state-of-th...
arXiv:2606. 05176v1 Announce Type: cross Abstract: While large language models (LLMs) show strong performance in natural language understanding and generation, their evaluation and adaptation to domain-specific constraints in telecommunications customer support remain limited.
By Lucas Tamic, Ilan Jaffeux-Cheniout, Xavier Marjou
arXiv:2609.36987v1 Announce Type: new
Abstract: Graph databases are increasingly queried through natural language, yet every existing benchmark evaluates isolated single-turn queries rather than the...
By Yuzhe Zhang, Weijie Zhu, Haolin Yang, Ziyun Zhang, Xianwei Xue, Mengke Chen, Qiutong Pan, Huaqian Cai
arXiv:2607. 20510v1 Announce Type: new Abstract: We introduce Telco-GAIA, a bilingual, multi-modal benchmark for evaluating tool-using agents on the data of a real-world telecommunications operator.
By Dmitrii Khizbullin, Zaid Alyafeai, Abdelrahman Eldesokey, Nourah AlSultan, Raghad Alshalan, David R. Pugh, Bernard Ghanem
SemVerBench is a benchmark that evaluates how well large language models (LLMs) understand and apply version-constraint resolution semantics, such as determining whether a version satisfies constraints like ^1.2.3 or >=2.0. The study finds that many models struggle with certain corner cases, with GPT‑5.1 performing poorly while Claude and Opus perform much better. The authors suggest that the failures stem from an activation/application gap rather than a lack of knowledge, and recommend that coding agents delegate version resolution to a dedicated resolver tool.
By Qibai Chen, Zeming Liu
CallScreenBench is a benchmark for evaluating small, on-device language models that act as phone secretaries, focusing on their ability to handle unknown inbound calls without a cooperative task. The benchmark measures owner endorsement through five call-and-note metrics, each paired with counter-metrics and uncertainty estimates, and includes guardedness diagnostics to identify safe, tool‑free proxies. Results across 4‑bit checkpoints of 0.6‑4 B parameter models show varying performance on service, recall, plausibility, and triage discrimination, highlighting trade‑offs between quality and guardedness.
By Jiaqi Gan, Haoyuan Tang, Jamey Z. Liang, Siying Chen, Ankit Raj, Kidus Zewde, Yuchen Zhou, Yuxin Zhang, Simiao Ren