← Back to all news

Related stories

arXiv AI
Aug 11

Length-MAX Tokenizer for Language Models

arXiv:2511. 20849v2 Announce Type: replace-cross Abstract: We introduce a new tokenizer for language models that minimizes the average tokens per character, thereby reducing the number of tokens needed to represent text during training and to generate text during inference.

By Dong Dong, Weijie Su
llmsragnlpefficiency
More like this →
arXiv Computation and Language
Sep 17

SEA-LION-v4.8: A Technical Report

The report introduces Nemotron-SEA-LION-v4.8, a family of Southeast Asian language models built on NVIDIA Nemotron 3, featuring 30B-A3B and 120B-A12B variants with both base and post‑trained checkpoints. The models are fine‑tuned on Southeast Asian, reasoning, code, and multilingual parallel datasets, then further refined with supervised fine‑tuning and online on‑policy distillation. On the SEA‑HELM benchmark, the 30B-A3B model raises the overall SEA score from 46.06 to 51.57, while the 120B-A12B model jumps from 49.30 to 63.44, with the largest improvements seen in instruction following, natural language reasoning, and understanding across seven Southeast Asian languages.

By Ahmed Mohammad Dabeer (David Wang Dawei), Ahn Jeongmi (David Wang Dawei), Anocha Sutaveephamochanon (David Wang Dawei), Antonyrex Sajeban (David Wang Dawei), Aulia Adila (David Wang Dawei), Chan Hok Teng (David Wang Dawei), Adwin (David Wang Dawei), Cheng Zi Yi (David Wang Dawei), Nicholas Zhuang Ziyi (David Wang Dawei), Choa Hsueh Mei Esther (David Wang Dawei), David Ong Tat-Wee (David Wang Dawei), Evelyn Tan Chor Phin (Li Chunren), Heng Cheng Peng (Li Chunren), Jonathan (Li Chunren), Lee Chwan Ren (Li Chunren), Leong Wai Yi (Huang Wenzong, Raymond), Leong Wei Qi (Huang Wenzong, Raymond), Leslie Teo Eng Sipp (Huang Wenzong, Raymond), Liew Rachel (Huang Wenzong, Raymond), Limkonchotiwat Peerat (Huang Wenzong, Raymond), Montalan Jann Railey Estrada (Huang Wenzong, Raymond), Muhammad Ridzuan Bin Mokhtar (Huang Wenzong, Raymond), Nagarajan Karthik (Huang Wenzong, Raymond), Ng Boon Cheong (Huang Wenzong, Raymond), Raymond (Huang Wenzong, Raymond), Ngui Jian Gang (Chen Xiaowei), Nguyen Thanh Ngan (Chen Xiaowei), Tasawong Panuthep (Chen Xiaowei), Pereira Mark Gregory (Chen Xiaowei), Phang Shi Wei Benjamin (Chen Xiaowei), Poon Yip Hung (Chen Xiaowei), Joseph (Chen Xiaowei), Rengarajan Hamsawardhini (Chen Xiaowei), Siow Wei Kang Bryan (Chen Xiaowei), Tai Ngee Chia (Chen Xiaowei), Tan Choon Meng (Chen Xiaowei), Tan Le Min (Chen Xiaowei), Sheryl (Chen Xiaowei), Tan Siao Wei (Chen Xiaowei), Tan Yi Xian, Tee Jun Yun, Teng Kok Wai, Tjhi William Chandra, Tuchinda Pume, Wu Donghang, Yong Xianbin, Yosephine, Zhang Zhou
fine-tuningefficiency
More like this →
arXiv AI
6d ago

IronLLM: Forging Compact Edge-Native Language Models for Real-Time Embodied Intelligence

IronLLM-0.6B is a 654‑million‑parameter language model engineered for efficient on‑device inference, featuring a hybrid attention architecture, X‑MTP multi‑token prediction, and a lightweight verification head that yields a 1.48× decoding speedup. Trained on roughly 6.2 trillion tokens with a quality‑oriented pipeline and further refined via Multi‑Domain On‑Policy Distillation, the model adopts an Instruct‑Only design to meet low‑latency requirements. A lighter variant, IronLLM‑0.6B‑Light, replaces RMSNorm with Dynamic Tanh and streamlines costly components to enhance inference and quantization efficiency, offering a strong performance‑efficiency trade‑off for resource‑constrained deployment.

By Changdi Yang, Fengquan Jiao, Haochih Lin, Haoran Yang, Jing Xiao, Liangyu Huo, Suxin Lu, Tiance Chen, Wei Liu, Yinggan Xu, Yunxiang Lu, Zai Zheng, Zhirui Xie, Zhongyang Che, Ziyan Tang, Zuoxiang Zhao, Jian Yao
llmsroboticsefficiencydata-engineering
More like this →