The survey reviews how Large Language Models (LLMs) are being used in High‑Performance Computing (HPC) programming, covering code generation, parallelization, frameworks, evaluation, and broader challenges. It finds that general‑purpose LLMs perform adequately on serial and OpenMP‑style tasks but struggle with distributed MPI workloads, while domain‑specialized models achieve higher accuracy yet are limited in scope and evaluation. The authors argue that LLMs will not replace HPC experts soon but can act as powerful collaborators, provided richer datasets, integration with performance tools, rigorous evaluation, and governance are developed.
By Strahinja Ljaljevic, Josep Jorba, Sergio Iserte
arXiv:2606. 07527v1 Announce Type: cross Abstract: The prevailing paradigm for training LLMs has evolved to rely on a massive post-training phase consisting of SFT and RL.
By Michael Hassid, Yossi Adi, Roy Schwartz
arXiv:2609.10226v1 Announce Type: new
Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in reasoning and code generation, raising the prospect that they could assist in...
By Leilei Ding, Shumin Wang, Yuting Huang, Fanqi Wan, Yinmin Zhang, Qi Han, Yiming Xu, Feiyuan Zhang, Xiaomeng Chu, Guoliang You, Wuyang Zhang, Daxin Jiang, Yanyong Zhang
arXiv:2605. 30407v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance on general tasks, while often struggling to adapt to specialized domains without high-quality domain-specific data.
By Yujie Luo, Xiangyuan Ru, Jingsheng Zheng, Jingjing Wang, Yuqi Zhu, Jintian Zhang, Runnan Fang, Kewei Xu, Ye Liu, Zheng Wei, Jiang Bian, Zang Li, Shumin Deng
OptSkills is an archetype‑centric agent that learns and reasons about optimization problems using large language models. It clusters problems by underlying archetypes, explores diverse modeling and solver configurations within each cluster, and distills successful trajectories into reusable workflow‑level skills. The system achieves state‑of‑the‑art accuracy on multiple datasets, outperforming prior methods on challenging benchmarks such as MIPLIB‑NL and OOD NLCO.
By Haochen Yang, Ke Zhao, Mengyuan Ma, Xingyu Lu, Xiangfeng Wang, Hong Qian
arXiv:2607. 17624v1 Announce Type: new Abstract: Transformers are remarkably versatile and their design is largely consistent across a variety of applications.
By Damien Teney, Liangze Jiang, Hemanth Saratchandran, Simon Lucey
The paper investigates how the skill paradigm—using modular abilities within agentic frameworks—works with small to medium open‑source language models (270 M–80 B parameters) in industrial, resource‑constrained settings. A large‑scale empirical study on an insurance‑claim classification task and a set of Python programming problems shows that very small models often fail to retrieve or select correct skills, while larger 30 B–80 B models benefit significantly from skill augmentation. The authors also present a deployment‑oriented GPU profile, reporting VRAM time occupancy (GB·min) across models, to help characterize GPU‑memory costs for different model scales.
"whyItMatters":"The study provides concrete evidence on how skill augmentation scales with model size, task complexity, and GPU budget, offering practical deployment insights for industrial applications using open‑source language models."
By Yangjie Xu, Lujun Li, Lama Sleem, Niccolo Gentile, Yewei Song, Yiqun Wang, Siming Ji, Wenbo Wu, Hui Huang, Radu State
arXiv:2609.09691v1 Announce Type: new
Abstract: When training data are limited, increasing parameter count is not the only way to improve language-model performance. A small parameter set, when repea...
By Tingshuo Fan, Hongtao Mu, Tianyu Zhou, Hansen Liu, Tao Ji
arXiv:2609.37891v1 Announce Type: cross
Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...
By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko
arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.
By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad
arXiv:2606. 30441v1 Announce Type: cross Abstract: A rigorous formalization of system requirements is a fundamental prerequisite for the verification of Multi-Agent Systems (MAS).
By Marco Aruta, Francesco Improta, Vadim Malvone, Aniello Murano, Vladana Perlic
arXiv:2509. 16456v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used in various domains, showing impressive potential on different tasks.
By Jiahao Yu, Zelei Cheng, Xian Wu, Xinyu Xing