arXiv Computation and Language October 22, 2026 By Ibrahim Khan, Thai Van Nguyen, Yi Xia, Pratch Suntichaikul, Mustafa Can Gursesli, Daniel Fernandez Galeote, Juho Hamari, Ruck Thawonmas

A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting

Read the original on arXiv Computation and Language →

The paper introduces Role-Enhanced Zero-Shot Chain-of-Thought (RE‑ZS‑CoT), a prompting framework that incorporates domain‑specific expert roles to improve reasoning in large language models without task‑specific examples. Two variants are evaluated: a static version that tests eight predefined roles across arithmetic, commonsense, and reading comprehension tasks, and a dynamic version where the model selects the most suitable role for each benchmark. Across 20 LLMs and seven benchmarks, RE‑ZS‑CoT consistently outperforms zero‑shot and zero‑shot‑CoT baselines, achieving up to 44 percentage‑point gains and demonstrating strong alignment between static and dynamic approaches, while ablation studies confirm the importance of task‑role alignment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Jul 17

Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation

arXiv:2607. 14109v1 Announce Type: cross Abstract: Probing the capabilities of Large Language Models (LLMs) and building robust solutions for Multiple-Choice Question Answering (MCQA) remain central challenges in natural language understanding.

By Inder Preet, Shuxin Lin, Dhaval Patel

llms nlp benchmarks

arXiv AI

Oct 1

When a Kindergartener Solves Calculus: Measuring Capability Leakage in Role-Prompted Reasoning Models

arXiv:2609.39846v1 Announce Type: cross Abstract: We investigate the problem of role-capability leakage (RCL), in which a role-prompted reasoning model generates convincing in-role text while continu...

By Pakhapoom Sarapat, Saksorn Ruangtanusak, Kunat Pipatanakul, Pittawat Taveekitworachai

llms benchmarks safety

arXiv AI

Aug 5

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve

arXiv:2608. 03550v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting remains the standard baseline for evaluating models' reasoning abilities.

By Denys Pushkin, Albert Q. Jiang, Aryo Lotfi, Colin Sandon, Emmanuel Abb\'e

llms

arXiv Computation and Language

Aug 27

Rethinking the Multilingual Reasoning Gap with Layer Swap

The study investigates the performance gap between native-language reasoning and English-pivoted reasoning in large language models. By creating extensive multilingual reasoning datasets and fine‑tuning specialists on Qwen/Qwen3-8B-Base, the authors find that the native reasoning gap is much smaller (1.9–3.5%) than previously reported. They analyze weight‑space changes, discover a language‑agnostic reasoning core in the middle layers, and propose a Layer Swap technique that transfers these mid‑layer updates from an English specialist to native specialists, effectively closing most of the gap while maintaining native chain‑of‑thought output.

By Maxence Lasbordes, Am\'elie Chatelain, Djam\'e Seddah

llms fine-tuning

arXiv AI

Jun 26

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

arXiv:2606. 26671v1 Announce Type: new Abstract: Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization.

By Qiaobo Hao, Yangqian Wu, Shunyi Wang, Zhongjian Zhang, Ziqun Li, Yayin He, Muqing Li, Chen Zhong

llms reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv AI

Oct 21

Attention-Aligned Reasoning for Large Language Models

arXiv:2510.03223v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) tend to generate a long reasoning chain when solving complex tasks. However, as the reasoning chain extends, cri...

By Hongxiang Zhang, Yuan Tian, Tianyi Zhang

llms benchmarks safety

One email a morning, machine-written

Related stories

Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation

When a Kindergartener Solves Calculus: Measuring Capability Leakage in Role-Prompted Reasoning Models

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve

Rethinking the Multilingual Reasoning Gap with Layer Swap

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

Attention-Aligned Reasoning for Large Language Models