← Back to all news

Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

28,685 stories · RSS feed

arXiv AI
Oct 1

Functional Subspace, where language models can use vector algebra to solve problems

The paper proposes that large language models (LLMs) encode high‑level concepts as linear directions within their activation space and that they can use subspaces and vector algebra to perform tasks. By analyzing functional modules and residual streams during in‑context learning (ICL), the authors find that LLMs can create evidence‑accumulating subspaces and solve ICL tasks through simple algebraic operations within those subspaces.

By Jung H. Lee, Sujith Vijayan
llmsrag
More like this →
arXiv AI
Oct 1

Speech-based Psychological Crisis Assessment using LLMs

The paper presents an LLM-based framework for automatically classifying crisis levels in psychological support hotlines, addressing variability in human judgments and staffing constraints. It introduces a paralinguistic injection method that embeds non‑verbal emotional cues into transcripts, allowing the model to consider acoustic nuances. A reasoning‑enhanced training strategy encourages the model to produce diagnostic reasoning chains, which regularizes and improves classification, achieving a macro F1‑score of 0.802 and accuracy of 0.805 in 5‑fold cross‑validation.

By Terumi Chiba, Yang Luo, Ziyun Cui, Yongsheng Tong, Chao Zhang
llms
More like this →
arXiv AI
Oct 1

LSR-Ben: A Logical and Scientific Reasoning Benchmark for Evaluating Process Reward Models

The paper introduces LSR‑Ben, a benchmark designed to evaluate process reward models (PRMs) on scientific and logical reasoning tasks, addressing a gap left by existing math‑focused benchmarks. Experiments on 22 models reveal that PRMs and LLMs perform poorly in non‑mathematical domains, with LLMs tending to over‑identify errors while PRMs tend to overlook them. LSR‑Ben aims to spur research that broadens PRM applicability and improves LLM reasoning.

By Zhouhao Sun, Xuan Zhang, Xiao Ding, Bibo Cai, Li Du, Kai Xiong, Xinran Dai, Fei Zhang, weidi tang, Zhiyuan Kan, Yang Zhao, Bing Qin, Ting Liu
llmsbenchmarks
More like this →
arXiv AI
Oct 1

A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?

A2Z GameSpec-Bench introduces a benchmark of 100 long‑form Game Design Documents (GDDs) to evaluate how faithfully coding agents can generate complete games from detailed specifications. The benchmark measures faithfulness by checking that the game satisfies the GDD requirements and preserves the relationships among them, using a dependency‑aware contract and a combination of source‑code inspection and agent‑generated test policies. Evaluations show that current agents struggle to meet interdependent requirements, but requirement‑specific feedback improves GDD fidelity by 10.9% after two revision rounds.

By Seonho Lee, Wonryeol Jeong, Alberto Cereser, Inha Kang, Hyeonjong Kim, Seungmin Kwak, Dongmin Park
llmsagentsbenchmarks
More like this →
arXiv AI
Oct 1

Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agents

The paper introduces Trajectory-guided Joint Policy Optimization (TJPO), a reinforcement‑learning framework that explicitly encourages diversity in the trajectories of large‑language‑model agents. By defining task‑specific trajectory descriptors, TJPO measures and optimizes diversity as a set‑level function, avoiding population‑based training. Experiments on Sokoban and ALFWorld demonstrate that TJPO yields diverse, interpretable behaviors while preserving strong task performance.

By Huaiyu Fu, Heng Cao, Hao Wang, Jian Ya, Tao Chen
llmsagentsreinforcement-learning
More like this →
arXiv AI
Oct 1

BARRAC: Adaptation of an English Aspect-based Sentiment Analysis Approach for Classification Tasks in Arabic Dialects

The paper introduces BARRAC, a method that adapts an English aspect‑based sentiment analysis framework for Arabic dialect classification tasks. It replaces English consumer‑review attribute pools with Arabic linguistic markers for sentiment, sarcasm, and dialect identification, and swaps noisy self‑training for a two‑stage training process. Evaluated on five Arabic dialect datasets, BARRAC achieves a mean macro‑F1 of 63.93%, surpassing the best few‑label state‑of‑the‑art by 3% and outperforming GPT‑4o on four of the five tasks, while error analysis highlights remaining challenges.

By Ali Almutairi, Gelareh Mohammadi, Imran Razzak, Aditya Joshi
llmsnlpbenchmarkssafety
More like this →
arXiv Computation and Language
Oct 1

LexReward: A Taxonomy-Driven Reward Framework for Legal Language Models

LexReward is a taxonomy-driven reward framework designed for legal language models, evaluating responses across three dimensions: Style (lexical and syntactic quality), Element (legal subjects, facts, statutes, and decisions), and Chain (order, completeness, correctness, and non-redundancy of reasoning). The framework introduces rubrics that define evaluation criteria and quality levels for each dimension, generating pairwise preference data used for Direct Preference Optimization (DPO) and reward-model training. Experiments demonstrate that rubric-based rewards effectively differentiate legal responses of varying quality, and that DPO training improves performance across all dimensions; the resulting reward models (LexRM) enable reinforcement learning to enhance policy performance in each specific dimension without needing reference answers.

By Yida Cai, Xin Dai, Bingxiang He, Huiyuan Xie, Yuxiao Ye, Zhenghao Liu, Yang Bai, Zhiyuan Liu
llmsreinforcement-learningsafety
More like this →