The paper PRiME investigates how large language models (LLMs) can perform pragmatic reasoning in the mental health domain. It introduces three datasets and tasks, evaluates 12 LLMs—including the specialized MentaLLaMA model—and finds that performance is comparable to human evaluation, with Mistral‑7B achieving the best results. The study also examines error asymmetry and uses first‑order attention rollout to analyze MentaLLaMA‑7B’s architectural behavior, urging careful analysis for high‑stakes applications.
arXiv:2510.03223v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) tend to generate a long reasoning chain when solving complex tasks. However, as the reasoning chain extends, cri...
The paper introduces PsyLLM, a large language model specifically designed for mental health counseling that integrates diagnostic reasoning and multiple therapeutic frameworks. It employs an automated data synthesis pipeline that transforms Reddit mental‑health posts into multi‑turn counseling dialogues, guided by DSM/ICD standards and therapies such as CBT, ACT, and psychodynamic approaches. A new benchmark and evaluation protocol show that PsyLLM outperforms baseline models, with results validated by professional counseling experts.
The paper presents a taxonomy of 28 cognitive elements derived from cognitive science to analyze reasoning in large language models (LLMs). Using a fine‑grained evaluation framework, the authors compare 192,000 model traces with 1,346 human think‑aloud traces, finding that while models deploy similar cognitive elements, they differ in usage patterns. The study introduces test‑time reasoning guidance that improves model accuracy on ill‑structured problems by 9.7 percentage points and highlights a research gap in self‑awareness, advocating for a shared vocabulary to diagnose failures and guide robust model development.
arXiv:2609.22700v1 Announce Type: new Abstract: Step-level reasoning evaluators are commonly based on autoregressive language models, whose causal attention restricts each step representation to the...
MindEval2 is a new evaluation framework for AI chatbots in mental health support, comprising Mental Health Quality Rubrics (MHQR) and the MindSim patient simulation harness. MHQR, developed with PhD-level psychologists, provides explicit, actionable criteria for assessing response quality, while MindSim offers realistic, longitudinal patient interactions with a memory component. The study evaluated 18 models, finding that although surface-level quality has improved, guidance calibration remains the main failure across all models.
The paper explores how Large Language Models (LLMs) such as GPT-4o-mini, Gemini-1.5-pro, and LLaMA can be used to automatically simplify complex medical texts into plain language, a process known as Plain Language Adaptation (PLA). It evaluates different prompting strategies, fine‑tunes models with QLoRA, and incorporates Mixture‑of‑Agents (MoA) techniques to improve adaptability and robustness. The results show that LLM‑driven PLA effectively makes healthcare information more comprehensible while preserving essential content.
The article surveys how large language models (LLMs) are being applied to mental health, outlining a three‑phase evolution: Phase I uses LLMs as passive information tools and pattern recognizers for assessment; Phase II employs them as empathetic conversationalists for stateless, in‑the‑moment interactions; Phase III aims to create longitudinal, personalized companions that act as stateful cognitive agents. It systematically reviews core technologies, agent architectures (Profile, Memory, Reasoning, Planning), and the datasets and benchmarks that support this progression, offering a coherent narrative and roadmap for future research. The survey also provides a curated resource list at https://github.com/Emo-gml/Awesome-Mental-Health-LLMs.
PragReST is a self‑supervised framework that enhances large language models’ pragmatic reasoning by generating counterfactual reasoning traces and fine‑tuning models with supervised learning and reinforcement learning, all without human‑labeled data. The method constructs pragmatic QA data, trains models to internalize counterfactual reasoning, and demonstrates significant accuracy gains on four pragmatic benchmarks compared to backbone models and other baselines. Error analysis shows that counterfactual reasoning reduces mistakes from failing to contrast observed utterances with plausible alternatives, and removing it markedly degrades performance while preserving out‑of‑domain general‑knowledge and mathematical reasoning.
Natural language understanding often depends on meanings that are implied rather than explicitly stated, requiring pragmatic reasoning. Despite strong performance on math and logical reasoning, large language models (LLMs) still struggle with making pragmatic inferences, often choosing literal interpretations.
The paper proposes a fragment‑based reasoning framework for large language model–based machine translation. It extracts parallel source‑target fragments from retrieved similar examples and uses these fragments as intermediate reasoning traces to generate the final translation. Experiments with the Qwen3 model across six languages and multiple domains show that this approach outperforms standard k‑shot or basic drafting methods.
PetQA is a Korean long‑form question‑answering benchmark designed to assess veterinary knowledge and clinical reasoning in large language and vision‑language models. It comprises 10,076 text‑only and 8,751 multimodal QA pairs about dogs and cats, with expert veterinarian answers, and a test split called PetQA‑Bench that includes question type and clinical condition annotations. The study evaluates 18 models across zero‑shot, retrieval‑augmented generation, and supervised fine‑tuning settings using ROUGE, BERTScore, and LLM‑as‑a‑judge metrics, revealing current models’ strengths and limitations and underscoring the need for better adaptation methods for clinically reliable veterinary AI; translated versions in five languages are also provided.
arXiv:2610.09209v1 Announce Type: new Abstract: Substance Use Disorder (SUD) counseling requires patient responses that reflect underlying cognitive states such as beliefs, coping strategies, and rea...