arXiv AI

HEARTS: Benchmarking LLM Reasoning on Health Time Series

arXiv:2603. 06638v3 Announce Type: replace-cross Abstract: The rise of large language models (LLMs) has shifted time series analysis from narrow analytics to general-purpose reasoning.

arXiv AI

Sep 7

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

MMTClinic is a new benchmark that tests large language models on complex reasoning and question‑answering tasks involving clinical time‑series data. It combines text, medical images, and multivariate physiological signals to create 30,000 QA pairs—including 15,000 multiple‑choice and 15,000 open‑ended questions—in five languages (English, Hindi, Bengali, Marathi, and Tamil). The benchmark covers mortality prediction, heart‑rate forecasting, and SOFA score estimation, and evaluates 13 state‑of‑the‑art LLMs across zero‑shot, few‑shot, and chain‑of‑thought settings, revealing significant performance gaps across tasks, languages, and modalities.

By Sourav Malakar, Harshit Nigam, Akash Ghosh, Sriparna Saha, Amlan Chakrabarti, Saptarsi Goswami, Priti Singh

llms nlp multimodal benchmarks

arXiv AI

Sep 15

TimeThink: Eliciting Compositional Reasoning in Timeseries Large Language Models

arXiv:2609.13457v1 Announce Type: new Abstract: Timeseries multimodal large language models (TS-MLLMs) have recently begun leveraging the reasoning capabilities of large language models (LLMs) for qu...

By Sudarshan Regmi, Arvind Pillai, Yu Yvonne Wu, Yuliang Chen, Bibek Panthi, Tess Z. Griffin, Michael V. Heinz, Lisa Marsch, Nicholas C. Jacobson, Andrew Campbell

llms reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Sep 10

HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care

arXiv:2609.06976v1 Announce Type: new Abstract: As medical wearables become integrated into daily chronic disease care, effectively interpreting longitudinal monitoring data is essential for patients...

By Yuchen Niu, Yanan Ma, Srinivasan Nandakumar, Maolin Chen, Viktor Schlegel, Kexin Wei, Ling Cheng, Anna Bird, Anil Anthony Bharath, Siew-Kei Lam

llms agents nlp benchmarks safety

arXiv AI

Jun 3

ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning

arXiv:2606. 02802v1 Announce Type: new Abstract: Large language models (LLMs) exhibit strong natural-language reasoning abilities for clinical decision support, but struggle to effectively model structured longitudinal electronic health records (EHRs).

By Bo-Hong Wang, Baicheng Peng, Ruilin Wang, Jun Bai, Ziyang Song, Yue Li

llms multimodal benchmarks safety

arXiv Computation and Language

Aug 28

BALMS: Benchmarking Agentic LLMs for Longitudinal Mental Health Sensing

BALMS is a benchmark for evaluating large language model (LLM) agents that analyze longitudinal wearable data to predict mental‑health wellbeing scores and generate evidence‑grounded rationales. It covers three real‑world datasets, two task families (score prediction and rationale generation), and tests five LLM backbones across open‑ and closed‑source paradigms. The study finds that zero‑shot agents rarely beat a simple mean baseline, and while chain‑of‑thought prompting helps reasoning, it does not ensure temporal grounding or numerical accuracy.

By Yu Yvonne Wu, Arvind Pillai, Yuliang Chen, Yuwei Zhang, Sudarshan Regmi, Tess Z. Griffin, Michael V. Heinz, Lisa A. Marsch, Nicholas C. Jacobson, Andrew Campbell

llms agents benchmarks

arXiv AI

Sep 2

LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning

arXiv:2601.13880v2 Announce Type: replace Abstract: Personalized lifestyle health analysis requires long-horizon, multi-dimensional reasoning over heterogeneous lifestyle signals, and recent advances...

By Ye Tian, Zihao Wang, Onat Gungor, Xiaoran Fan, Tajana Rosing

llms benchmarks

arXiv AI

Jul 29

A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series

arXiv:2607. 25947v1 Announce Type: new Abstract: Question answering (QA) over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications.

By Frank Nie, Ethan B Liu, Yuan Zhu, Wei Fan, Jindong Han

llms rag nlp efficiency multimodal benchmarks safety

arXiv Computation and Language

Sep 7

WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

WearableQA is a new benchmark that tests AI systems on health reasoning using real-world wearable data from 200 users, each with up to 500 days of daily measurements. It contains 4,084 ten‑option multiple‑choice questions derived from wearable time series, blood biomarkers, and demographics, and is organized into 16 question types that distinguish data‑driven computation from physiological interpretation and single‑signal from cross‑signal reasoning. Evaluation of 14 large language models shows wide performance gaps, indicating that the benchmark remains challenging and useful for diagnosing model capabilities.

By Ji Soo Lee, Xilun Chen, Pierce Chuang, Ashish Shenoy, Jason Wei, Dohwan Ko, Hyunwoo J. Kim, Benoit Corda

llms benchmarks

arXiv Machine Learning

Sep 22

Large language models in medical time series analysis

arXiv:2609.22262v1 Announce Type: cross Abstract: Medical time series (MedTS), including electrocardiograms (ECG), electroencephalograms (EEG), photoplethysmography (PPG), and vital-sign recordings,...

By Yu Han, Cigdem Beyan, Xiang Zhang, Xiaofeng Liu, Nan Liu, Jimeng Sun, Shenda Hong, Cheng Ding, Vittorio Murino

llms

arXiv Machine Learning

Jun 2

OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction

arXiv:2510. 17532v2 Announce Type: replace-cross Abstract: Predicting cancer treatment outcomes requires models that are both accurate and interpretable, particularly in the presence of heterogeneous clinical data.

By Raghu Vamshi Hemadri, Geetha Krishna Guruju, Kristi Topollai, Anna Ewa Choromanska

llms nlp reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Sep 22

Fathom-Vaidya: Advancing Medical Reasoning with Rubric-Based Rewards

arXiv:2609.24480v1 Announce Type: cross Abstract: Deploying Large Language Models (LLMs) in healthcare requires robust performance across two complementary dimensions - diagnostic reasoning: the conv...

By Kalash Shah, Kunal Singh, Snehan J, Shreyas Singh

llms reinforcement-learning benchmarks

arXiv AI

Jun 12

Representing Time Series as Structured Programs for LLM Reasoning

arXiv:2606. 12481v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong reasoning and instruction-following capabilities, making them potentially powerful tools for time-series analysis.

By Jaeho Kim, Changhun Oh, Seokhyun Lee, Irina Rish, Changhee Lee

llms nlp

Related stories