arXiv Machine Learning

Toward Generalizable Cognitive Impairment Detection with Speech-Based Multimodal Large Language Models

arXiv:2607. 21496v1 Announce Type: cross Abstract: Cognitive impairment (CI) is a growing public health concern.

arXiv AI

Jul 1

Listening Between the Lines: Joint Learning of ASR Embeddings and LLM-Augmented Linguistics for Dementia Detection

arXiv:2606. 30675v1 Announce Type: cross Abstract: Early detection of dementia through speech analysis offers a non-invasive screening alternative, but capturing both acoustic and linguistic biomarkers remains challenging.

By Olivier Jiyoun Jung, Jonghyeon Park, Myungwoo Oh

llms rag multimodal benchmarks

arXiv Machine Learning

Jun 18

Fair Cognitive Impairment Detection Through Unlearning

arXiv:2606. 18571v1 Announce Type: new Abstract: Mild Cognitive Impairment (MCI) is a medical condition characterized by a noticeable decline in memory, language, or thinking abilities.

By William Nguyen, Jiali Cheng, Hadi Amiri

rag multimodal benchmarks

arXiv AI

Jul 1

ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection

arXiv:2606. 30646v1 Announce Type: cross Abstract: Speech recruits the same executive, attentional, and working memory processes underlying instrumental activities of daily living, or IADLs, providing a non-invasive proxy for cognitive assessment.

By Chukwuemeka Ugwu, Oluwafemi Richard Oyeleke

llms rag multimodal benchmarks

arXiv AI

Jun 16

Unifying Acoustic Features and Text with Multimodal LLMs for Neurodegenerative Screening

arXiv:2606. 14788v1 Announce Type: cross Abstract: Voice-based screening offers a scalable and non-invasive way to assess neurodegenerative diseases such as Alzheimer's disease (AD) and Parkinson's disease (PD), but their staging remains challenging due to the difficulty of integrating heterogeneous data.

By Qingfeng Zhang, Yuanxiong Guo, Yanmin Gong

llms rag multimodal

arXiv AI

Jul 14

Transcript-Free Lightweight Detection of Alzheimer's Disease from Spontaneous Speech Using Handcrafted MFCC-Dominant Acoustic Biomarkers

arXiv:2607. 10168v1 Announce Type: cross Abstract: It is still hard to find Alzheimer's disease (AD) early, especially when neuroimaging is expensive or tools that depend on language are not available.

By Rashin Gholijani Farahani, Azam Bastanfard

multimodal

arXiv Computer Vision

4d ago

Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment

arXiv:2501.18157v2 Announce Type: replace-cross Abstract: Building reliable speech systems often requires combining multiple modalities, like audio and visual cues. While such multimodal solutions fr...

By Joanna Hong, Sanjeel Parekh, Honglie Chen, Jacob Donley, Ke Tan, Buye Xu, Anurag Kumar

multimodal

arXiv AI

Jun 17

SpeechDx: A Multi-Task Benchmark for Clinical Speech AI

arXiv:2606. 17339v1 Announce Type: new Abstract: Speech offers a uniquely informative window into health by simultaneously engaging neurological, motor, respiratory, and vocal systems.

By Sejal Bhalla, Larry Kieu, Aina Merchant, Eyal de Lara, Alex Mariakakis

benchmarks

arXiv Computation and Language

Sep 1

A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification

arXiv:2512.07571v3 Announce Type: replace Abstract: This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tune...

By Nicolas Calbucura, Jose Guillen, Valentin Barriere

llms rag nlp fine-tuning multimodal

arXiv AI

Jun 9

Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis

arXiv:2606. 07533v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) effectively integrate text and audio to interpret context in complex interactive dialogues.

By Pawe{\l} Pozorski, Jakub Muszy\'nski, Maria Ganzha

llms nlp multimodal safety

arXiv Computation and Language

Sep 21

Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation

The paper evaluates pre‑trained speech embeddings from four state‑of‑the‑art speech foundation models for cross‑lingual Parkinson's disease severity assessment. Experiments span three datasets in zero‑shot and k‑shot settings, showing that these embeddings can transfer meaningfully across languages, though performance varies with dataset characteristics, preprocessing, and adaptation strategy. Misclassifications linked to inter‑speaker variability and atypical speech patterns underscore the need for more robust feature extraction, modeling, and explainability to support reliable clinical insights.

By Simon Pals, Cristian Tejedor-Garcia

rag benchmarks

arXiv AI

Jun 30

LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features

arXiv:2606. 28445v1 Announce Type: cross Abstract: Early detection of dementia enables timely intervention, and reflecting cognitive impairment, spontaneous speech offers a non-invasive screening modality.

By Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo Oh

llms fine-tuning multimodal benchmarks

arXiv AI

Jul 28

Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training

arXiv:2607. 22794v1 Announce Type: cross Abstract: Automatic depression detection with deep learning has shown promise but often suffers from limited generalization due to domain shift arising from inter-speaker variability.

By Ali Tabaraei, Federico Simonetta, Stavros Ntalampiras

multimodal benchmarks safety

Related stories