arXiv AI

Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention

arXiv Machine Learning
Aug 27

Large Language Model Few-Shot Prompting with Dilemma Training Outperforms Human Surrogates in Predicting Patient Preferences

The paper introduces P4-DT, a personalized patient preference predictor that uses dilemma training to elicit context‑dependent decision reasoning. In a study of 12 patient‑surrogate pairs, P4‑DT achieved 81.7% accuracy in predicting patient treatment choices, outperforming unassisted surrogates (55.0%) and surrogates aided by a simpler P4 model (61.7%). The authors show that incorporating contextual scenarios and open‑ended text into prompts improves accuracy by 15 percentage points over static value ratings.

By Natasha Ureyang, Sebastian Porsdam Mann, Yuxin Liu, Zuriel Hassirim, Melanie Almonte, Wenhao Chen, Joyce Ng, Thant Nay Lin, Aung Thiha, Gerald CH Koh, Brian David Earp, Pin Sym Foong
arXiv Computation and Language
Sep 1

MedConceal: A Benchmark for Clinical Hidden-Concern Reasoning Under Partial Observability

MedConceal is a new benchmark for evaluating medical dialogue systems on hidden‑concern reasoning under partial observability. It features 300 curated cases and 600 clinician‑LLM interactions, using an interactive patient simulator that hides latent concerns and tracks their revelation and resolution through theory‑grounded communication signals. The benchmark assesses both confirmation (surfacing hidden concerns) and intervention (addressing the primary concern), revealing that current models excel on different metrics while human clinicians still outperform them on intervention success.

By Yikun Han, Joey Chan, Jingyuan Chen, Mengting Ai, Simo Du, Yue Guo
arXiv AI
Oct 21

When Health Assistants Agree Too Readily: A Benchmark of Time-Critical Self-Diagnosis

The paper introduces SelfDx, a benchmark that tests whether AI models appropriately respond to health queries by either cautioning against self-diagnosis or agreeing when evidence rules out serious alternatives. It evaluates 27 frontier models on 100 paired cases, finding that most models inappropriately agree in caution-required scenarios, achieving only a 77.0% appropriate caution rate. The study shows that prompt engineering can raise caution but at the cost of agreement, while supervised fine‑tuning on balanced examples improves both metrics.

By Marilyn Zhang, Tianfeng Chen, Patricia M Pechter, Mark E. Whiting
arXiv Computation and Language
3d ago

Accounting for Irregular State Transitions through Belief Estimation in Online Medical Consultation

The paper introduces PRISM, a framework that separates belief estimation from action selection to handle irregular state transitions in online medical consultations. By maintaining a posterior belief over consultation states, PRISM improves clinical decision accuracy by 22.40 percentage points and response quality by 5.51 BERTScore_F1 points compared to leading LLMs and clinical agents. Ablation studies confirm that oracle state information further enhances response quality across baselines.

By Mingzhe Lu, Xiangru Chen, Jian Qin, Yunpeng Li, Xihan Gao
arXiv Computation and Language
Sep 24

Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue

The paper introduces Expected‑Severity‑Risk (ESR), a new objective for selecting questions in proactive medical dialogue that prioritizes reducing the expected severity of diagnostic errors rather than merely uncertainty. ESR uses population statistics to marginalize over possible answers and distills its rankings into a prefix‑only language policy, enabling deployment without teacher‑side risk computation. Experiments on DDxPlus show ESR cuts high‑severity diagnostic misses by 29.5% and boosts accuracy while adding only 0.14 extra questions per dialogue.

By Chenxuan Li, Xinrong Chen, Luyan Zhang, Peidong Jia, Runfan Zheng, Zhongyu Zhao, Xuecheng Shang, Peixing Wan