The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper investigates how misleading context—specifically fabricated evidence and bare assertions—affects large language models’ medical question‑answering performance. Experiments on MedMisBench show that models are more prone to adopt answers based on assertions than fabricated evidence, and that these misleading cues are often disclosed in reasoning traces but rarely in final responses. A monitor that reads open reasoning traces can detect most corrupted decisions, whereas monitoring only responses is less effective.
arXiv:2611.00460v1 Announce Type: cross Abstract: Large language models (LLMs) now answer medical exam questions at or above expert level, but it is unclear whether such scores reflect clinical compe...
The study evaluates how clinical language models perform when patient narratives are presented in different voices—clinical, socioeconomic, and cultural—using 1,000 MedQA cases paired with three retellings each. All 12 tested models show a significant drop in accuracy (about 26% more errors) when the patient speaks in a marked voice, a penalty that persists even when the original vignette is included verbatim and is not explained by measured retelling properties. The authors find that rewriting the patient’s account into a clinical vignette before answering can halve this accuracy gap, though structured schema pipelines and additional reasoning tools offer limited improvements.
RetiJev is a family of 0.8B‑9B decision models designed for ophthalmic clinical decision support, offering fast, accurate, and privacy‑preserving inference without autoregressive decoding. The authors created RetiJev‑Data, a source‑grounded corpus of 78,158 clinical‑decision and diagnosis questions, 50,074 guideline‑grading questions, and a 24,662‑question evaluation benchmark. RetiJev outperforms a 27B LLM and the commercial Jev 1.13 model on guideline grading and clinical decisions, achieving the highest accuracy while answering a case in 158 ms on a single GPU—3.5× faster than LLM generation and 470× faster than reasoning.
The paper introduces MedQA-MM, a benchmark that exposes shortcut reasoning in medical multimodal multiple-choice questions. By auditing prompts, images, and modalities, the authors show that models often rely on textual cues rather than visual evidence, with full-input accuracy at 62.63% but only 5.21% when restricted to text. The study highlights the need for route-level evidence to validate true medical image reasoning.
arXiv:2604. 04593v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds large language models in external medical knowledge, yet standard retrievers frequently surface hard negatives that are semantically close to the query but describe clinically distinct conditions.