arXiv Machine Learning
Oct 3

OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

OmniMed-Jev is a new medical multimodal model that represents each decision as a Choice, Noul, or Score over a runtime-supplied candidate set, returning a full probability distribution for each decision. By unifying diverse imaging modalities and prediction tasks into a single candidate-conditioned probability model, it makes heterogeneous outputs comparable probabilities rather than task-specific strings. In controlled comparisons against a generative baseline, OmniMed-Jev’s reported probabilities align more closely with observed correctness, reducing calibration error by up to an order of magnitude and reliability error by up to two, while maintaining comparable point-prediction performance.

By Luyao Tang, Cheng Chen
arXiv Machine Learning
Oct 23

The Value of System 1 Models in Medicine

The study evaluates four zero‑shot System 1 models on 86 public clinical tasks across 14 domains and 20 languages, measuring performance via AUROC. The best models achieved high AUROC scores—0.94 for medical knowledge, 0.93 for text understanding, 0.83 for clinical judgement from text, 0.89 for image perception, 0.84 for 2D image findings, and 0.69 for volumetric findings—yet performed worse on imaging tasks and varied significantly with language. System 1 models were fast (0.34–0.44 s) and energy‑efficient (24–81 J) but lagged behind clinicians on triage and were outperformed by generative models in some metrics.

By Xiufeng Yang, Konstantinos Vrettos, Zeineb Ben Chaaben, Fabian Drexel, Jannik L\"ubberstedt, Johannes Moll, Anirudh Narayanan, Frederike Stumpf, Cosmin I. Bercea, Jiawen Wang, Evangelia E. Vassalou, Krischan Braitsch, Sebastian Ziegelmayer, Markus Graf, Marcus R. Makowski, Lisa Adams, Nassir Navab, Michail E. Klontzas, Keno Bressem
arXiv Computation and Language
Oct 23

RetiJev: Fast and Source-Grounded Ophthalmic Decision Models

RetiJev is a family of 0.8B‑9B decision models designed for ophthalmic clinical decision support, offering fast, accurate, and privacy‑preserving inference without autoregressive decoding. The authors created RetiJev‑Data, a source‑grounded corpus of 78,158 clinical‑decision and diagnosis questions, 50,074 guideline‑grading questions, and a 24,662‑question evaluation benchmark. RetiJev outperforms a 27B LLM and the commercial Jev 1.13 model on guideline grading and clinical decisions, achieving the highest accuracy while answering a case in 158 ms on a single GPU—3.5× faster than LLM generation and 470× faster than reasoning.

By Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu
arXiv AI
2d ago

Decision and System One models in rheumatology: accuracy and calibration on specialty-training examination questions and diagnostic case vignettes

The study evaluated the accuracy and calibration of several Decision and System One models on rheumatology specialty‑training exam questions and diagnostic vignettes. On the RheumaMIR exam dataset, Jev achieved 93.0% accuracy in Spanish and 91.9% in English, while GPT‑6 Sol reached 99.4%. On the RheumBench diagnostic vignettes, GPT‑6 Sol again led with 76.9% accuracy, with Jev and other System One models scoring significantly lower (50.0%–59.6%).

By Alfredo Madrid-Garc\'ia, Diego Benavent, Fabian Lechner, Phillip Kremer, Johannes Knitza, Beatriz Merino-Barbancho