← Back to all news
arXiv AI

Can JEV Support Clinical Trial Criteria Assessment?

The study evaluates JEV, a non‑generative decision model, against the schema‑constrained Qwen Flash for clinical trial prescreening tasks. Across 2,048 patient‑condition pairs from TrialGPT and MIMIC‑III/IV datasets, JEV matched Qwen in macro‑F1 on TrialGPT and MIMIC‑IV but significantly outperformed it on MIMIC‑III, largely due to higher recall for conditions with insufficient information. JEV also demonstrated lower latency and cost per condition, suggesting it could serve as an initial assessment tool that flags information gaps for further review.

Related stories

arXiv AI
Oct 2

Jev in Medicine: A Benchmark Evaluation

The study evaluates Jev 1.13, a non‑generative model that selects from predefined answer options, on four medical benchmarks: MetaMedQA, PubMedQA, DiagnosisArena‑MCQ, and the NEJM Case Challenges. Jev’s top‑1 accuracy matches GPT‑6 Sol with medium reasoning on PubMedQA but falls behind on MetaMedQA, DiagnosisArena‑MCQ, and NEJM cases. While Jev shows strong calibration on MetaMedQA and is fast and inexpensive, its performance on examination and complex diagnostic tasks is substantially lower, indicating the need for task‑specific validation before clinical deployment.

By Alfredo Madrid-Garc\'ia, Beatriz Merino-Barbancho
llmsbenchmarks
More like this →
arXiv Machine Learning
Jul 7

Coverage-Controlled Preference Mining from Noisy Claim Verification for Evidence-Grounded Generation

arXiv:2603. 10494v2 Announce Type: replace-cross Abstract: Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that simply say less.

By Weixin Liu, Congning Ni, Qingyuan Song, Susannah L. Rose, Murat Kantarcioglu, Bradley A. Malin, Zhijun Yin
llmsragnlpreinforcement-learning
More like this →
arXiv Machine Learning
Oct 3

OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One

OmniMed-Jev is a new medical multimodal model that represents each decision as a Choice, Noul, or Score over a runtime-supplied candidate set, returning a full probability distribution for each decision. By unifying diverse imaging modalities and prediction tasks into a single candidate-conditioned probability model, it makes heterogeneous outputs comparable probabilities rather than task-specific strings. In controlled comparisons against a generative baseline, OmniMed-Jev’s reported probabilities align more closely with observed correctness, reducing calibration error by up to an order of magnitude and reliability error by up to two, while maintaining comparable point-prediction performance.

By Luyao Tang, Cheng Chen
multimodal
More like this →
arXiv AI
Aug 11

CliniCARE-Bench: Clinical Calibrated Audit of Medical Reasoning in EHR

arXiv:2608. 07796v1 Announce Type: new Abstract: Large language models perform strongly on medical knowledge benchmarks, but reliable clinical deployment requires agents to conduct defensible investigations over heterogeneous, longitudinal records: determining what evidence is needed, retrieving and reconciling structured and free-text data, grounding conclusions in verifiable evidence, and deferring cases that cannot be resolved reliably.

By Veronica Chatrath, Bryan Zhu, George Pu, Jingxuan Fan, Apaar Shanker, Varun Ursekar, Anahita Sharma, Jason Qin, Keqi Han, Soham Dinesh Tiwari, Soham Dan, Vijay Kalmath, Yuan Li, Daniel Yue Zhang, Chenguang Wang, Zainab Doctor, Zhijun Yin, Nigam H. Shah, Yuan Xue
llmsagentsbenchmarks
More like this →