arXiv AI By Ken Enda, Jien Saito, Masami Mukai, Feng Han, Emir Turkes, Tatsumi Kusakabe, Reona Umeki, Kazuhiro Sumiyoshi, Mitsuya Numata, Takumi Tanikawa, Akira Endoh, Kenji Hirata, Atsushi Yoshida, Kohsuke Kudo, Shinya Tanaka

The LLENS Platform for On-Premises Large Language Model Agents in a Japanese University Hospital: Development and Evaluation Study

Read the original on arXiv AI →

The LLENS platform is a clinical support AI system deployed inside Hokkaido University Hospital’s closed network, running the open‑weight model Kimi K2.6 on a single host with eight NVIDIA H200 GPUs. It includes ten harness components—such as page image reading, full‑text attachment input, a sandbox, and self‑checking—allowing the model to read and interact with hospital documents. In evaluation, the system achieved 97.2 % accuracy on 180 document‑based questions when using interactive access, outperforming fixed retrieval by 15.9 percentage points, and has been operational since September 14, 2026.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Aug 13

A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench

arXiv:2608. 12138v1 Announce Type: cross Abstract: General-purpose large language models (LLMs) have recently been reported to match or exceed specialized clinical AI tools on medical benchmarks, but such comparisons draw on a narrow set of systems and on benchmarks developed largely in high-income settings.

By Praveen Reddy, Charuta Mandke, Suvrankar Datta, Sarah Khan, Siddharth Reddy Anthireddy, Shitij Arora, Vishal Singh
arXiv Computation and Language
Oct 16

Making every stage of medical device document assessment with large language models deterministic

The paper demonstrates that large language models (LLMs) used to preliminarily assess medical device documentation can produce inconsistent results across different commercial deployments and settings. By constructing a fully deterministic pipeline—including OCR, chunking, embedding, indexing, retrieval, and inference—the authors achieved byte‑identical outputs across repeated runs, varied machines, and even interruptions. The study shows that end‑to‑end determinism is possible and provides a verifiable foundation for audit and change control in regulatory compliance.

By Ryosuke Suzuki
arXiv AI
Jul 28

OpenAIs HealthBench in Action: Evaluating an LLM-Based Medical Assistant on Realistic Clinical Queries

arXiv:2509. 02594v3 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) on their ability to generate high-quality, accurate, situationally aware answers to clinical questions requires going beyond conventional benchmarks to assess how these systems behave in complex, high-stakes clinical scenarios.

By Sandhanakrishnan Ravichandran, Shivesh Kumar, Rogerio Corga Da Silva, Miguel Romano, Reinhard Berkels, Michiel van der Heijden, Olivier Fail, Valentine Emmanuel Gnanapragasam
arXiv AI
Oct 21

MedPAO-Assist: Edge-Deployable, Modality-Aware Agentic AI for Radiology Report Structuring

MedPAO-Assist is an edge‑deployable, agentic AI framework that structures radiology reports from radiologist dictation and voice commands using only about 3.3 GB of RAM and an average latency of 31 seconds on a standard laptop CPU. It supports four imaging modalities—Chest X‑ray, Chest CT, Abdomen CT, and Brain MRI—and operates entirely with a 0.6 B‑parameter small language model, avoiding cloud APIs or high‑end GPUs. The system follows an Extract‑Ground‑Generate pipeline, performing concept extraction, SNOMED‑CT grounding, structured report generation, and a speech‑driven human‑in‑the‑loop correction phase, enabling a fully “no‑touch” reporting workflow that requires no keyboard input or internet connectivity.

By Shrish Shrinath Vaidya, Minmini Selvam, Ganapathy Krishnamurthi
arXiv AI
Jun 2

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

arXiv:2606. 01961v1 Announce Type: new Abstract: Autonomous agents are increasingly expected to support end-to-end medical-AI research workflows, moving beyond isolated prediction tasks or short-form clinical question answering.

By Junqi Liu, Salena Song, Yuhan Wang, Jiawei Mao, Hardy Chen, Xiaoke Huang, Tianhao Qi, Pengfei Guo, Yucheng Tang, Yufan He, Can Zhao, Andriy Myronenko, Dong Yang, Daguang Xu, Yuyin Zhou