The paper introduces CERTA, a decision‑level statistical certificate that decides when a quantized clinical language model can replace its native BF16 reference and escalates uncertain decisions to the reference model. Across eleven instruction‑tuned models (7‑72 B parameters) and 2,500 clinical‑verification decisions from 2,484 de‑identified patients, 8‑bit quantization achieved 99.8‑100% coverage, while 4‑bit configurations cut deployment costs to roughly 26–28% of the BF16 baseline with peak memory below a 48 GB accelerator. On external named‑entity benchmarks, the chosen NF4 settings showed only a 0.045 mean absolute F1 change, indicating minimal performance loss. whyItMatters":"CERTA enables risk‑controlled deployment of smaller, cheaper models for clinical decision support, potentially making advanced language models more accessible in resource‑constrained healthcare settings."
arXiv:2608.03854v4 Announce Type: replace Abstract: Quantized large language models can run on consumer hardware, which motivates interest in on-premises processing of sensitive data. The reliability...
The study evaluates how quantization affects accuracy and safety of five 7‑8B language models on clinical benchmarks. INT8 GPTQ shows minimal degradation (≤1.9%) across tasks, while INT4 causes substantial, model‑dependent drops, especially in high‑risk scenarios and safety metrics. Recovery methods such as clinical calibration substitution and QLoRA fine‑tuning yield mixed results, underscoring the need for task‑specific validation.
arXiv:2608. 15383v1 Announce Type: new Abstract: Sparse mixture-of-experts (MoE) language models reduce arithmetic by activating only a small subset of experts per token, yet deployment still requires storing and moving the full expert bank.
The paper demonstrates that large language models (LLMs) used to preliminarily assess medical device documentation can produce inconsistent results across different commercial deployments and settings. By constructing a fully deterministic pipeline—including OCR, chunking, embedding, indexing, retrieval, and inference—the authors achieved byte‑identical outputs across repeated runs, varied machines, and even interruptions. The study shows that end‑to‑end determinism is possible and provides a verifiable foundation for audit and change control in regulatory compliance.
arXiv:2607. 20453v1 Announce Type: cross Abstract: Large language models show promise for clinical prediction, but zero-shot performance on specialized tasks is limited by incomplete domain knowledge, especially for smaller locally deployable models.
The paper demonstrates that greedy decoding from large language models is not precision‑invariant: the same model, prompt, and decoding algorithm can produce different outputs when run in BF16 versus FP16 on identical hardware. Across six models (1.1B–7B parameters, four families, and 12B) and three benchmarks, 49–100 % of prompts diverge, with a single token flip often cascading into trajectory‑level divergence. The authors develop an empirical error‑propagation analysis that identifies the top‑two logit margin at the LM head as the key factor, and they propose a low‑overhead intervention—selective FP32 LM head recomputation—that improves exact agreement by 22–36 percentage points with less than 4 % latency overhead. "whyItMatters":"The findings reveal that precision choices can fundamentally alter model outputs, challenging the assumption of deterministic greedy decoding and highlighting the need for precision‑aware inference strategies."
AMLworld-Compact introduces an importance‑weighted downsampling protocol for evaluating large language models on AMLworld’s test splits, retaining all illicit transactions while sampling benign ones from supervised‑score strata. The HT‑Coreset method reduces evaluation targets by 271× on HI‑Small and 611× on LI‑Small, providing unbiased confusion‑count estimates with exact recall and ratio estimates for precision and F1. Experiments with boosted‑tree strata and Graph Feature Preprocessor features show lower F1 RMSE compared to uniform sampling, and audit tools reveal that lexical checks do not reliably indicate correctness of model outputs.
arXiv:2608.28592v1 Announce Type: new Abstract: Large language models (LLMs) achieve high scores on medical knowledge examinations, yet real-world oncology is not a knowledge test--it is a sequence o...
The study evaluates whether a fine‑tuned open‑weight model (Gemma‑3‑12B) can match the performance of GPT‑4o in extracting multi‑label intracranial hemorrhage acuity from non‑contrast head‑CT reports. Using a 2×2 design that varied adaptation strategy (classification head vs. instruction fine‑tuning) and training‑data source (distilled real GPT‑4o labels vs. synthetic GPT‑4o‑generated reports), the distilled instruction‑tuned model achieved macro‑F1 scores comparable to GPT‑4o and surpassed the untuned base model. The key finding is that the source of training data—distilled real reports—was more important than the fine‑tuning method, and that the entire fine‑tuning and inference process fits on a single 24 GB consumer GPU.
arXiv:2609.14825v1 Announce Type: cross Abstract: Large language models (LLMs) are often deemed unsafe for clinical question answering because of their tendency to hallucinate. Retrieval augmentation...
arXiv:2504. 15610v4 Announce Type: replace Abstract: Fine-tuning a 7B language model for specialized advising is attractive in resource-constrained settings, but multi-epoch runs routinely exceed the wall-clock limits of the free-tier GPUs (Kaggle, Colab) such users rely on.
The study evaluates whether large language models (LLMs) with in‑context learning can better identify institution‑specific protected health information (PHI) in electronic health records than existing de‑identification systems. Using 100 pediatric oncology notes from Texas Children’s Hospital, eight LLMs were compared to two purpose‑built systems and pattern‑based baselines under three progressively specific prompts. The best LLM achieved an F1 score of 0.918, recovering 79% of previously missed PHI categories and reaching a recall of 0.981 after iterative prompt refinement, demonstrating that calibrated single‑pass prompting can close the institutional PHI gap while balancing precision and recall.