MedPsy introduces a family of 1.7B and 4B text‑only medical language models optimized for edge deployment, combining synthetic medical data, chain‑of‑thought targets from a 235B teacher, and a four‑stage post‑training curriculum. On seven closed‑ended medical benchmarks, MedPsy‑4B outperforms MedGemma‑27B while being 6.75× smaller, and MedPsy‑1.7B surpasses MedGemma‑1.5‑4B by 11.42 points despite being less than half its size. The models also reduce response length, disk footprint, and end‑to‑end latency on real‑device inference, keeping peak RAM below 5 GB on laptops and smartphones.
arXiv:2607. 14661v1 Announce Type: new Abstract: Deploying large language models (LLMs) as personal assistants on mobile devices demands privacy, low latency, and offline availability, yet the computational cost of giant models clashes with strict edge-hardware budgets.
The paper introduces CERTA, a decision‑level statistical certificate that decides when a quantized clinical language model can replace its native BF16 reference and escalates uncertain decisions to the reference model. Across eleven instruction‑tuned models (7‑72 B parameters) and 2,500 clinical‑verification decisions from 2,484 de‑identified patients, 8‑bit quantization achieved 99.8‑100% coverage, while 4‑bit configurations cut deployment costs to roughly 26–28% of the BF16 baseline with peak memory below a 48 GB accelerator. On external named‑entity benchmarks, the chosen NF4 settings showed only a 0.045 mean absolute F1 change, indicating minimal performance loss. whyItMatters":"CERTA enables risk‑controlled deployment of smaller, cheaper models for clinical decision support, potentially making advanced language models more accessible in resource‑constrained healthcare settings."
ThinkBit is a two‑bit scalar quantization framework designed to preserve reasoning capabilities in language models. It assigns each layer a ternary, 2‑bit, or 3‑bit tier based on sensitivity, then uses a Gumbel‑softmax search to find optimal codewords block‑by‑block. After freezing codewords, it recalibrates shared scales via knowledge‑distillation and performs a brief quantization‑aware training, finally selecting the best checkpoint with paired reasoning tests.
IronLLM-0.6B is a 654‑million‑parameter language model engineered for efficient on‑device inference, featuring a hybrid attention architecture, X‑MTP multi‑token prediction, and a lightweight verification head that yields a 1.48× decoding speedup. Trained on roughly 6.2 trillion tokens with a quality‑oriented pipeline and further refined via Multi‑Domain On‑Policy Distillation, the model adopts an Instruct‑Only design to meet low‑latency requirements. A lighter variant, IronLLM‑0.6B‑Light, replaces RMSNorm with Dynamic Tanh and streamlines costly components to enhance inference and quantization efficiency, offering a strong performance‑efficiency trade‑off for resource‑constrained deployment.
arXiv:2610.33839v1 Announce Type: new Abstract: De-identifying clinical notes with a hosted language model discloses the information the step is meant to protect. We present EdgeDeID, a 0.6B-paramete...
HyCo is a hybrid drafting method that assigns depth to a server model’s multi‑token prediction head for execution on a mobile NPU, while alternative branches are handled by a small language model in a single pass. By returning only the hidden state, input embedding, and key‑value entries, HyCo reduces server GPU time per token by 38.6% on Spec‑Bench and 29.2% on 32K‑token summarization, and achieves faster delivery and higher savings than an SLM‑only drafter. The approach also incorporates quantization, a restricted vocabulary, a 1K‑token window, and an ε‑threshold rule to relax exact sampling distribution preservation.
arXiv:2608. 15693v1 Announce Type: new Abstract: Running large AI models on resource-constrained edge devices requires model compression to reduce model size and computation.
SpecQuant is a training‑free framework that merges speculative decoding with multi‑parent quantization to enable adaptive, efficient inference of large language models. It generates several quantized variants (INT4, FP8, FP16) from a single base model and routes queries to the appropriate variant based on predicted complexity, using lightweight models for simple tasks and full‑precision models for complex reasoning. Evaluations on Qwen2.5 models across MMLU, AlpacaEval, and GSM8K show 35‑43% speedups with less than 2% accuracy loss, facilitating practical on‑device LLM deployment without specialized infrastructure.
arXiv:2603. 16867v2 Announce Type: replace Abstract: Large language models (LLMs) with chain-of-thought reasoning achieve state-of-the-art performance across complex problem-solving tasks, but their verbose reasoning traces and large context requirements make them impractical for edge deployment.
arXiv:2609.36965v1 Announce Type: new Abstract: System One models such as Jev offer an efficient alternative to generative language models for tasks that require decisions rather than open-ended resp...
The paper identifies a new problem in clinical natural language processing called the clinical lost‑in‑the‑middle (CLitM) effect, where large language models perform poorly on information located near the center of long electronic health record (EHR) documents. Using the MedAlign dataset, the authors quantify a 21.9‑percentage‑point accuracy gap across 2,196 instruction‑response pairs and six models, showing that most critical facts lie in the CLitM trough. They propose Query‑Conditioned Clinical Suppression (QCCS), a lightweight context‑selection gate that outperforms traditional retrieval methods (BM25, dense retrieval, cross‑encoder reranking) on a held‑out set of 83 instructions, achieving up to 25.3% accuracy for middle‑position queries. whyItMatters":"The study demonstrates that standard retrieval strategies fail to reliably surface central clinical information, and that a query‑aligned selection mechanism can substantially improve model performance on critical EHR data."
arXiv:2604. 25421v2 Announce Type: replace-cross Abstract: Federated fine-tuning provides a practical route to adapt large language models (LLMs) on edge devices without centralizing private data, yet in mobile deployments the training wall-clock is often bottlenecked by straggler-limited uplink communication under heterogeneous bandwidth and intermittent participation.