← Back to all news
arXiv AI By Hyojung Han, Ryangkyung Kang, Youngeun Kim, Seung-Hun Jeon

When PTQ Calibration Helps or Hurts: Dose-Driven Collapse and Validation-Gated Abstention in Quantized Text Embedders

Read the original on arXiv AI →

The paper investigates when post‑training quantization (PTQ) calibration helps or hurts text embedders, focusing on the amount of calibration (dose) and whether to calibrate at all. Across three models and 161 calibration runs at 4‑bit weight quantization, the effect ranges from significant gains to catastrophic degradation, driven mainly by the functional displacement of the quantized model. A validation‑gated abstention rule is proposed that avoids harmful calibrations while capturing most benefits, highlighting the trade‑off between safety and opportunity.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 8

Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders

The paper introduces a label‑free mixed‑precision post‑training quantization technique for text embedders called Quantize by Drift. It uses quantization‑induced representation drift—measured by re‑encoding a corpus after quantizing a module—to estimate module sensitivity without relevance labels. Across five development embedders, drift‑based ordering of mixed‑precision plans correlates strongly with held‑out retrieval quality (macro Spearman 0.911) and outperforms uniform precision under a hard byte budget, though it does not fully replace fine‑grained redistribution strategies.

By Hyojung Han, Jongmin Kim, Seung-Hun Jeon
ragefficiency
More like this →
arXiv AI
Sep 15

Option-Aware Retrieval and Task-Specific VLM Adaptation for Medical VQA

The paper presents a system for the MedReason 2026 challenge that tackles both multiple‑choice and open‑ended medical visual question answering using offline, containerized inference. Key findings include that comparing answer semantics rather than labels boosts retrieval‑only accuracy from 20.0 % to 57.5 % on a 200‑case holdout, and that varying the number of in‑prompt retrieved examples has minimal impact on final accuracy (93.5 %–94.0 %). The final system achieves 94.0 % MCQ accuracy on the development set and 93.20 % on the official pre‑evaluation, far surpassing the off‑the‑shelf baseline. "whyItMatters":"The results demonstrate that semantic‑aware retrieval and careful adapter tuning can dramatically improve medical VQA performance, offering a practical approach for high‑accuracy, offline inference in clinical settings."

By Tristan Kirscher (ICube, Institut Strauss), Niklas C. Koser (CAU), Soren Pirk (CAU)
llmsnlpfine-tuningmultimodal
More like this →
arXiv Computation and Language
Sep 16

Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families

arXiv:2609. 16391v1 Announce Type: cross Abstract: Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact.

By Hyojung Han
llmsragefficiency
More like this →