The paper introduces SAM3-LoRA, a parameter‑efficient adaptation of the SAM3 segmentation foundation model using Low‑Rank Adaptation for multi‑class structural defect segmentation. It presents a supervision method that trains the model directly from COCO‑style instance segmentation by using category names as prompts, eliminating the need for prompt templates or learned embeddings. The authors also identify a failure mode where the model responds to any prompt due to positive‑prompt‑only annotations and resolve it with exhaustive hard‑negative prompting, achieving significant improvements in pixel IoU and instance recall on both a tunnel lining dataset and the Structural Defects Dataset.
arXiv:2607. 28967v1 Announce Type: cross Abstract: Prompt tuning adapts vision--language models with few trainable parameters, but existing approaches trade off efficiency and adaptation: static textual prompts can overfit source classes, image-conditioned prompts add per-instance computation, and multimodal tuning modifies the visual branch.
arXiv:2606.16996v2 Announce Type: replace-cross Abstract: Segment Anything Model 3 (SAM 3) provides a strong frozen backbone for concept-prompted segmentation, but applying it directly to open-vocabu...
arXiv:2607. 16122v1 Announce Type: new Abstract: Evaluations should do more than measure a models current performance.
The paper explores soft prompting for few‑shot object detection with vision‑language models, showing that optimizing a small number of continuous prompt tokens—especially when placed at the cross‑modal boundary and initialized from an empty space token—can match LoRA performance while training far fewer parameters. Soft prompting also avoids catastrophic forgetting, transfers to newer models, and can be verbalized into readable prompts. The study extends these findings to manipulation tasks, indicating that VLMs already contain much of the necessary knowledge for specialized domains, and the main challenge is learning how to ask for it.
arXiv:2607. 14125v1 Announce Type: new Abstract: Pre-trained vision-language models (VLMs) enable zero-shot image classification by computing the similarity score between an image and textual descriptions, typically formed by inserting a class label (e.
arXiv:2609.01433v1 Announce Type: new Abstract: Concept erasure aims to suppress unsafe, privacy-sensitive, or undesirable generations in text-to-image diffusion models while preserving benign semant...
The paper introduces Exemplar Partitioning (EP), an unsupervised technique that constructs interpretable feature dictionaries from large language model activations by clustering streamed activations into Voronoi regions defined by exemplars and their averages. EP allows comparison of dictionaries across layers, checkpoints, and architectures, and demonstrates utility in interpreting model behavior, tracking training dynamics, detecting hidden concepts, and enabling targeted interventions. Experiments on Gemma‑2‑2B and Llama‑3.1‑8B show EP can reveal how instruction tuning reorganizes harmful prompt activations, facilitate interventions that alter model responses, and achieve high concept‑detection performance while requiring far fewer construction tokens than comparable methods.
arXiv:2607. 02072v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in domains requiring guardrails to detect unsafe, off-topic, or adversarial prompts.
The paper investigates how much semantic information is lost when frozen foundation models are combined for few‑shot 3D segmentation. By varying the number of retained semantic alternatives before fusion, the authors show that keeping the full distribution of class scores yields higher harmonic‑mean IoU than collapsing to a single class. Experiments on ScanNet200 and ScanNet++ confirm that full‑distribution fusion consistently outperforms top‑1 and other operators, and that most useful information is recovered by retaining a compact set of plausible alternatives.
arXiv:2605.17630v3 Announce Type: replace Abstract: Frozen segmentation foundation models often fail when the target class appears in a form that is weakly represented during pretraining. To address...
Prompt-driven vision-language models (VLMs) hold immense promise for accelerating dense remote sensing (RS) annotation, but static models suffer from severe performance degradation when deployed on novel scenes, unseen categories, or visually confusing backgrounds. Moreover, existing unified paradigms primarily rely on intra-image specific prompts, lacking flexible task routing to adapt to multi-intent operational workflows.
arXiv:2606. 27091v1 Announce Type: cross Abstract: LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data.