arXiv:1805. 11546v3 Announce Type: replace-cross Abstract: We examine the benefits of visual context in training neural language models to perform next-word prediction.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.
arXiv:2606. 00275v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have demonstrated impressive performance on multimodal tasks through scaled architectures and extensive training.
The paper introduces an experiment where a small masked language model (DeBERTa) is initialized with visual embeddings for tokens that correspond to image regions, following St. Augustine’s ostensive definition of word learning. The visual initialization leaves a measurable imprint that persists through training, yet it does not improve performance on most BabyLM benchmarks that test abstract grammatical knowledge. However, the seeded models show a consistent advantage in zero‑shot object‑property tasks and in a custom Visual‑Property Swap benchmark that probes color, material, size, and shape knowledge, with the advantage confined to the seeded words and transferable to newly seeded words.
arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.
arXiv:2502.16435v5 Announce Type: replace-cross Abstract: Humans develop perception through a bottom-up hierarchy: from basic primitives and Gestalt principles to high-level semantics. Current Multim...
The paper investigates why multimodal large language models (MLLMs) struggle with vision‑centric tasks when visual evidence conflicts with pretrained language knowledge. Using image reconstruction and a new WhatIfVis benchmark, the authors show that MLLMs preserve coarse‑grained visual attributes but fail to consistently use them, and that supervised fine‑tuning and activation patching can improve controllability of visual context sensitivity. The study demonstrates that the main bottleneck lies in the models’ inability to reliably regulate their reliance on visual evidence rather than in visual perception itself.
arXiv:2602.02326v3 Announce Type: replace Abstract: If multilingual language model representations contain separable language-associated structure, then shifting between languages may require only si...