The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper investigates how different design choices in visual-head discovery scores affect the attention heads selected in vision‑language models. By systematically varying query type, discovery corpus, and attention source in Qwen3‑VL‑8B‑Instruct, the authors find that the attention source and query type have a larger impact on head selection than the corpus. While necessity scores distinguish discovery settings, steerability does not show a significant advantage over random head sets.
The study investigates the internal workings of an audio language model (Qwen3-Omni) by applying a logit lens to its middle layers. It finds that the model’s reasoning about spoken questions becomes legible in words before any token is emitted, revealing language‑agnostic, paralinguistic, and temporally distinct signals that are causally used in the network’s decision process. The authors demonstrate that these signals can be isolated and mapped to specific layers, providing a qualitative account of how the model processes audio input.
arXiv:2607. 01002v1 Announce Type: cross Abstract: In long-context use, large language models frequently synthesize answers from the meaning of a relevant context span rather than literally copy-pasting them.
The paper introduces Triage, a method that predicts the attention distribution of audio tokens before a language model processes them, enabling early pruning of less important tokens. By fitting a linear map to encoder outputs, Triage achieves high correlation (ρ ≥ 0.69) with full-model attention across eleven of thirteen large audio language models. Using this prediction, Triage compresses audio inputs while maintaining near‑full performance, outperforming baselines in transcription accuracy and significantly increasing the amount of audio that fits within a model’s context window.
arXiv:2510.00628v3 Announce Type: replace-cross Abstract: Large audio-language models (LALMs) are often used in tasks that involve reasoning over ordered options. An open question is whether their pr...
The paper investigates what open audio‑language models actually attend to when judging generated music. By designing four probes—removing audio, swapping clips, changing clip order, and rewording instructions—the authors test whether the models respond to the music, the text prompt, or the question framing. Across five judges and thousands of clips, the results show that most models rely on the prompt alone, only a few follow the audio for prompt alignment, and instruction wording can shift correlations as much as changing the model itself.