The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.
By Zongyang Qiu, Yihan Wu, Kaixuan Fan, Bo Li, Hui Xiong
JUICE (Joint Unified multimodal modeling In Continuous Encoder spaces) is a single flow‑matching Transformer that jointly models images and text in pretrained continuous encoder spaces, allowing each modality to be generated by the same process with its own noise level. The model can perform tasks such as image generation, captioning, visual question answering, and joint generation by deciding which modalities are clean and which are generated. Experiments show that joint and conditional denoising together outperform either alone, and that JUICE achieves strong performance on GenEval, VQAv2, and visual spatial planning tasks, even enabling coupled sampling where both modalities are generated simultaneously.
By Jiahan Li, Huangyuan Su, Shengbang Tong, Shusheng Yang, Ruihan Guo, Suozhi Huang, Hexu Zhao, Pratyusha Sharma, Sherry Yang, Jiatao Gu, Rob Fergus, Kyunghyun Cho
The paper investigates how many transformer components influence a token prediction by measuring the absolute contribution of each unit and channel to the logit. It finds that thousands of components contribute to a single prediction, yet a small subset—often just dozens—carries the majority of the predictive mass. Across models ranging from 124 M to 7 B parameters, the proportion of the model involved in a prediction remains around one to three percent, independent of size, and the study demonstrates that specific components can be directly read and written to modify model behavior without additional training.
By Mark Oskin
arXiv:2608. 10251v1 Announce Type: cross Abstract: A transformer's answer lives on one axis: the direction its unembedding reads.
By Mark Oskin
arXiv:2610. 17117v2 Announce Type: replace-cross Abstract: Prior research into interpretability does not test whether a representation stays consistent under paraphrasing.
By Bryce Grant, Peng Wang
arXiv:2610.22796v1 Announce Type: new
Abstract: Reusing a Transformer block across depth increases computation without replicating its parameters, but successive passes have no dedicated state throug...
By Ilia Zaznov
The paper introduces Causal Visual Recurrent Reasoning (CVRR), a method that forces multimodal models to rely on latent visual states by using recurrent computation as the sole image‑conditioned path to prediction. CVRR initializes recurrence from the question hidden state after a pretrained vision‑language model has processed the image, repeatedly updates this state while re‑reading the same visual evidence, and removes all other visual traces before decoding. Experiments on several benchmarks show that CVRR maintains strong performance while other latent reasoners lose visual competence, and causal interventions confirm that predictions depend on the recurrent visual trajectory.
By Suhyeong Park, Junha Jung, Jaewoo Kang
The paper investigates how treating each entity in a graph or language model as having its own coordinate frame—viewed as a symmetry—constrains the form of attention mechanisms. It shows that with orthogonal transports, invariant attention scores depend only on norms and transported inner products, while equivariant messages lie in the span of the entity vectors and their transported counterparts. The authors demonstrate that standard attention corresponds to a flat shared frame, and that incorporating per‑entity frames allows relations to be represented as operators that compose via parallel transport, yielding empirical gains on synthetic relational tasks and natural language benchmarks.
By Aakash Sarkar, Ananya Mukherjee, Motoshige Sato
The paper demonstrates that a single-pass multimodal model struggles to produce faithful, comprehensive reviews of long recordings or documents, often omitting a third of the content and embellishing the rest. By splitting the task into two passes—first transcribing the source and then reviewing the transcript—the authors show improved faithfulness and coverage across a diverse set of 21 sources. The benefit is most pronounced for longer or weaker baseline cases, while the approach introduces new failure modes such as space constraints and memory confabulation.
By Bojie Li, Noah Shi
The study investigates how text-to-image models interpret number words beyond ten objects. By keeping the object type and initial noise constant, researchers varied the requested count and observed that models reliably follow the number only up to about ten items; beyond that, the count influence weakens and the image composition is largely governed by the coarse noise. Models such as SD3.5‑Medium and FLUX.1‑dev lose adherence after ten, while FLUX.2 and Qwen‑Image loosely track counts up to twenty‑five, yet no clear directional effect is achieved beyond the models’ limits.
By Liang Yue, Eric A. Vance
arXiv:2607. 13188v1 Announce Type: new Abstract: Human cognition does not separate understanding and generation.
By Minh-Quan Le, Armand Comas, Alexandros Lattas, Stylianos Moschoglou, Pedro V\'elez, Amit Raj, Aaron Germuth, Thabo Beeler, Dimitris Samaras, Di Qiu
arXiv:2608. 19739v1 Announce Type: cross Abstract: Multimodal LLMs can see a document, but they often can't read it reliably.
By Alin-Ionut Popa