← Back to all news
arXiv AI By Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodr\'iguez, Xavier Suau

Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation

Read the original on arXiv AI →

The study investigates whether a multimodal model like Gemma4-12B-it uses a shared internal concept of numbers across different modalities and operations. By applying causal interventions to the model’s residual stream, the authors identify a small subspace where swapping coordinates changes the model’s arithmetic answer. They find that while the subspaces for text and image are not identical, a single rotation and rescaling aligns them, revealing a shared geometric structure that encodes numbers in hidden directions used to generate multi‑digit outputs over time.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Aug 19

Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.

By Zongyang Qiu, Yihan Wu, Kaixuan Fan, Bo Li, Hui Xiong
diffusionmultimodalsafety
More like this →
arXiv Computer Vision
Oct 22

JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

JUICE (Joint Unified multimodal modeling In Continuous Encoder spaces) is a single flow‑matching Transformer that jointly models images and text in pretrained continuous encoder spaces, allowing each modality to be generated by the same process with its own noise level. The model can perform tasks such as image generation, captioning, visual question answering, and joint generation by deciding which modalities are clean and which are generated. Experiments show that joint and conditional denoising together outperform either alone, and that JUICE achieves strong performance on GenEval, VQAv2, and visual spatial planning tasks, even enabling coupled sampling where both modalities are generated simultaneously.

By Jiahan Li, Huangyuan Su, Shengbang Tong, Shusheng Yang, Ruihan Guo, Suozhi Huang, Hexu Zhao, Pratyusha Sharma, Sherry Yang, Jiatao Gu, Rob Fergus, Kyunghyun Cho
llmsdiffusionnlpmultimodal
More like this →
arXiv Computation and Language
Sep 10

Through the Looking Glass: Directly Reading and Writing Transformers

The paper investigates how many transformer components influence a token prediction by measuring the absolute contribution of each unit and channel to the logit. It finds that thousands of components contribute to a single prediction, yet a small subset—often just dozens—carries the majority of the predictive mass. Across models ranging from 124 M to 7 B parameters, the proportion of the model involved in a prediction remains around one to three percent, independent of size, and the study demonstrates that specific components can be directly read and written to modify model behavior without additional training.

By Mark Oskin
llmsrag
More like this →