arXiv Computation and Language November 4, 2026 By Haoquan Zhang, Feiyang Liu, Juanxi Tian, Wanhua Li, Haodong Duan, Weiyang Liu

PixelBench: Can LLMs Understand and Generate Images Pixel by Pixel?

Read the original on arXiv Computation and Language →

PixelBench is a benchmark that evaluates large language models on their ability to generate and understand images pixel by pixel, without relying on learned image tokenizers or decoders. The benchmark includes text-to-pixel (T2P) and image-to-pixel (I2P) generation tasks across multiple resolutions, as well as a pixel-understanding evaluation. Models produce an RGB palette followed by a grid of indices that map each pixel to a palette color, and the study shows that strong models can generate recognizable images from text and outperform specialized pixelization algorithms at low resolutions, with performance correlating with pixel-understanding ability and improving up to a point before declining at high resolutions. Additionally, Pixel Reconstruction Tuning (PRT) augments visual instruction tuning with caption‑palette‑grid trajectories, and post‑training Molmo2‑4B with PRT data improves performance on 16 visual‑centric benchmarks, demonstrating the utility of explicit pixel reconstruction as auxiliary supervision.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computer Vision

Oct 28

Know, then Draw: A New Learning Paradigm for Visual Generation

The paper introduces a new learning paradigm for visual generation that separates knowledge acquisition and high-fidelity rendering. Autoregressive pretraining is used to acquire semantic knowledge, while diffusion models are employed afterward for detailed image synthesis. The authors demonstrate that this approach improves learning efficiency and final generation quality, and they provide compute‑optimal scaling laws that enable training a 3B‑parameter text‑to‑image model with only 6,600 H100 GPU‑hours, achieving state‑of‑the‑art scores on GenEval and DPG‑Bench.

By Yucheng Mao, Chongjian Ge, Zhuowen Tu, Hao Tan, Hanwen Jiang

diffusion nlp

arXiv Computation and Language

Sep 2

On the Design Fundamentals of Pixel Text Representation Learning

arXiv:2609.01147v1 Announce Type: cross Abstract: Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders strug...

By Chaohao Yuan, Ruifeng Yuan, Zhuoxu Huang, Yu Rong, Hong Cheng, Hou Pong Chan, Chenghao Xiao

multimodal benchmarks safety

arXiv AI

Jun 6

Image Generators are Generalist Vision Learners

arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.

By Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender, Shuyang Sun, Yanan Bao, Karen Truong, Zhicheng Wang, Wenlei Zhou, Jonathan T. Barron, Kyle Genova, Nithish Kannen, Sherry Ben, Yandong Li, Mandy Guo, Suhas Yogin, Yiming Gu, Huizhong Chen, Oliver Wang, Saining Xie, Howard Zhou, Kaiming He, Thomas Funkhouser, Jean-Baptiste Alayrac, Radu Soricut

llms diffusion computer-vision benchmarks

Hugging Face Trending Papers

Jul 7

From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models

Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Existing generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets.

diffusion computer-vision fine-tuning benchmarks

arXiv Machine Learning

Aug 10

Symbolic Graphics Programming with Large Language Models

arXiv:2509. 05208v2 Announce Type: replace-cross Abstract: Large language models (LLMs) excel at program synthesis, yet their ability to produce symbolic graphics programs (SGPs) that render into precise visual content remains underexplored.

By Yamei Chen, Haoquan Zhang, Yangyi Huang, Zeju Qiu, Kaipeng Zhang, Yandong Wen, Weiyang Liu

llms reinforcement-learning multimodal benchmarks

arXiv AI

Aug 28

PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference

PACE introduces a training‑free Condense‑and‑Extract framework that speeds up Vision‑Language Model inference by first adaptively downsampling visual inputs before encoding and then selectively retaining essential tokens during decoding. The Adaptive Pixel Compressor (APC) reduces encoder workload while preserving global context, and the Dynamic Dual‑Attention Extractor (DDAE) keeps task‑critical details by fusing visual and language signals. Applied to Qwen2.5‑VL‑7B, PACE maintains 93.8% of performance using only 10% of visual tokens, achieving a 3.1× speedup in time to first token.

By Junjie Liu, Shengyuan Ye, Xu Chen

llms efficiency multimodal