Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

29,398 stories · RSS feed

arXiv Computer Vision

Sep 30

CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning

arXiv:2609.35823v1 Announce Type: new Abstract: Vision-language models (VLMs) have made substantial progress in autonomous driving, but their success has primarily been studied in ego-centric scenes....

By Kang Yang, Shuai Liu, Hang Li, Yance Fang, Deying Li, Yongcai Wang

llms agents nlp robotics multimodal benchmarks

arXiv Computer Vision

Sep 30

Persistence Forcing: Exploiting Feature Specialization in Pixel-Space Diffusion

arXiv:2609.36014v1 Announce Type: new Abstract: Pixel-space diffusion Transformers (DiTs) directly operate on high-dimensional visual data, yet their hidden representations typically undergo uniform...

By Chong Wang, Zixuan Fu, Shiqi Huang, Siyuan Yang, Hao Cheng, Bihan Wen

llms diffusion

arXiv Computer Vision

Sep 30

Xiaomi-OCR-0 Technical Report

arXiv:2609.36136v1 Announce Type: new Abstract: Compact OCR-specific vision-language models achieve strong document parsing performance, but often rely on costly supervision and focus primarily on vi...

By Xin Chen, Anan Du, Feng Feng, Pei Fu, Jian Luan, Longwei Xu, Shaojie Zhang, Hang Li, Heng Qu, Cheng Tan

llms computer-vision reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 30

LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning

arXiv:2609.36219v1 Announce Type: new Abstract: Perspective taking is a fundamental component of spatial intelligence, requiring models interpret spatial relations from a specified viewpoint, such as...

By Bang Xiao, Wenqi Jia, Ozgur Kara, Tiancheng Shen, Yibo Yang, Bolin Lai, Junho Kim, James Matthew Rehg

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding

arXiv:2609.36217v1 Announce Type: new Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...

By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang

llms rag computer-vision benchmarks safety

arXiv Computer Vision

Sep 30

Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

arXiv:2609.36628v1 Announce Type: new Abstract: Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, partic...

By Yanan Wang, Tingsong Li, Kaixun Jiang, Chongyang Zhong, Chenwei Xoe, Zhaohe Liao

llms reinforcement-learning multimodal benchmarks safety

arXiv Computer Vision

Sep 30

Reprogramming Vision-Language Models via Structured Prompt Reparameterization

arXiv:2609.36680v1 Announce Type: new Abstract: Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In v...

By Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari, Feng Liu, Joseph West, Kourosh Khoshelham

llms rag multimodal benchmarks

arXiv Computer Vision

Sep 30

Learning via Self-Consistency for Diffusion-based Video Reasoning

arXiv:2609.36826v1 Announce Type: new Abstract: Video generation models have demonstrated emerging zero-shot capabilities for visual reasoning, perception, and other vision tasks. However, diffusion-...

By Zhenghao Ni, Weimin Qiu, Meng Tang

llms diffusion computer-vision fine-tuning efficiency

arXiv Computer Vision

Sep 30

GlassFormer: Learning Real-time Glass Segmentation using Radar-Depth Fusion

arXiv:2609.36844v1 Announce Type: new Abstract: Transparent surfaces are ubiquitous in built environments, yet they remain a persistent failure case for robotic perception. RGB cameras perceive the b...

By Suhani Grover, Astik Srivastava, Viswas Dinesh, Avinash Sharma, K. Madhava Krishna

llms computer-vision robotics multimodal

arXiv Computer Vision

Sep 30

Representation Dynamics Reveal Semantic Saliency and Similarity for Visual Token Pruning in MLLMs

arXiv:2609.36916v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention...

By Weixuan Li, Zikun Zhou, Xinyi Zhuang, Xinyan Guo, Rui Tian, Chuyao Zhang, Lin Gao

llms efficiency multimodal

arXiv Computer Vision

Sep 30

OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models

arXiv:2609.37052v1 Announce Type: new Abstract: Omnimodal large language models (Omni-LLMs) encode audio and visual streams into temporally interleaved token sequences for multimodal reasoning. Howev...

By Yuchen Deng, Zidang Cai, Feidiao Yang, Yufei Wang, Jie Wang, Hai-Tao Zheng, Yuxing Han

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

Real2Gym: Building Gyms from Videos, Bringing Skills to Robots

arXiv:2609.37089v1 Announce Type: new Abstract: Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, exec...

By Kerui Ren, Yingxiang Xu, Kaiwen Song, Linning Xu, Bo Dai, Mulin Yu, Tao Lu

llms agents robotics

arXiv Computer Vision

Sep 30

Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack

arXiv:2609.37096v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. I...

By Liwei Che, Yihao Quan, Sen Fang, Hongyi Wang, Ranjay Krishna, Ruixiang Tang, Vladimir Pavlovic

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation

arXiv:2609.37283v1 Announce Type: new Abstract: Medical multimodal large language models (MLLMs) are increasingly expected not only to answer clinical questions, but also to localize the visual evide...

By Xuyang Cao, Enyou Liu, Jun Zhao, Zhuoyun Liu, Jintao Fei, Leo

llms computer-vision fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Sep 30

What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

arXiv:2609.37317v1 Announce Type: new Abstract: Omnimodal evaluation should go beyond independent text, image, and speech production: individually plausible outputs may not express a coherent shared...

By Sieun Hyeon, Yejoon Lee, Mintaek Lim, Woojin Kim, Jaeik Kim, Jaeyoung Do

llms agents diffusion multimodal benchmarks

arXiv Computer Vision

Sep 30

When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs

arXiv:2609.37345v1 Announce Type: new Abstract: Streaming video understanding requires Video Large Language Models (Video-LLMs) to reason over continuous visual streams under causal constraints. As t...

By Xiang Hu, Jiazuo Yu, Lu Zhang, Yunzhi Zhuge, Huchuan Lu

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

Evaluating the Evaluators: Diagnosing Large Multimodal Models for AI-Generated Image Assessment

arXiv:2609.37576v1 Announce Type: new Abstract: With the rapid advancement of text-to-image (T2I) generation, robust evaluation becomes critical yet challenging, as traditional metrics fail to captur...

By Yu Zhao, Jiarui Wang, Huiyu Duan, Ye Zhao, Jutao Tang, Juntong Wang, Guangtao Zhai, Xiongkuo Min

llms diffusion multimodal benchmarks safety

arXiv Computer Vision

Sep 30

Texture Space Material Diffusion

arXiv:2609.37654v1 Announce Type: new Abstract: We present a method for generating high quality materials for 3D objects entirely in texture space. We finetune a video diffusion transformer for text-...

By Jacob Munkberg, Peter Kocsis, Jon Hasselgren

llms diffusion benchmarks

arXiv Computer Vision

Sep 30

Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding

arXiv:2609.37655v1 Announce Type: new Abstract: Advancing spatial intelligence in Multimodal Large Language Models (MLLMs) is bottlenecked by the scarcity of complex, scalable 3D question-answer (QA)...

By Jiayu Ying, Qijian Tian, Ruijie Xu, Xinnan Zhu, Daoguo Dong, Jiachen Xu, Xin Tan

llms agents nlp robotics fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning

arXiv:2609.37656v1 Announce Type: new Abstract: Large vision-language models (LVLMs) exhibit strong reasoning capabilities, yet the visual and textual evidence supporting the generated responses rema...

By Bowen Yuan, Danny Wang, Ruihong Qiu, Zijian Wang, Zi Huang

llms multimodal benchmarks

Get the daily briefing