Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,384 stories · RSS feed

arXiv Computer Vision
Sep 18

SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenes

SnapPhysics is a training‑free framework that reconstructs 3D objects and estimates their physical properties—mass, friction, and center of gravity—from a single image. It combines instance‑level 3D reconstruction with a physics‑aware scene graph to provide geometric grounding and inter‑object relationships for vision‑language model reasoning. Experiments on 3D‑FRONT and real captured scenes show significant improvements over existing methods, enabling physically interactive mixed reality experiences without manual tuning.

By Suji Kang, Seok-Young Kim, Young Bin Kim, Taewook Ha, Dieter Schmalstieg, Shohei Mori, Woontack Woo
arXiv Computer Vision
Sep 18

A Smaller Transformer in Your Transformer

The paper introduces Transformer-Within-Transformer (TWT), a post‑hoc technique that merges contiguous redundant layers in Vision Transformers into a single surrogate layer. By doing so, TWT cuts both parameter count and inference compute while maintaining competitive performance on natural image tasks with only half the depth. In histopathology applications, TWT not only matches but sometimes surpasses the baseline model’s performance.

By Dhananjay Tomar, Marius Aasan, Andreas Kleppe, Ad\'in Ram\'irez Rivera
arXiv Computer Vision
Sep 18

MoSSGate: Memory-Modulated State-Space Gating for Skin Lesion Segmentation

MoSSGate is a plug‑and‑play module for U‑Net that improves skin lesion segmentation by combining boundary‑aware spatial gating, an external memory modulator, and parallel 2D state‑space modeling for efficient global context aggregation. The design limits long‑range propagation to informative regions, adapts dynamically to each sample, and preserves sharp lesion boundaries while keeping computational cost low. Experiments on ISIC 2017 and 2018 show state‑of‑the‑art accuracy (86.3%/85.9% mIoU, 92.6%/90.6% Dice) with fewer FLOPs than most CNN baselines.

By Anum Awan, Mahnoor Buriro, Muhammad Younas Khan, Md Imam Ahasan
arXiv Computer Vision
Sep 18

AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images

AgriScope is a unified pixel‑grounded multimodal framework designed for agricultural image understanding. It supports image‑level, region‑level, and pixel‑level tasks such as grounded caption generation, referring expression segmentation, and multi‑turn multimodal interaction. The authors also introduce AgriGround, a large‑scale dataset with over 500K images and 11M instruction‑following samples, created via an automatic annotation pipeline that combines caption generation, phrase‑level grounding, segmentation mask creation, and instruction synthesis.

By Abderrahmene Boudiaf, Mohamad Alanssari, Irfan Hussain, Sajid Javed
arXiv Computer Vision
Sep 18

DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering

DocAttriBench (DAB) is a large‑scale benchmark for fine‑grained, element‑level source attribution in Document Visual Question Answering (VQA). It introduces MAPPET, a Mask‑based Perplexity‑Derived Attribution method that uses document layout and language modeling to identify the most informative layout element for each answer. The benchmark contains 237k documents and 296k question‑answer pairs with element‑level grounding, and it evaluates multimodal LLMs on answer accuracy, attribution accuracy, and overall answer quality, revealing that even strong models often fail to localize supporting elements.

By Luca De Grandis (University of Modena and Reggio Emilia, Modena, Italy), Silvia Cappelletti (University of Modena and Reggio Emilia, Modena, Italy), William Raccagni (University of Modena and Reggio Emilia, Modena, Italy, University of Pisa, Pisa, Italy), Marcella Cornia (University of Modena and Reggio Emilia, Modena, Italy), Lorenzo Baraldi (University of Modena and Reggio Emilia, Modena, Italy), Rita Cucchiara (University of Modena and Reggio Emilia, Modena, Italy)
arXiv Computer Vision
Sep 18

MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

The paper introduces MiX, a micro‑inverted‑scaling format that replaces shared exponents with shared mantissas to avoid microscaling collapse in low‑bit vision‑language models. An adaptive dual‑format inference framework (MiX‑MX) maps this format to a custom accelerator, replacing multipliers with shifters. Experiments show 4.5‑bit MiX matches or outperforms NVFP4 accuracy while improving area efficiency by 25 % and delivering 2.3–4.5× speedup with 1.4–2.9× energy savings over the Focus accelerator.

By Yuan Liao, Jae-sun Seo
arXiv Computer Vision
Sep 18

SalsaAgent: A multimodal embodied language model for interactive dance generation

SalsaAgent is a multimodal embodied language model that generates expressive, full‑body salsa follower motions in response to a human leader and music. The approach treats partner interaction as nonverbal token passing, extending a large language model’s vocabulary to include discrete motion, pairwise relation, and audio tokens. A two‑stage token‑to‑diffusion pipeline, combined with full‑body and pairwise‑relation tokenizers and alignment with automatically derived text descriptions of skeleton dynamics, yields improved motion quality, spatial coordination, and music‑partner synchrony compared to prior baselines.

By Payam Jome Yazdian, Zoe Stanley, Angelica Lim
arXiv Computer Vision
Sep 18

Attention-Steered Vision-Language Models for Sign Language Translation

The paper introduces AttnSign, a vision‑language model that improves sign language translation by steering spatial‑temporal attention. It first supervises attention on sign‑relevant regions such as faces and hands in each frame, then uses an RL‑based motion‑cadence method to focus on keyframes. Experiments on How2Sign and OpenASL show AttnSign consistently outperforms existing methods.

By Meibo Hu, Guohao Sun, Annemarie D. Ross, Sheng Li, Zhiqiang Tao
arXiv Computer Vision
Sep 18

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

HunyuanOCR-1.5 is a lightweight, end‑to‑end OCR‑specialized vision‑language model that unifies document parsing, text spotting, information extraction, text‑image translation, and multi‑image document understanding. It builds on the HunyuanOCR‑1.0 architecture, improving efficiency with DFlash‑based OCR decoding for faster inference (6.37× Transformer speedup, 2.14× under vLLM) and enhancing capability through an Agentic Data Flow system that autonomously constructs high‑quality training data for long‑tail OCR tasks. The model achieves top‑tier performance on OmniDocBench v1.6 and sets new milestones in ancient‑script OCR, chart/table parsing, multilingual parsing, and hallucination evaluation, while remaining lightweight for deployment.

By Gengluo Li, Xingyu Wan, Shangpin Peng, Weinong Wang, Hao Feng, Yongkun Du, Binghong Wu, Zheng Ruan, Zhiqiong Lu, Liang Wu, Pengyuan Lyu, Huawen Shen, Zibin Lin, Shijing Hu, Jieneng Yang, Hongbing Wen, Guanghua Yu, Hong Liu, Bochao Wang, Can Ma, Han Hu, Chengquan Zhang, Yu Zhou
arXiv Statistics ML
Sep 18

Portfolio-Based Constrained Multi-Objective Bayesian Optimization for Materials Design

The paper presents a portfolio-based approach to constrained multi-objective Bayesian optimization for materials design, framing acquisition‑function selection as an adaptive policy problem. Two controllers—UCB‑Bandit, a modified UCB multi‑armed bandit, and Agentic‑Switch, a multi‑agent system powered by a large language model—were tested against fixed‑policy baselines on synthetic benchmarks and two real materials design case studies. The adaptive policies achieved competitive results in cumulative feasibility counts and feasible hypervolume improvement, outperforming individual acquisition functions that excelled only in a single metric.

By Sushant Sinha, Christofer Hardcastle, Robert Robinson, Shakti Prasad Padhy, Brent Vela, Douglas Allaire, Raymundo Arroyave
arXiv AI
Sep 18

Characterizing Web Search by Conversational LLM Agents: From Search Decisions and Strategies to Results and Responses

The study examines how conversational LLM agents—specifically ChatGPT, Claude, Grok, and DeepSeek—use Web search, combining real user interactions with controlled API experiments. It finds that agents differ in when they decide to search, how they craft queries, and which domains they favor, and that more frequent searching does not always improve answer quality. While most responses are grounded in search results, some claims are unsupported, raising attribution concerns.

By Mahsa Amani, Seungeon Lee, Abhisek Dash, Asmaa El Fraihi, Yunah Jang, Elisabeth Kirsten, Qinyuan Wu, Krishna P. Gummadi, Manish Gupta, Abhilasha Ravichander, Muhammad Bilal Zafar, Soumi Das
arXiv AI
Sep 18

When Hiring Becomes Agent-Mediated: Evaluating Access and Recurrence in Two-Agent R\'esum\'e Screening

The study examines a two‑agent résumé screening process where both employer‑side and candidate‑side agents exchange evidence before deciding who advances, contrasting it with the traditional one‑call automated screening. Using GPT‑5.5 and Claude Opus 4.7 on 600 constructed résumé‑job pairs, the two‑agent method increased the proportion of applications advanced (up to 39.3% for GPT‑5.5) and raised pass rates for borderline cases from 4.5% to 26.2% (GPT‑5.5) and 6.5% to 16.1% (Opus 4.7). The results show that the screening procedure itself, rather than just the underlying model, determines which candidates reach human review and how consistently that access recurs.

By Jian Gao, Hang Jiang
arXiv AI
Sep 18

QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training

QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.

By Davide Vitabile, N. Ranjan, Akshay Nambiar, Kamal K. Gupta, Amril Nazir
arXiv AI
Sep 18

EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Data

EconSkills is a skill library and evaluation framework that transforms verified EconWebArena trajectories into parameterized standard operating procedures for retrieving live economic data. Each skill captures scope, navigation steps, site-specific guidance, verification checks, and recovery steps, replacing instance-specific values with placeholders. The framework evaluates skill transfer to new tasks and the effectiveness of library retrieval, showing that matched skills improve success rates and reduce steps compared to no-skill prompting, while abstraction outperforms raw trajectory replay. At scale, retrieval performs competitively overall, excelling on directly covered tasks and compensating for uncovered tasks through approximate matches.

By Yinzhu Quan, Zefang Liu
arXiv AI
Sep 18

Agentic AI Networking for Heterogeneous Unmanned Aerial Systems in Low-Altitude Wireless Networks

The paper introduces a hierarchical hybrid architecture combining large language models (LLMs) and multi-agent reinforcement learning (MARL) to manage heterogeneous unmanned aerial systems in low‑altitude wireless networks (LAWNs). An outer LLM‑driven loop interprets service requirements and operator intent to reconfigure objectives and resource priorities, while an inner MARL loop executes decentralized policies under the updated game. A logistics‑monitoring case study demonstrates the framework’s ability to coordinate diverse services and adapt to changing conditions without retraining the MARL policies.

By Nguyen Duc Minh Quang, Chang Liu, Shuangyang Li, Derrick Wing Kwan Ng
arXiv AI
Sep 18

AutoData: Agentic Search for Pre-training Data Selection

AutoData is an agent that autonomously searches for pre‑training data selection algorithms by exploring a program space of scoring, stratification, and stochastic rules. It iteratively refines these algorithms using validation feedback from a proxy model, discovering feature interactions that outperform existing human‑designed curation pipelines. The resulting selection recipe, found in an overnight search, transfers to larger scales and improves the downstream CORE metric.

By Yan Meng, Dhruv Srikanth, Bingchen Zhao, Zhengyao Jiang, Yuxiang Wu
arXiv AI
Sep 18

MaSCoD: A Multi-Agent Framework for Structural-Context-Guided Candidate Causal Graph Generation

MaSCoD is a multi‑agent framework that explicitly organizes candidate third variables and local structural patterns before making direct‑edge judgments in causal discovery. Using GPT‑5.4 and GPT‑4o on datasets such as Auto‑MPG, DWD, and Sachs, the study shows that providing structural hypotheses in a separate phase (Full) generally improves recall and F1 scores compared to constructing them during judgment, though it can also raise false‑positive rates. Ablation experiments reveal that the benefit of supplying both structural components is not guaranteed, and the advantage varies across datasets and model backbones.

By Yudai Nakada, Yuichiro Nishiura, Jin Michael Splichal
arXiv AI
Sep 18

PAPC: Platform Mediation for Privacy-Propagation Externalities in AI-Mediated Workflows

PAPC is a platform-mediated mechanism designed to mitigate privacy-propagation externalities in AI‑mediated workflows. It intercepts information‑moving events—such as memory writes, shared‑workspace updates, inter‑agent messages, or tool events—before they update shared state or external channels, using policy, provenance, topology/fanout, privilege, and content signals to decide whether to release a policy‑safe abstraction, quarantine raw content, block a transition, or narrow onward rights. Experiments on retrieval‑memory and multi‑agent workflow benchmarks show that PAPC preserves deterministic task completion while eliminating measured exact raw‑value and external raw‑value exposure, demonstrating that event‑level mediation can serve as a platform‑governance primitive for agent‑mediated online work.

By Tao Huang, Guosen Wu, Chen Hou, Guolong Zheng
arXiv AI
Sep 18

Detecting Soft Errors in Parallel Software with LLM-tuned Instruction Duplication

The paper introduces PaRID, a software-directed soft error detection framework that uses parallel instruction duplication and LLM-tuned performance modeling. PaRID is designed to work at compile time for multithreaded programs, handling mixed serial and parallel regions while keeping performance overhead low. Experiments on NPB benchmarks show that PaRID cuts protection overhead from 162.79% to 59.84% on average and can achieve up to a 5× speedup while still detecting all errors.

By Yafan Huang, Guanpeng Li
arXiv AI
Sep 18

Large Language Model Agents for Evidence Based Genetic Disease Severity Classification

The paper presents an autonomous AI agent that combines ReAct and Retrieval-Augmented Generation to classify the severity of genetic diseases using 10,211 Human Phenotype Ontology terms. It applies ACMG severity guidelines and ACOG quality-of-life criteria, retrieving PubMed literature to produce interpretable reasoning chains and verify claims. The agent achieved 93.55% accuracy on phenotype-level classification and identified 3,283 autosomal recessive gene pairs with severe presentations, with 95.2% concordance in external validation.

By Tohid Ghasemnejad, Ahmadreza Argha, Mark Grosser, John Wang, Min Yang, Thantrira Porntaveetus, Tony Roscioli, Nigel H. Lovell, Mahmoud Aarabi, Hamid Alinejad-Rokny