Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

27,837 stories · RSS feed

arXiv Computation and Language
Oct 5

A Guideline-Augmented Multi-Agent Framework for Schema-as-Code Biomedical Named Entity Recognition

The paper introduces GAMA, a guideline-augmented multi-agent framework designed to improve biomedical named entity recognition (BioNER) using large language models (LLMs). GAMA constructs dataset-specific guideline memory by inducing and verifying annotation rules from training data, then employs a planning component to generate span-type hypotheses with rationales, a coding component to produce schema-constrained entity objects, and a verification module for structural compliance and dual-loop refinement. Experiments across five BioNER datasets demonstrate that GAMA consistently outperforms strong LLM-based baselines, with ablation studies confirming the effectiveness of each component.

By Songtao Li, Yijia Zhang, Shidi Zhang, Jianyuan Yuan, Fengyu Zhang, Hongfei Lin
arXiv Computer Vision
Oct 5

A Vision-Language Model (VLM)-based Pipeline for End-to-End Procedural Modeling of Field-Grown Maize from Point Clouds

The paper introduces an automated pipeline that reconstructs editable 3D procedural models of field‑grown maize directly from raw 3D point clouds, eliminating the need for manual tuning or species‑specific training data. It uses a vision‑language model to annotate leaf midlines in rendered views, then applies deterministic geometric algorithms and differentiable NURBS fitting to generate accurate plant descriptors and refine leaf surfaces. The method achieves a median Chamfer distance of 5.4 mm on 100 diverse maize plants and recovers 99.4% of reference leaves with high overlap, outperforming previous semi‑automated approaches.

By Mozhgan Hadadi, Talukder Z. Jubery, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
arXiv Computer Vision
Oct 5

DEPICT: Scoring Text-to-Image Alignment by Answer Agreement

DEPICT is a new training‑free metric for evaluating text‑to‑image alignment. It replaces fixed reference answers with an agreement rule that compares image‑based and caption‑only responses, weighting questions by how decisively the caption determines them. By merging this agreement score with a holistic score, DEPICT improves negation accuracy dramatically and outperforms existing training‑free metrics while matching or exceeding fine‑tuned evaluators on several benchmarks.

By Vasco Ramos, Sandra Godinho Silva, Joao Magalhaes, Ricardo Rei, Pedro Henrique Martins
arXiv AI
Oct 5

Weave Forcing: Compositional Memory Routing for Interactive Long Video Generation

Weave Forcing is a training‑free framework that enhances interactive long video generation by enabling compositional memory routing. It uses an LLM to split user prompts into character and background slots, then applies masked memory weaving with semantic masks to selectively retrieve relevant historical references. The method also introduces coverage‑adaptive RoPE to adjust temporal offsets based on reference coverage, reducing visual artifacts and improving cross‑shot consistency while preserving visual quality and text alignment.

By Ziyi Wang, Junchi Yao, Heqian Qiu, Wenbo Shi, Chengjiu Wang, Jinyang He, Binkai Hong, Hongliang Li
arXiv Computer Vision
Oct 5

FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views

FRUC is a feedforward 3D Gaussian Splatting framework that reconstructs dynamic scenes from uncalibrated collaborative driving views. It uses a visual‑grounded geometric Transformer backbone for one‑shot, calibration‑free inference and introduces an ego‑centric causal occlusion field to model occlusion evolution across agents. The method performs cross‑agent integration as a deterministic residual denoising process, achieving state‑of‑the‑art rendering quality and efficiency on V2X‑Real and UrbanIng‑V2X datasets.

By Yihang Tao, Yu Guo, Zhengru Fang, Haonan An, Yuguang Fang
arXiv Computation and Language
Oct 5

How Robust Is Multimodal Claim Verification to LLM Rewriting?

The paper investigates how stylistic changes introduced by large language models (LLMs) affect multimodal claim verification, a task that determines whether a textual claim is supported by given evidence. Two rewriting strategies are used: natural rewriting, mimicking typical academic polishing, and controlled injection, adding a single LLM-associated word. Across 11 open‑weight models (2B–38B parameters) from five VLM families, the study finds that most models remain robust to these modifications, showing no significant accuracy drop, though consistent probability shifts—especially under hedging conditions—are observed.

By Yun-Ang Wu, Xanh Ho, Andre Greiner-Petter, Sunisth Kumar, Tian Cheng Xia, Florian Boudin, Akiko Aizawa
arXiv Computation and Language
Oct 5

Automatic Evaluation of Mental Health Stigma in Online Communication

The paper presents a new benchmark for automatically evaluating mental health stigma in online text, featuring a fine‑grained taxonomy that covers stigma mode, domain, and specific components across multiple mental health conditions. The authors annotate naturally occurring news and social media posts and test large language models and classifiers for sentiment, toxicity, and hate speech, finding that these models poorly capture stigma and often overpredict it without explicit rules. The benchmark, annotations, exemplar cases, and code are publicly released on GitHub.

By Naomi Baes, Jemima Kang, Nick Haslam, Chris Groot, Alsa Wu, Luc Raszewski, Yulia Otmakhova
arXiv Computation and Language
Oct 5

Adaptive Mutual Distillation for Balanced Multi-Task Post-Training of Large Language Models

The paper introduces Adaptive Mutual Distillation (AMD), a post‑training framework that jointly trains two large language models using different task‑balancing strategies. AMD evaluates and selects distillation weight adjustments via short training probes and task‑wise validation, leading to models that outperform supervised fine‑tuning baselines on six benchmarks across three backbones. Merging the two AMD models further improves performance, surpassing multi‑task fine‑tuning by an average of 2.91 points.

By Baohang Li, Xiaocheng Feng, Yichong Huang, Chengpeng Fu, Wenshuai Huo, Zekun Zhou, Zekun Yuan, Tingjia Zhang, Bing Qin
arXiv Computation and Language
Oct 5

Evaluating VQA in Vision Language Models using Cooperative Principles

The paper evaluates Vision Language Models (VLMs) on Visual Question Answering tasks where questions violate Grice's maxims. By generating question modifiers that add non-essential, ambiguous, or false information, the authors show that VLMs such as ChatGPT, Claude, Gemini, and Llava exhibit reduced performance. They also compare human pragmatic reasoning to VLM reasoning, noting differences in how each handles human‑induced versus AI‑generated violations, and find that humans spend less time resolving VLM‑induced violations while VLMs are less accurate in those cases.

By Monika Shah, Sudarshan Balaji, Somdeb Sarkhel, Sanorita Dey, Deepak Venugopal
arXiv Computation and Language
Oct 5

The Geometry of Knowledge Accessibility in Large Language Models

The paper investigates how easily large language models (LLMs) can retrieve knowledge for a given query, introducing the concept of knowledge accessibility. It discovers that the accessibility of a query is reflected in its geometric position in the model’s representation space: queries closer to a central point are more accessible, while those farther away are less so. This geometric insight identifies a knowledge boundary, shows that accessibility ordering is consistent across datasets, and informs which interventions—such as query rewriting, chain-of-thought reasoning, or retrieval—are most effective depending on a query’s position relative to the center.

By Lihu Chen
arXiv Computation and Language
Oct 5

OmniConfess: Eliciting Token Confessions to Mitigate Omni-Modal Hallucination

OmniConfess is a training‑free method designed to reduce hallucinations in omni‑modal large language models (OmniLLMs) that handle text, images, audio, and video. The approach fixes a candidate response and re‑scores it at token resolution while selectively intervening on evidence from each modality, producing a token‑by‑channel confession that shows which evidence supports each part of the response. Using this confession, OmniConfess preserves grounded content and corrects commitments that rely on irrelevant or contradictory evidence. The authors evaluated the method on OmniHalluBench, a 3,540‑example benchmark drawn from six datasets across multiple modalities and tasks, and found that OmniConfess mitigates hallucinations across diverse settings.

By Huiqiang Rong, Haoran Luo, Hui Feng, Zhonghong Ou, Kaiwen Xue, Guoxin Zhang, Yifan Zhu
arXiv Machine Learning
Oct 5

Drive vs. Decay: On the Training Dynamics of Joint-Embedding Predictive Architectures

The paper introduces a stability theory for Joint-Embedding Predictive Architectures (JEPAs), showing that training dynamics involve a driving force and a decay effect that determine representation collapse. By linearising the gradient flow, the authors derive a per‑mode stability ratio that separates data‑side and predictor‑side contributions, predicting a phase boundary confirmed across 800 configurations. Using this insight, they propose ResidualPred, a transformer predictor that biases attention toward the identity at initialization, improving representation rank and downstream accuracy on tabular and image benchmarks.

By Jos\'e Lucas De Melo Costa, Seong Woo Ahn, Fabrice Popineau, Arpad Rimmel, Bich-Li\^en Doan
arXiv Computation and Language
Oct 5

A generative-informed neuro-symbolic framework for syntactic ambiguity resolution: Evidence from Arabic DPs

The paper introduces a generative‑informed neuro‑symbolic framework that combines generative syntactic theory with AraBERT to resolve structural ambiguity in Modern Standard Arabic noun phrases. By treating ambiguity as a candidate‑based decision task, the model explicitly constructs and evaluates linguistically motivated alternatives, achieving high accuracy (96.88%) and strong F1 scores on an unseen evaluation set. Analysis shows uneven performance across attachment types, with near‑perfect recall for high/VP attachment but lower recall for low/NP/embedded attachment, highlighting challenges in recovering embedded interpretations.

By Mohammed Damom, Muneef Y. Alshawsh, Ashraf A. Naji, Mustafa Ali Alhamzi, Fawwaz An-Nashef, Jameel Ahmed Elayah, Mohammed Q. Shormani, Noman AL-Sayadi
arXiv Machine Learning
Oct 5

Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

The paper investigates how to reduce the per-user adaptation burden in personalized large language models by separating reusable personalization capacity from user-specific adjustments. Through empirical studies, it shows that shared low‑rank factors can capture much of the cross‑user structure, while a tiny user code suffices for individual correction. The proposed LINEUP framework achieves state‑of‑the‑art performance on six personalized tasks while using only eight scalars per user compared to millions of private‑LoRA parameters.

By Songyuan Sui, Srikanth Malla, Chiho Choi, Joon Hee Choi
arXiv Computer Vision
Oct 5

Spatial Memory Intelligence: Endowing World Models with Understanding-Driven Long-Term Memory

The paper introduces Spatial Memory Intelligence (SMI), a framework that enhances long‑video world models by systematically managing spatial memory using an understanding model. SMI employs four coordinated operations—spatial clustering, within‑cluster sparsification, action‑aware retrieval, and reliability‑aware filtering—to handle increasingly complex and lengthy memory sequences. Experiments across various baselines and benchmarks show that SMI improves memory sparsity, generation stability, and spatial consistency, demonstrating its effectiveness and generalizability.

By Ying Yang, Guiyu Zhang, Lianghua Huang, Chang Nie, Chenyang Si, Haofan Wang, Shaoshuai Shi, Li Jiang
arXiv AI
Oct 5

What do your logits know?

The paper investigates how much information can be extracted from different internal representations of vision‑language models, focusing on two bottlenecks: low‑dimensional projections of the residual stream (via tuned lenses) and the final top‑k logits. It systematically compares the amount of retained information at these levels and finds that even the easily accessible top‑logit bottleneck can leak task‑irrelevant details from image queries, sometimes matching the leakage seen from full residual projections.

By Masha Fedzechkina, Eleonora Gualdoni, Rita Ramos, Sinead Williamson
arXiv AI
Oct 5

Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning

The paper introduces GraphMemory, a lightweight graph-based memory system designed to improve token efficiency in test-time continual learning for large language models. By retrieving only relevant subgraphs for each query, GraphMemory keeps the amount of retrieved memory constant as more examples are processed, avoiding the token cost and performance degradation of traditional shared-context approaches. Experiments demonstrate that GraphMemory achieves competitive downstream performance while using roughly 81‑85% fewer memory‑construction tokens than baseline methods.

By Yehya Farhat, Michael Desmond, Anastasios Kyrillidis
arXiv AI
Oct 5

PAPER2LLM++: Continual Self-Evolution of LLMs from Research Papers

PAPER2LLM++ is a framework that enables large language models to continually evolve by learning from new research papers. It treats each paper as a source of evidence and supervision, extracting findings, testing them against the current model, and converting them into learning signals when necessary. A try‑evaluate‑commit loop ensures that updates improve targeted behaviors without forgetting prior gains or harming general capabilities, allowing models to progressively incorporate new findings while retaining earlier improvements.

By Hongji Pu, Yilun Zhao, Wenpeng Yin