Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,598 stories · RSS feed

arXiv Computer Vision
Sep 18

Monocular Visual Odometry without Calibration or Test-time Optimization

The paper introduces CalfVO, a monocular visual odometry system that operates without camera intrinsics, test‑time optimization, bundle adjustment, or loop closure. Using a transformer, it predicts relative poses with separate rotation and translation confidences over overlapping image windows, then aggregates these predictions via a confidence‑weighted module to produce a single trajectory. CalfVO achieves the highest accuracy among calibration‑free methods across five benchmarks and runs at 53 FPS, outperforming all baselines.

By Vladimir Yugay, Duy-Kien Nguyen, Theo Gevers, Cees G. M. Snoek, Martin R. Oswald
arXiv AI
Sep 18

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

BioPhys-Bridge is a newly released benchmark dataset designed to evaluate language models on evidence‑grounded scientific reasoning within biophysical literature. Each of its 500 cases includes evidence blocks, stable IDs, quantitative values, units, equations, assumptions, mechanisms, and next‑step decisions, covering six biological domains and nine physical model families. The dataset enforces strict quality gates and has already been evaluated against several models, with DeepSeek‑V4‑Flash achieving the highest evidence‑ID F1 score of 0.360.

By Qingyang Xu
arXiv AI
Sep 18

From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization

The paper introduces a 202-scenario benchmark to evaluate how large language models (LLMs) handle safety-critical authorization decisions for vehicle voice commands. It tests two local open-weight models and three API-based LLMs, finding alignment scores ranging from 40.1% to 89.1% and noting persistent false execution errors. The study concludes that structured LLM decisions alone are insufficient for safety, recommending an independent enforcement layer to verify tool permissions and vehicle-state constraints before any vehicle function is invoked.

By Diba Afroze, Xingli Zhang, Yazhou Tu, Xiali Hei
arXiv AI
Sep 18

Replan, Repair, or Edit? A Unified Empirical Evaluation of Travel Agents for Itinerary Revision under Resource Disruptions

The paper evaluates three approaches—LLM-Z3 full replanning, IPyHOPPER hierarchical repair, and iTIMO local-revision—for revising travel itineraries after disruptions such as flight cancellations or hotel unavailability. Using two TREK-derived benchmark sets (500 single-disruption cases and 200 compound-disruption cases), the study compares effectiveness, plan stability, and computational cost. Results show LLM-Z3 with Gemini achieves the highest compound-disruption success, IPyHOPPER nearly matches single-disruption success while preserving more of the original itinerary, and iTIMO makes fewer edits but consumes more tokens, offering practical guidelines for balancing feasibility recovery, commitment preservation, and computational cost.

By Xiaofei Yuan, Yan Zhang, Shaobo Qiao, Huangleshuai He, Leyan Ni, Mingchen Ju, Lujia Yang, Sijia Xu, Yifu Tang, Zhengyi Yang
arXiv AI
Sep 18

WiCleanData: Guaranteeing the Type Consistency of Wikidata by Taxonomy Refinement and Constraint Enforcement

WiCleanData is a refined version of Wikidata that eliminates type inconsistencies and constraint violations. The authors built an automated pipeline that cleans the taxonomy with language‑model assistance, aggregates type constraints hierarchically, and filters facts to ensure no type violations remain. The resulting knowledge graph is publicly available through a web interface for easy exploration and downstream use.

By Yiwen Peng (IP Paris), Marc Jeanmougin (IP Paris), Thomas Bonald (IP Paris)
arXiv AI
Sep 18

Language-model groups overstate consensus when replaying human deliberation on a reasoning task

The study compares human deliberation in Wason selection tasks with large language model (LLM) agent groups that are seeded with participants’ pre-discussion beliefs. Across various scoring definitions, human consensus rates ranged from 24.0% to 57.0%, whereas LLM agents consistently achieved higher consensus, with gaps of 34–44 percentage points in two sensitivity analyses. Even when early stopping was removed or memorizable answers were eliminated, LLM groups still reached near-unanimous agreement, often on incorrect answers, indicating that simulated consensus does not reflect collective accuracy.

By Tengfei Shao
arXiv AI
Sep 18

KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

KoNeoBench is a curated dataset designed to evaluate large language models’ understanding of Korean neologisms. It contains 1,785 recently attested Korean words from online news since 2020, each accompanied by usage examples, word‑formation analyses, and dictionary‑style definitions. The authors define four evaluation tasks, report results from recent models and a human baseline, and find that current LLMs struggle with recovering source components, distinguishing semantic categories, and generating accurate definitions.

By Soha Lee, Soojin Lee, Heesung Yang, Hyunju Song, Hyunji Lee, Jinsan An, Jeongwan Shin, Jin Hyun Park, Jun Lee, Hyeyoung Park, Kilim Nam
arXiv AI
Sep 18

SCICONVBENCH: Benchmarking LLMs on Multi-Turn Clarification for Task Formulation in Computational Science

SCICONVBENCH is a benchmark designed to evaluate large language models (LLMs) on multi‑turn clarification tasks in computational science. It focuses on two key abilities: eliciting missing information (disambiguation) and resolving contradictory requests (inconsistency resolution) across four domains—fluid mechanics, solid mechanics, materials science, and partial differential equations. The benchmark pairs a structured task ontology with a rubric‑based evaluation framework, measuring LLM performance in clarification behavior, conversational grounding, and final‑specification fidelity, and reveals that even top models only resolve about 52.7% of disambiguation cases in fluid mechanics while often making ungrounded assumptions.

By Nithin Somasekharan, Youssef Hassan, Shiyao Lin, Gihan Panapitiya, Patrick Emami, Anurag Acharya, Sameera Horawalavithana, Shaowu Pan
arXiv AI
Sep 18

Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design

The paper investigates how reinforcement learning can be effectively applied to diffusion models for visual tasks, focusing on the role of likelihood estimation. By systematically separating policy‑gradient objectives, likelihood estimators, and rollout sampling schemes, the authors find that using an evidence lower bound (ELBO) based likelihood estimator computed from the final generated sample is the key factor for stable and efficient RL optimization, outweighing the choice of loss function. Experiments on SD 3.5 Medium across multiple reward benchmarks confirm that this approach improves GenEval scores from 0.24 to 0.95 in 90 GPU hours, outperforming existing methods such as FlowGRPO and the current state‑of‑the‑art without reward hacking.

By Jaemoo Choi, Yuchen Zhu, Wei Guo, Petr Molodyk, Bo Yuan, Jinbin Bai, Yi Xin, Molei Tao, Yongxin Chen
arXiv Machine Learning
Sep 18

Beyond Flattened Tokens: Structure-Preserving EEG Decoding with Reusable TriDim Blocks

The paper introduces TriDim, a reusable block that preserves the three EEG axes—channel, short‑term temporal, and long‑term temporal—by applying feed‑forward transformations and cross‑axis attention. Stacking these blocks yields TriDimEEG, a standalone EEG decoder that outperforms fifteen other models on eight datasets, achieving a 4.3% relative accuracy gain. Replacing Transformer blocks in existing EEG foundation models with TriDim blocks improves downstream accuracy by 7.4% on average while reducing parameters by 17.0% to 47.3%.

By Shiyue Su, Song Wang, Zekai Zhan, Junjie Zeng, Ziling Lu, Zongsheng Li, Xinyuan Ye, Zhiyuan Ma, Xinke Shen, Quanying Liu
arXiv Machine Learning
Sep 18

Transformer fault diagnosis using an efficient simulation-driven variational quantum classifier with domain-aware feature encoding

The paper introduces a simulation-driven framework for diagnosing transformer faults using dissolved-gas analysis. It employs a variational quantum classifier (VQC) with a two-qubit representation, domain-aware feature modeling from Duval geometry, and a hybrid ZX-YY feature map to capture nonlinear gas interactions. Extensive simulation, noise-aware emulation, and limited hardware tests show high diagnostic accuracy, strong generalization, and robustness to realistic noise while using minimal quantum resources.

By Huy Hoang Le, Ba Tu Phung, Dai Huynh, Kim-Anh Nguyen
arXiv Computer Vision
Sep 18

Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models

The paper introduces REVEAL, a diagnostic benchmark that stresses Video‑Language Models (VidLMs) on five controlled probes—camera‑motion sensitivity, cross‑frame integration, video sycophancy, language‑only shortcuts, and temporal expectation bias—to assess how well these models encode and use visual evidence. Experiments on 12 VidLMs reveal systematic failures: some visual signals are never reliably encoded, while others are overridden by model priors, leading to performance below chance on several probes that humans solve with high accuracy. Mechanistic probes further pinpoint where and why visual evidence is lost, demonstrating that under assertive prompts a model’s output becomes nearly invariant to real versus random video input, rendering visual evidence causally inert.

By Sethuraman T V, Savya Khosla, Aditi Tiwari, Vidya Ganesh, Rakshana Jayaprakash, Aditya Jain, Vignesh Srinivasakumar, Onkar Kishor Susladkar, Srinidhi Sunkara, Aditya Shanmugham, Rakesh Vaideeswaran, Abbaas Alif Mohamed Nishar, Simon Jenni, Rohan Maheshwari, Derek Hoiem
arXiv Machine Learning
Sep 18

Measurement Under Selection: Decoy-Calibrated Failure Audits for Language Models

The paper introduces Janus, a method for validating error patterns in language models by comparing error rates across predefined yes/no properties and using shuffled decoy labels to set significance thresholds. Janus requires that a pattern’s error difference surpasses the decoy-derived threshold and is replicated on held‑out data before reporting. Experiments on a controlled code‑finding task confirm several meaningful error patterns, while on MuSiQue and LongBench v2 Janus reports no confirmed patterns for the tested properties, contrasting with standard shuffling tests that sometimes confirm patterns.

By Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh
arXiv Computation and Language
Sep 18

Towards Proactive Detection of User-Side Implicit Conflicts in Human-LLM Dialogue

The paper introduces UC-Bench, a human‑annotated benchmark for detecting user‑side implicit conflicts in Human‑LLM dialogue, a problem largely overlooked compared to LLM‑side conflicts. Experiments show current LLMs struggle with these conflicts, especially when they stem from implicit incompatibilities in dialogue history. To address this, the authors propose SynUC, a constraint‑guided data synthesis method that generates a new training set, UC‑Data, which improves performance of lightweight LLMs on UC‑Bench compared to larger general‑purpose models and existing synthesis approaches.

By Jinqiang Wang, Tao Zhu, Huansheng Ning
arXiv Computation and Language
Sep 18

JustMem: Just-Enough Memory Access for Long-Term Conversations

JustMem is a new memory system for long‑term conversational AI that stores conversation history as compact atomic memories and adapts its access strategy to each query. It introduces two dimensions of memory access—discovery breadth and reading fidelity—implemented through LOOKUP for local evidence, COMPOSE for distributed evidence, and REPLAY for fidelity‑sensitive evidence. Experiments on LoCoMo and LongMemEval‑S show that JustMem outperforms competing memory systems in accuracy and recall while using fewer generative‑model tokens for memory construction and inference.

By Guanhua Chen, Yanting Wang, Wenjing Zhi, Lei Sha
arXiv Computation and Language
Sep 18

Before the Arrest: Benchmarking LLMs on Criminal Profiling from Incomplete Evidence

The paper introduces the Profiling, Investigation, and Judgment (PIJ) benchmark, which contains 2,500 real homicide cases from five countries to evaluate large language models (LLMs) on pre‑arrest criminal investigation tasks. It assesses LLMs across criminal profiling, crime process reconstruction, and sentence prediction, revealing that performance drops as tasks require more implicit reasoning about unknown suspect profiles. The study finds that LLMs lag behind human experts, especially on inferential tasks like motivation and victim‑offender relationships, and exhibit biases in gender, age, and motive attribution.

By Yutong Yao, Yanjie Cao, Guanhua Chen, Xu Yang, Junchao Wu, Zeyu Wu, Lidia S. Chao, Derek F. Wong
arXiv Computer Vision
Sep 18

From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning

The paper introduces TAR (Traffic Anomaly Reasoning) and its evaluation suite TAR-Bench, designed to train and assess video‑language models on tasks beyond simple anomaly detection. TAR offers 44,040 chain‑of‑thought annotations covering 10 tasks across 3,670 CCTV videos, while TAR‑Bench supplies 960 human‑curated test annotations from 80 clips of 17 YouTube videos. Experiments show that strong question‑answering performance does not guarantee temporal or scene reasoning, and multi‑task fine‑tuning on TAR consistently improves model scores, with a 10‑task model outperforming its zero‑shot baseline by 21.4 points. The datasets serve as the official training and evaluation data for AI City Challenge 2026 Track 3.

By Han Zhang, Yilin Zhao, Zaid Pervaiz Bhat, Zheng Tang, Varun Praveen, Vidya N. Murali, David C. Anastasiu, Tomasz Kornuta
arXiv Machine Learning
Sep 18

Placement Is Free, Composition Is Not: The Latin Square as a Provably-Balanced Construction for Heterogeneous Sequence-Mixer Stacks

The paper introduces Aether‑7B‑5Attn, a 6.59B‑parameter mixture‑of‑experts Transformer that arranges seven distinct sequence‑mixing mechanisms in a 7×7 Latin square across 49 layers, ensuring each mechanism appears exactly once per row and column. Experiments with a 4×4 Latin‑square proxy and larger 1.514B‑parameter models show that balanced heterogeneous composition across depth yields near‑optimal performance, while clustering mechanisms or using a homogeneous stack incurs measurable penalties. The authors provide detailed cost profiles, multilingual evaluations, a causal‑safety audit, and release all code and weights.

By Taebong Kim, Youngsik Hong, Minsik Kim, Sunyoung Choi, Jaewon Jang, Minseo Kim
arXiv AI
Sep 18

Local Sparsity Enables Unsupervised LLM Safety Detection

The paper proposes a new unsupervised safety detection method for large language models that relies on anomaly detection rather than supervised training on unsafe data. By leveraging local sparsity in a linear representation space obtained via a sparse autoencoder, the authors develop a framework for locally masked SAE-based anomaly detection, providing theoretical support and empirical validation across multiple architectures and datasets. When calibrated with only 1% out-of-distribution data, the method achieves near‑optimal performance while using just 1–2% of SAE neurons for computation.

By Xin Chen, Gil Kur, Alexander Shevchenko, Andreas Krause
arXiv Machine Learning
Sep 18

Uni-LaDiR: Latent Diffusion Unifies Multimodal Reasoning

Uni-LaDiR (Unified Latent Diffusion Reasoner) is a new framework that unifies multimodal reasoning by mapping teacher reasoning steps from different modalities into a shared latent space of thought tokens. It employs a diffusion model to predict the next block of thought tokens, jointly training the encoder and reasoner with shared weights to ensure tokens are both useful and predictable. The approach achieves relative gains of 7.3% on visual reasoning benchmarks and 6.1% on robot manipulation tasks compared to the strongest baselines.

By Haoqiang Kang, Yizhe Zhang, Nikki Lijing Kuang, Yian Ma, Lianhui Qin