Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

31,598 stories · RSS feed

arXiv AI
Sep 18

Geopolitical Divisions Across Languages in Large Language Models

The study examines how language influences AI chatbot responses to questions about the war in Ukraine, revealing that the same AI systems (GPT, Claude, Gemini) produce varying political stances across 112 languages. By evaluating 20 statements in 112 languages, the researchers found that Russia‑leaning versus Ukraine‑leaning answers differ by language, mirroring global political attitudes such as public support for Russia, UN voting patterns, and aid levels. This pattern persists across all three models and even when specific statement pairs are removed, suggesting that information warfare could embed geopolitical biases into AI training data.

By Maxim Chupilkin
arXiv AI
Sep 18

Can Data Attribution Filter Out Subliminal Learning? Not Reliably

The paper investigates whether gradient‑based data attribution methods can filter out subliminal learning in language models. Three attribution techniques—GradCos, a contrastive variant, and EK‑FAC—are evaluated on three models and compared to divergence tokens, a strong baseline that requires counterfactual teacher models. Results show that EK‑FAC can mitigate some subliminal effects at the token level, but overall the methods are inconsistent and generally less effective than divergence tokens, especially when filtering entire samples.

By Moritz Weckbecker, Sweta Jena, Jonas M\"uller, Ponnurangam Kumaraguru, Sebastian Lapuschkin, Wojciech Samek, Louis Jaburi, Gon\c{c}alo Paulo
arXiv AI
Sep 18

Tailored to you: longitudinal effects of personalising language models

The study examined how personalising language models affects user interactions over five days, comparing a non‑personalised baseline with memory‑based and survey‑based personalisation. Results showed that many interaction changes were due to repeated exposure, but personalisation influenced specific behaviors: memory‑based users disclosed more and found the model less creepy, while survey‑based users felt more regret about sharing personal data. The authors emphasize the nuanced, approach‑specific impacts on user attitudes and the need for careful design of personalised AI.

By Canfer Akbulut, Justine Breuch, Arianna Manzini, Lujain Ibrahim, Matija Franklin, Roma Patel, Iason Gabriel, Kristian Lum, Laura Weidinger
arXiv AI
Sep 18

FreqCondNorm: Towards Cross-domain Predictive Maintenance through a Frequency-Conditioned Transformer Foundation Model

The paper introduces FreqCondNorm, a Transformer-based architecture that adds a frequency-conditioned normalization layer to unify heterogeneous time-series data for predictive maintenance. The model is pretrained on five public datasets using masked auto‑encoding and contrastive learning, achieving 99.2% accuracy on CWRU and 82.1% zero‑shot accuracy on MFPT, showing strong transfer across sampling frequencies. However, it does not improve remaining useful life prediction, indicating a mismatch between pretraining and RUL objectives that requires further study.

By Zaynab Raounak, Camille LHermine, Zhiguo Zeng
arXiv AI
Sep 18

How Do Agent Harnesses Create Value? Planning Information and Release Control in Stateful LLM Agents

The paper investigates how agent harnesses—specifically planning guidance, execution organization, and completion verification—affect performance in retail and airline pilot tasks. By comparing fixed, task‑specific plans to shuffled policy text of equal length, the study finds that fixed plans improve success rates by about 7 percentage points, especially on complex tasks. A read‑only verifier rejects a majority of invalid episodes while incurring minimal cost, and its impact varies with the penalty for erroneous acceptance, often matching the full planning‑plus‑verification benefit at a lower cost.

By Yukun Zhang, Kemu Xu, Yishen Chen
arXiv AI
Sep 18

Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models

Deep Noir is a framework that autonomously discovers optimal activation‑steering parameters in transformer models by leveraging Logit Lens convergence and causal head‑level attribution. It demonstrates significant performance gains across models ranging from 1B to 9B parameters, improving spam detection by up to 42 percentage points and SST‑2 sentiment classification by 13.1 percentage points without code changes. The approach also reveals that increased steering magnitude expands a predictable prompt‑injection attack surface, highlighting security implications for agent systems using steered classifiers.

By Frank E. Bobe III, Gregory D. Vetaw, Darshan W. Bryner, Matthew G. Cook, Jose L. Salas-Vernis
arXiv AI
Sep 18

Q&A on Any Spreadsheet Requires Interpreting Its Grid Structure

The paper proposes a new framework that improves spreadsheet chunking for large language model (LLM)-driven retrieval-augmented generation (RAG) systems by adding semantic cell annotations. This approach outperforms current state‑of‑the‑art methods but is limited by the inherent two‑dimensional, unstructured nature of spreadsheets, which cannot be fully captured by finite classification categories. The authors argue that future progress requires dimensionality‑reduction techniques to flatten spreadsheets into one‑dimensional text, simplifying downstream RAG interpretation and generation.

By Zofia Smole\'n
arXiv AI
Sep 18

An Empirical Study of Harness Design for Coding Agents

The study investigates how individual components of a coding harness—planning, action space, and context management—affect autonomous coding agents’ performance. By fixing the execution loop and varying these components across 176 settings on SWE‑Bench Verified and Terminal‑Bench 2.1, the authors find that context management is most valuable when context windows are tight, staging rule‑based elision before LLM summarization yields the best efficiency, planning serves as an accuracy scaffold for weaker models and a cost saver for stronger ones, and predefined tools help models with limited bash skills while bash‑capable models benefit from a bash‑only interface. Trajectory‑level analysis shows that context management lengthens execution paths, planning alters where trajectories terminate, and the action space determines code granularity, offering a modular framework for future harness design.

By Run-Ze Fan, Zihao Zhang, Simin Ma, Yebowen Hu, Shouju Wang, Kaiqiang Song, Fei Liu, Hamed Zamani, Xiaoyang Wang
arXiv AI
Sep 18

MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators

MeshKV is a Network‑on‑Chip key‑value cache fabric designed to improve transformer decoding on tiled accelerators. It uses packetized flows, TaKV affine striping, Mare multicast with duplicate suppression, and Pad to overlap prefetch, multiply, and softmax operations, thereby converting back‑pressure into useful KV transfer. In an 8×8 FPGA prototype with LLaMA‑2‑7B and Mistral‑7B, MeshKV cuts interconnect traffic by up to 58 %, doubles KV bandwidth utilization, and boosts multi‑stream throughput by up to 1.9×.

By Dong Liu, Yanxuan Yu
arXiv AI
Sep 18

Layer-wise Curriculum Learning for Efficient LLM Compression

The paper proposes a layer-wise curriculum learning strategy for compressing large language models (LLMs). By partitioning the model into layer segments and starting training with easier tasks before progressing to harder ones, the method accelerates convergence and stabilizes knowledge transfer from teacher to student models. Additional techniques such as feature caching with multi-threading improve GPU utilization, leading to state‑of‑the‑art compression results and over 50% reductions in memory usage and training time on BERT and GPT‑2, while outperforming other pruning methods on LLaMA‑family and Qwen models.

By Donggeon Lee, Dooyeon Na, Seungmin Oh, Jongbin Ryu
arXiv AI
Sep 18

YNU-HPCC at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Using Multiple Prediction Headers

The YNU-HPCC team participated in Subtask A of SemEval‑2025 Task 11, "Bridging the Gap in Text‑Based Emotion," using a RoBERTa model with a single prediction head to process one emotion at a time. Their system achieved an official ranking score of 0.44 across all languages after translating the dataset into English with Google Translate. Analysis showed that a single head outperformed six simultaneous heads and that training on the uniformly translated English data improved results.

By Hao Yang, Jin Wang, Xuejie Zhang
arXiv AI
Sep 18

Why Pretraining Fails to Share Cross-Lingual Knowledge

The paper investigates why large language models (LLMs) fail to transfer knowledge across languages. By pretraining 360M- and 7B-parameter models, the authors show that poor cross‑lingual generalization arises during pretraining and persists despite standard fixes. Using a controlled bilingual setup with identical text but disjoint token spaces, they demonstrate that token disjointness alone causes knowledge compartmentalization, and that mapping languages into a shared token space via word‑wise translation markedly improves cross‑lingual performance, recovering up to 12.6% of native‑language learning efficiency.

By Adam Gaber, Uriel Dolev, Elisabeth Fittschen, Bobby Cheng, Yuval Marton, Leshem Choshen
arXiv AI
Sep 18

Can Vision-Language Models Judge Olympic Diving? From Reasoning to Scores in Zero-Shot Action Quality Assessment

The paper investigates whether open‑source Vision‑Language Models (VLMs) can perform zero‑shot action quality assessment (AQA) on Olympic diving videos. Using the AQA‑7 benchmark, the authors propose a regression framework that combines VLM‑generated semantic reasoning, phase‑level sub‑scores, TF‑IDF vectorization, dimensionality reduction, and ensemble learning to predict final competition scores. While individual VLMs achieve moderate Spearman correlations (<0.32), the ensemble approach boosts performance to 0.67, demonstrating that VLM‑derived textual reasoning features are more informative than raw numerical sub‑scores for AQA. whyItMatters":"The study shows that VLMs can serve as explainable, semi‑automated tools for evaluating sports performance, potentially aiding expert judging in complex, subjective Olympic events."

By Henry O. Velesaca, David Freire-Obregon, Luigi Miranda, Abel Reyes-Angulo
arXiv AI
Sep 18

How to Guide Your Language Flow

The paper introduces probe guidance, a technique that leverages frozen internal states of a diffusion model to generate a guidance signal without requiring an extra forward pass during inference. This method improves continuous diffusion language models, achieving state‑of‑the‑art results on unconditional generation and enhancing performance on multiple‑choice question answering for a 1.7B model. The authors also use probes to analyze autoguidance, revealing that the weak model must originate from a low‑entropy training region to align dynamics with the strong model.

By Rohit Dilip, Tianrong Chen, Yuyang Wang, David Van Valen, Joshua Susskind, Miguel Angel Bautista
arXiv AI
Sep 18

From Rollout to Reset: A Graph-Based Harness for Autonomous Long-Horizon Manipulation Evaluation

The paper introduces HALTER, a graph-based system that automates the reset and evaluation of long-horizon robot manipulation tasks. HALTER constructs a spatial scene graph from point clouds and vision models, uses an LLM to score rollouts, plan resets, and verify success, all without labeled success images. In experiments on a Franka arm, HALTER restores scenes in 76% of episodes, improves skill completion estimation, and reduces operator time by 72% compared to manual reset.

By Jing Jiang, Yue Yang, Xinkai Jiang, Gedas Bertasius, Daniel J. Szafir, Rudolf Lioutikov
arXiv AI
Sep 18

From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

The survey "From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning" reviews over 300 works on efficient multimodal learning (EML), proposing a structured taxonomy that spans model, algorithm, and system layers. It synthesizes how cross‑layer co‑design addresses the Efficiency‑Utility‑Privacy trade‑off and illustrates this through a case study of multimodal large language models. The paper also offers optimization blueprints for various domains, discusses a shift toward self‑regulating intelligence, and outlines open challenges for future EML research.

By Pan Wang, Siwei Song, Hui Ji, Siqi Cao, Heng Yu, Zhijian Liu, Huanrui Yang, Yingyan Celine Lin, Beidi Chen, Mohit Bansal, Xiaoming Liu, Pengfei Zhou, Ming-Hsuan Yang, Tianlong Chen, Jingtong Hu
arXiv AI
Sep 18

Efficiently Linking Unstructured Data for Multi-step Reasoning

The paper introduces DASE, a query engine designed to efficiently link unstructured data for multi-step reasoning tasks. DASE combines a multi-step reasoning model, a sparse materialized embedding-similarity join index (SemJI), and a co‑designed execution layer to perform multi‑attribute filtering, multi‑vector search, exact relational joins, and thresholded embedding‑similarity joins. In scientific discovery workloads, DASE outperforms traditional RDBMS, rerank, and vector‑database baselines by 6x to 46x in retrieval speed while maintaining comparable recall, and it serves as a high‑recall prefilter that reduces downstream LLM evaluation cost and improves accuracy on benchmarks such as SemBench E‑Commerce.

By Jiaming Liang, Haydn Jones, Jacob R. Gardner, Mark Yatskar, Zachary Ives
arXiv AI
Sep 18

For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances

For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances explores whether a language model can embed a signal in natural language that another independent instance can detect without shared memory or coordination training. The study introduces a cooperative signalling game where a Sender describes two words, one hidden, and a Receiver must identify the target. Seven contemporary models from four architectural families were tested on 300 word pairs, revealing that most struggle to coordinate when signals must be undetectable, though one frontier model performs near-perfectly even after filtering, and that models can also use this capability for deliberate misdirection.

By Alexander Shirnin, Aleksey Kudelya
arXiv AI
Sep 18

AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation

AURORA is a natural‑language‑driven framework that treats air‑ground scenario generation as a compilation process with verification. It introduces the Air‑Ground Scenario Graph (AGSG), a typed intermediate representation linking agents, missions, events, communication, and success conditions, enabling joint grounding, temporal planning, pre‑execution checks, runtime verification, failure localization, and bounded repair. The authors also present AURORA‑Bench to evaluate not only execution but faithful realization of requested interactions, showing that structured execution and runtime verification improve reliability and that explicit intermediate representations facilitate verifiable and repairable co‑simulation.

By Keshu Wu, Hao Zhang, Rui Gan, Xiangbo Gao, Xiaopeng Li, Zhengzhong Tu, Yang Zhou
arXiv AI
Sep 18

DeltaSelect: Affordable A/B Testing for Coding Agents

DeltaSelect is an open‑source method that selects a small, cost‑effective set of tasks for A/B testing coding agents, using Pearson correlation to identify tasks whose single‑run results reliably reflect full‑benchmark performance. It maps fractional verifier outputs to a common score via linear regression and operates within a specified dollar budget, making it suitable for repeated baseline‑versus‑candidate comparisons during development rather than for model ranking. In a case study with gpt‑5.6‑luna, DeltaSelect reduced costs by 58.1% and improved calibrated scores across 13 evaluations.

By Nicholas J. Conn