← Back to all news

Large language models

Model releases, architecture work and prompting research on large language models — from frontier-lab announcements to the arXiv papers behind them.

27,837 stories · RSS feed

arXiv Machine Learning
Oct 5

Constraint-Aware Training

The paper introduces a constraint‑aware training objective that externalizes program analysis constraints used during inference, avoiding redundant learning of these constraints during model training. It presents three theorems demonstrating that this approach can reduce model size and improve data efficiency. A controlled synthetic experiment confirms that constraint‑aware training achieves lower prediction loss than standard cross‑entropy training at the same parameter count and data volume.

By Jinwoo Kim
llms
More like this →
arXiv AI
Oct 5

JOVE: Joint Execution and Verification for Resource-Aware LLM Task Graphs

JOVE is an online framework that jointly assigns large language model (LLM) executors and selects intermediate outputs for paid verification in directed acyclic task graphs. It balances immediate execution costs with future learning by solving per-query mixed‑integer linear programs and updating task‑dependent LLM quality estimates through verification feedback. Experiments on four reasoning benchmarks show that JOVE matches standard inference accuracy while cutting average cost and latency by at least 3.17×.

By Haoran Zhang, Dongjun Kim, Seohyeon Cha, Kevin S Chan, Ananthram Swami, Gustavo De Veciana, Haris Vikalo
llmsbenchmarks
More like this →
arXiv Machine Learning
Oct 5

Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations

This thesis introduces methods for assessing and enhancing the robustness of large language models (LLMs) against adversarial input variations. It proposes a generative robustness metric, R_stab(f), based on Jensen‑Shannon divergence, and proves bounds for localized attacks. The work also presents adaptive attacks, Trojan detection techniques, defense strategies for multi‑layer systems, and lightweight attestation for Model Context Protocol (MCP) agentic systems, all implemented in the JudgeGuard, TrojanArmor, and MCPSec suites.

By Narek Maloyan
llmsagentsroboticsmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 5

Refinement Buys Intelligibility, Search Buys Identity: What Test-Time Compute Buys in Masked-Diffusion TTS

The study investigates how two computational dimensions—model depth and refinement steps—affect intelligibility and speaker identity in masked-diffusion text‑to‑speech systems. Experiments with 15 models (19–133 M parameters) and up to 16 refinement steps show that refinement improves intelligibility more than identity, with a 1.86× asymmetry that persists even after retraining. Best‑of‑K search can recover identity when refinement fails, and analysis indicates that depth and steps target distinct bottlenecks, requiring separate optimization.

By Nityanand Mathur, Hamees Sayed, Ayush Pratap Singh
llmsdiffusionmultimodal
More like this →
arXiv AI
Oct 5

Preserving Mathematical Reasoning in Compressed Diffusion Language Models via Trajectory-Aware Low-Rank Approximation

The paper tackles the challenge of compressing diffusion language models (dLLMs) by addressing the mismatch between calibration on clean activations and inference on partially masked states. It introduces a trajectory‑aware low‑rank objective that considers corruption levels and masking realizations, and proposes Traj‑MC, a Monte Carlo method to efficiently estimate the trajectory second moment. Experiments show that trajectory‑aware calibration yields better reconstruction along the generation trajectory and preserves significantly more mathematical reasoning compared to clean calibration.

By Tian Liang, Zishan Shao, Yiran Chen
llmsdiffusionbenchmarks
More like this →
arXiv AI
Oct 5

Jumping the Line: Exploiting Length Predictions in LLM Scheduling

The paper introduces JIL, an attack that manipulates length predictions used by large language model (LLM) schedulers to gain higher priority and reduce completion time. By optimizing an adversarial suffix, JIL causes lightweight output-length probes to underestimate request lengths, leading to up to an 83.4 % reduction in predicted length and an average 1.53× faster completion in end‑to‑end serving experiments. The study also evaluates defenses, showing that grouping predictions into coarse intervals can mitigate JIL’s advantage and protect benign requests.

By Yuyang Dai, Rana Shahout, Mahmood Sharif
llmssafety
More like this →
arXiv AI
Oct 5

From Benchmarks to Production: A Text-to-SQL System for Complex Financial Data

The paper introduces FLINT, a domain‑specialized Text‑to‑SQL system designed for complex financial databases where data is stored as opaque integer keys. FLINT employs a lookup agent to resolve natural‑language concepts, uses embedding‑based retrieval of query templates from a compact expert‑authored bank, and prunes large schemas by traversing foreign‑key chains. Evaluated on 359 questions across two production financial schemas, FLINT outperforms state‑of‑the‑art baselines and is already deployed in a financial data retrieval service.

By Arijit Sehanobish, Bruno Gomes Coelho, Guillaume Michel, Sophia Zhi, Valerie Faucon-Morin, Kristen Howell
llmsragagentsbenchmarksdata-engineering
More like this →
arXiv Machine Learning
Oct 5

LS-AR: Future-Predictive Latent Steering in Autoregressive LLMs

The paper introduces LS-AR, a dual‑channel architecture that separates continuous goal steering from discrete token decoding using FiLM conditioning. It employs a Static Goal Encoder for long‑term macro‑objective retention and a Dynamic State Tracker for recurrent latent updates. Experiments show LS‑AR achieves perfect target recall on long‑horizon retrieval tasks, improves throughput by ~35%, reduces VRAM usage by 52.8%, and maintains higher completion rates in Blocksworld planning under perturbations.

By Anubha Gupta, Eduardo Pignatelli
llmsefficiency
More like this →
arXiv AI
Oct 5

HazardWeaver: Scientific Route Selection for Hazard Analysis Agents

HazardWeaver is an AI agent designed to select and execute appropriate scientific methods for hazard analysis by leveraging a Hazard Knowledge Compiler and a Hazard Capability Graph. It dynamically revises its route choices as new evidence and execution results emerge, ensuring that selected methods are both applicable and executable with available data and tools. The system is evaluated on the Hazard Weaver Benchmark, which includes 141 instances across multiple hazard domains, and demonstrates superior performance over existing agent systems, especially when multiple scientific routes are viable.

By Wangshu Zhu, Xueqi Cheng, Liang Wu, Yushun Dong
llmsagentsbenchmarks
More like this →
arXiv Machine Learning
Oct 5

Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

Page-EntroKV is a new KV‑cache eviction framework designed for grouped‑query attention (GQA) in long‑context autoregressive language models. It pools token importance scores across query heads within each physical group using sink‑isolated Renyi‑2 entropy, then projects these pooled scores onto page frames and performs eviction at the hardware tuple level. The authors prove theoretical guarantees on union overhead, budget preservation, and needle‑retention, and demonstrate on a Qwen2.5‑1.5B‑Instruct pilot that Page‑EntroKV achieves exact union overhead control, eliminates sink masquerade, and retains 100% of critical tokens compared to mean‑pooling baselines.

By Inbasekaran S
llms
More like this →