Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
Vision-Language-Action (VLA) models generalize across tasks, but their autoregressive decoders need sequential forward passes in proportion to the action horizon, each bounded below on embedded aerial...
A team serving a 9B-20B open-weight language model on vLLM or SGLang must choose a quantization recipe, and needs two numbers: the speedup over BF16 and the accuracy given up. We assemble every public...
Parameter-efficient linearization turns a pretrained autoregressive Transformer into an efficient model by hybridizing sliding-window attention (SWA) with a constant-size recurrent memory. But does ev...
The paper introduces ANGLER, an adaptive framework that uses state-based verifiers from the REEF benchmark to guide the evolution of web‑agent attacks. REEF provides 9 verifier‑grounded websites and 5 attack types, enabling realistic red‑teaming of VLM‑based agents. ANGLER improves attack success rates to 20.36% versus 15.24% for non‑adaptive methods and lowers benign task success from 61.9% to 44.4%.
Ax-Prover is a multi‑agent framework that uses Large Language Models (LLMs) combined with Lean tools via the Model Context Protocol (MCP) to generate formally correct proofs in mathematics and quantum physics. It can operate autonomously or collaboratively with human experts, and its performance on public math benchmarks and new Lean benchmarks in abstract algebra and quantum theory shows it is competitive with state‑of‑the‑art provers and often outperforms them. The system also demonstrated practical assistant capabilities by helping a mathematician formalize a complex cryptography theorem.
Speech-DRAME is a framework for evaluating spoken role‑play that considers linguistic content, prosody, emotional expression, character embodiment, and contextual fit. It includes a bilingual, human‑annotated benchmark (Speech‑DRAME‑EvalBench) with contrastive mismatch conditions and a fine‑tuned audio‑language evaluator (DRAME‑Eval). While DRAME‑Eval performs well on synthesized data, it shows a significant performance drop on authentic human acting, highlighting a synthetic‑to‑authentic transfer gap.
The paper introduces an open, reproducible biomedical knowledge graph ecosystem built on the Rust-native Samyama graph database. It details three foundational knowledge graphs—Pathways, Clinical Trials, and Drug Interactions—along with their expansion into a 22‑graph, 2‑cohort biomedical store containing over 328 million nodes and 1.44 billion edges, including synthetic patient data. The authors describe a reproducible ETL workflow, cross‑graph integration via query‑time joins and identifier deduplication, and a schema‑driven Model Context Protocol that achieves high accuracy on the BiomedQA benchmark.
The paper introduces $R^3$ (Reject, Remember, Retry), a black-box multi-LLM framework that improves multi-turn jailbreak attacks by conditioning on a compact state instead of the full conversation transcript. In $R^3$, failed attack turns are removed from the target’s context but stored by the attacker along with the reason for failure, creating an asymmetric memory. Experiments on two benchmarks and six target LLMs show that $R^3$ outperforms state‑of‑the‑art baselines by up to 53% in ASR@5 under matched query budgets, and even a low‑capacity attacker using $R^3$’s state can surpass high‑capacity attackers without it by up to 34% in ASR@5.
The paper introduces a specification‑aware action distribution optimization framework that enforces Signal Temporal Logic (STL) constraints during the execution of pretrained robotics foundation models without altering their parameters. At each decision step, the method minimally adjusts the action distribution to satisfy hard STL feasibility constraints by reasoning over the remaining horizon with forward dynamics propagation. The approach is validated in simulation using state‑of‑the‑art robotics foundation models across multiple environments and complex specifications.
Hierarchical World Models with Implicit Dynamics (ID‑WM) introduces a two‑level approach that separates state‑space reachability from local control. A high‑level model samples feasible transitions across the state manifold, while a low‑level world model acts as an inverse dynamics solver to translate these transitions into precise actions. This design enables planners to identify reachable subgoals without being constrained by low‑level execution details, producing novel long‑horizon plans that outperform state‑of‑the‑art policy and diffusion‑based planners by 38.5% and single‑level MPC by 135% in complex, unseen configurations.
The paper introduces ERASE, a multimodal defense against indirect prompt injection that erases malicious instructions directly within the input media. By replacing the injected region with modality‑appropriate blank content and using on‑policy self‑distillation, ERASE trains a student model to learn from a frozen base model’s erased view, providing token‑level feedback during its own rollouts. Experiments on image, audio, and video tasks show near‑zero attack success while preserving task performance close to clean‑input levels, with significant improvements over prior methods such as ARGUS.
SecPO introduces a principled adversarial training framework to defend large language models against prompt injection attacks. By approximating an ideal injection‑invariant reference model and employing a new SecPO loss, the authors improve security while preserving utility. Combined with an efficient adversarial training pipeline that synthesizes strong injected inputs, the resulting SecPO + AT model significantly reduces attack success rates compared to existing defenses and maintains performance across multiple benchmarks.
STRIATUM-CTF is a modular agentic framework that uses the Model Context Protocol to mediate between large language model reasoning and the execution of offensive‑security tools. By exposing tools as strictly typed, schema‑validated functions, the protocol rejects invalid tool calls and returns structured observations that the agent can use to self‑correct. In controlled experiments the system achieved 77.8–86.7% success across 15 CTF challenges, and in a live multi‑team CTF at Florida State University it placed first among more than 30 teams.
Steel scrap composition estimation is an important part of yield planning and fair supplier compensation. Measuring the composition of a multi-ton pile is an inherently laborious process, so the asses...
Aggregate scores for continual learning algorithms rest on a distribution over task orders that is rarely stated. We formulate algorithm comparison as an expectation over a declared order distribution...
Mainstream sequence-processing AI is temporally reactive: each update is driven by input and learned state transitions, rather than by an independently evolving internal temporal reference. Biological...
Why do language models fail on long inputs? The common answer is length. We show that, for retrieval, length is cheap and competition is not. On five open models (0.5B to 3B parameters, four families)...
EgoActionBench is a new benchmark for dense action captioning in egocentric videos, featuring 1,407 episodes and 41,132 fine‑grained action annotations that specify which hand (left, right, or both) performs each temporally localized atomic action. The benchmark introduces an evaluation protocol that assesses semantic fidelity, temporal overlap, action precision, and recall, thereby measuring what happens, when it happens, and which hand is involved. Twenty baseline models were evaluated, and two backbones (Qwen3‑VL‑4B and LLaVA‑OneVision‑2‑8B) were fine‑tuned and reinforced, achieving significant score improvements and outperforming all open‑source baselines.
The paper introduces D&A (Describe & Assess), a prompt‑based method that separates vision‑language model progress estimation into a textual description of scene dynamics and a rubric‑based assessment. By requiring the model to explicitly define absolute success before scoring, D&A reduces confirmation bias and provides fine‑grained, partial progress metrics that accurately penalize failures and interpolate successes. The approach generalizes across tasks, embodiments, and benchmarks, matching or surpassing post‑trained reward models while enabling efficient downstream reinforcement learning.
CoLa3D is a lightweight, query‑based transformer that performs composable 3D scene decomposition directly from a single whole‑scene latent, whether derived from an image or an existing mesh. It outputs a set of object meshes with a consistent spatial layout that can be reassembled into a coherent scene, avoiding fragile pose and scale estimation used by other methods. The model achieves state‑of‑the‑art performance on synthetic and real‑world datasets, improving global scene consistency and object quality while running twice as fast and using less than 1% of SAM 3D’s training data.