Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
arXiv:2609.33676v2 Announce Type: replace Abstract: LLM agents increasingly take consequential actions through interactions with users, policies, and external tools. Auditing these agents requires au...
A2Z GameSpec-Bench introduces a benchmark of 100 long‑form Game Design Documents (GDDs) to evaluate how faithfully coding agents can generate complete games from detailed specifications. The benchmark measures faithfulness by checking that the game satisfies the GDD requirements and preserves the relationships among them, using a dependency‑aware contract and a combination of source‑code inspection and agent‑generated test policies. Evaluations show that current agents struggle to meet interdependent requirements, but requirement‑specific feedback improves GDD fidelity by 10.9% after two revision rounds.
DyRAD introduces a novel radar novel‑view synthesis framework that models dynamic driving scenes by separating static background reflectors from motion‑tracked dynamic point reflectors, enabling the rendering of full range‑azimuth‑Doppler (RAD) tensors. The method derives reflector velocities from object tracks, projects them onto the line of sight, and uses a fixed analytic point‑spread function to avoid embedding sensor‑induced spread into the scene representation. This design allows accurate scene reconstruction and zero‑shot transfer to different radar configurations, achieving a 90.7% recovery of radar detections on the RADIal dataset compared to 26.9% for the best baseline.
The paper introduces BARRAC, a method that adapts an English aspect‑based sentiment analysis framework for Arabic dialect classification tasks. It replaces English consumer‑review attribute pools with Arabic linguistic markers for sentiment, sarcasm, and dialect identification, and swaps noisy self‑training for a two‑stage training process. Evaluated on five Arabic dialect datasets, BARRAC achieves a mean macro‑F1 of 63.93%, surpassing the best few‑label state‑of‑the‑art by 3% and outperforming GPT‑4o on four of the five tasks, while error analysis highlights remaining challenges.
The paper investigates how generative behavioral-cloning policies handle multimodal expert behavior, identifying bottlenecks in both latent-variable and action-space policy designs. For latent-variable policies, preserving demonstrated modes depends on action-conditioned latent representations, and excessive posterior-prior regularization can suppress this information. In action-space generative policies, multimodality is limited by the smoothness of the base-to-action transport, requiring either sharp transitions or off-support bridge regions to capture many well-separated modes. Experiments on synthetic navigation and a physical-robot manipulation task confirm these findings, while standard robotic simulation benchmarks show limited conditional multimodality, making deterministic regression competitive.
arXiv:2504.04711v2 Announce Type: replace Abstract: Current large language models (LLMs) are constrained by human-derived training data and limited by a single level of abstraction that impedes defin...
arXiv:2609.38201v1 Announce Type: new Abstract: Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. Th...
arXiv:2609.34330v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated impressive performance in multimodal understanding, but processing large numbers of visu...
arXiv:2609.39157v1 Announce Type: new Abstract: Video object removal presents a uniquely difficult editing challenge. Because a removal prompt specifies only what to erase rather than what to generat...
ViLegalExpert is a large-scale Vietnamese legal benchmark built from real citizen–lawyer consultations, comprising over 172,000 questions across 34 legal domains with professional answers and expert-verified evidence. It supports legal information retrieval, extractive QA, and abstractive QA. Experiments show that while pretrained language models perform well on QA, hybrid retrieval methods achieve the best evidence retrieval, highlighting significant challenges in grounding legal answers to authoritative sources.
The paper introduces the QuantCode Model, a specialized language model for generating executable algorithmic trading code. It combines continued pretraining on trading framework code with supervised fine‑tuning on validated request‑to‑code pairs, achieving significant gains on the QuantCode‑Bench benchmark and improving both single‑turn and multi‑turn agentic success rates. The study also highlights the importance of retaining parsing and tool‑calling capabilities during domain specialization.
MetaSteer is a new method for steering large language models that learns nonlinear, context-dependent interventions applied to attention projection matrices. Unlike traditional linear, context-independent techniques, MetaSteer adapts its effects based on the input, requiring no linear concept-geometry assumption. Trained once on a pooled preference corpus, it transfers zero‑shot to unseen concepts and out‑of‑distribution contexts, matching or surpassing strong task‑specific baselines on multiple benchmarks and model families.
The paper introduces AutoMIP, a reusable agent skill designed to manage long‑horizon autoresearch for mixed‑integer linear and nonlinear programming problems. AutoMIP combines idea pooling with an algorithm tree search to maintain a persistent pool of complementary candidate ideas and organize experiments into a structured tree, allowing the agent to preserve unexplored hypotheses, refine promising algorithms, and switch methodological directions based on historical states. Experiments on MILP and MINLP benchmark cohorts show that AutoMIP achieves the highest final success rates, discovering new best solutions for 31 of 60 MIPLib instances and 52 of 60 MINLPLib instances, and ablation studies confirm the complementary contributions of idea pooling and algorithm tree search.
LatentHarness unifies memory access and latent reasoning by treating them as sequential latent actions—THINK, RECALL, and EXIT—within a language model. It is trained via counterfactual policy distillation, which evaluates the impact of each action on the emitted token and learns when to recall evidence versus continue reasoning. On six long‑context reasoning benchmarks, a 1.4B‑parameter LatentHarness model outperforms the strongest baselines by 2.8% and 10.0% relative, while running 5.9× faster than the leading long‑context baseline.
BELIEFRAG is a closed‑loop controller that maintains an explicit evidence state—tracking sufficiency, reliability, conflict, uncertainty, evidence gaps, and acquisition cost—to guide adaptive retrieval‑augmented generation (RAG). By updating this state, it selects among retrieval, query rewriting, verification, answering, stopping, and abstention, achieving higher token‑F1 scores and lower token usage on six QA benchmarks compared to fixed iterative retrieval. The approach shows that corrective re‑retrieval and calibrated answerability are key to its performance gains.
MADBench is a benchmark that evaluates the security of Multi-Agent Debate (MAD) systems, which allow large language models to exchange and critique answers. The study categorizes attacks into a layered taxonomy aligned with the MAD workflow and tests six attack families across 356 source tasks and 3,958 test cases. Results indicate that while MAD can reduce answer accuracy attacks compared to single-agent baselines, it may amplify unauthorized reads or writes, and even with collusion among agents, the final answer changes from correct to wrong only 28.30% of the time.
The paper introduces EvolvingNav, a system that builds a time‑indexed belief about moving targets in dynamic environments by combining timestamped 3D object histories with a persistence‑relocation model. It uses an event‑driven filter to update beliefs over time, incorporates RGB‑D evidence, and applies a zero‑shot vision‑language controller for action selection. The authors also present EvoWorld‑Bench, a large benchmark of human‑activity‑based scenes, and demonstrate that EvolvingNav outperforms baselines in both simulation and real‑robot experiments, especially when temporal patterns are learnable.
PCB-MC is a new dataset for detecting missing components on printed circuit boards, featuring 197 distinct board types with footprint-level annotations derived from the RF100 dataset. The dataset includes multiple augmented samples per board type and introduces board type-aware cross‑validation splits to prevent layout leakage between training and test sets. Benchmark results show that supervised models suffer high false‑negative rates on unseen board designs, while unsupervised anomaly detection methods fail due to misalignment with board-specific references, highlighting the remaining challenges in missing component detection across diverse PCB layouts.
Soft Spatial Reasoning introduces a post‑training framework for Large Vision‑Language Models that replaces hard, token‑by‑token chain‑of‑thought reasoning with a soft, continuous state formed by mixing token embeddings at each intermediate step. The method employs AdaptSoft, a controller that adjusts the degree of softness based on hidden states and predictive uncertainty, guided by a gradient‑alignment learning objective that requires no intermediate supervision. Experiments on diverse spatial benchmarks show that this approach outperforms both hard and fixed‑soft chain‑of‑thought baselines and several existing LVLMs.
Switching Linear Attention (SwiLA) is a new sequence layer that improves upon standard softmax attention by maintaining a fixed-size recurrent state while enhancing representational capacity. It derives its recurrence from a test-time regression framework, using online expectation-maximization in a mixture of linear regressions model. In various benchmarks—including associative recall, in-context language learning, and language modeling—SwiLA achieves strong performance, narrowing the gap to softmax attention and even surpassing it in some settings.