Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.
arXiv:2609.12397v1 Announce Type: new Abstract: Multi-modal image generation, particularly subject-driven customization, has garnered growing attention in recent years. Despite the rapid advancement...
The paper introduces MTD3-BC, a model‑free offline reinforcement learning algorithm that optimizes yaw control for wind farms amid changing wind directions. By learning from a pre‑collected dataset and incorporating an action consistency term, it reduces the need for extensive simulator interactions. Experimental wind‑tunnel tests show that MTD3‑BC improves farm‑level power output by about 10% compared to a greedy baseline and matches a model‑based benchmark, all while cutting training costs dramatically.
The paper investigates the environmental impact of running large language models (LLMs) on mobile devices. It evaluates 18 different LLM configurations on two smartphones and a server, measuring energy per token, latency, accuracy, and battery-cycle consumption. Findings reveal that on-device inference is about three times less energy‑efficient than batched server inference, that energy consumption varies non‑monotonically with quantization bit‑width, and that most models are not on the Pareto front of accuracy and energy efficiency. The study concludes that local AI is not inherently more sustainable than cloud inference, with the majority of environmental impact stemming from device embodied carbon.
The paper presents a curriculum‑based adversarial heterogeneous agent reinforcement learning (HARL‑AC) approach for autonomous quad‑copter landing on a ship deck in maritime settings. Using Heterogeneous‑Agent Proximal Policy Optimization (HAPPO) in NVIDIA Isaac Lab, the authors train a cooperative control policy that outperforms domain‑randomized baselines, achieving up to 97.5% success on in‑distribution sea states and higher median success and lower crash rates on out‑of‑distribution sea states. The adversarially trained policy also exhibits more cautious behavior, slightly increasing timeouts but improving safety in severe, unseen conditions.
The paper introduces a new criticality metric specifically designed for vulnerable road users (VRUs) and a scenario‑independent prediction framework that applies to all traffic participants. The VRU‑centric metric improves pedestrian criticality classification by up to 50 %, while the prediction framework surpasses state‑of‑the‑art metrics by 275 %, achieving an F1‑score of 0.96 on the DeepAccident dataset. These advances enable more accurate, scenario‑agnostic safety assessments for autonomous driving systems.
R2VC is a modular fact‑checking system that separates retrieval, reasoning, verification, and confidence calibration. It uses hybrid sparse‑plus‑dense Wikipedia retrieval, a fine‑tuned generator for structured verdicts, an NLI cross‑encoder for selecting evidence‑based candidates, and a lightweight calibrator for confidence and abstention. On the FEVER benchmark, R2VC improves accuracy by 13.74% over a baseline and shows that verifier‑based candidate selection and calibration are key contributors to performance.
GAUGE is a new offline protocol that evaluates whether the common practice of using an LLM-as-a-judge to rank task‑oriented agents actually aligns with a verifiable reward. Across 25 agents from six providers on two benchmarks, GAUGE finds that user satisfaction scores are largely uncorrelated with task success, and that the judge’s ranking loses precision when agents are closely matched in performance. The study highlights a gap between ranking validity and construct validity in current evaluation practices.
BRIDGE-EEG is an efficient multi‑task EEG classification pipeline that leverages self‑supervised pretraining while dramatically reducing model size. It maps heterogeneous EEG recordings to a unified 62‑channel time‑frequency representation, pretrains an SE‑ResNet18 teacher with SimCLR, and distills it into smaller SE‑ResNet8 and SE‑ResNet4 students. The compact models achieve accuracy comparable to or better than larger foundation models on abnormality detection and emotion recognition, and they consume up to three times less energy on edge devices, enabling deployment on wearable hardware.
The paper investigates whether small models distilled from larger ones behave similarly when using byte versus token tokenization. It introduces two methods—Marginalize‑It (approximate) and End‑Of‑Token (exact)—to convert token logits to byte logits, and conducts a large‑scale study on decoder‑only dense transformers ranging from 1 billion to 1 trillion bytes of data. Results show that while token‑based models excel early, byte‑based models eventually surpass them with more compute, achieving higher performance ceilings, greater data efficiency, and lower logit storage costs.
BEAST is the first Bayesian Swin Transformer for atmospheric forecasting at 0.25° global resolution, capable of quantifying both aleatoric and epistemic uncertainty. The authors introduce a 4D‑parallelization scheme with domain‑tensor‑parallelism and a novel uncertainty parallel method, allowing a 2.4‑billion‑parameter model to reach 3.96 EFLOP/s on 20,480 NVIDIA GH200 GPUs. A 700‑million‑parameter BEAST trained on 40 years of data achieves predictive skill comparable to state‑of‑the‑art probabilistic AI models, predicts extreme events with high accuracy, and generates large ensembles 3–4× faster than the current best AI model.
The paper introduces PA-CDM, a new position‑aware character detection matching metric for handwritten mathematical expression recognition that improves on existing render‑based and tree‑edit metrics by incorporating position‑forest encoding and divergence‑level weighting. It also presents StructPerturb v2.0, a benchmark of 1,340 controlled perturbation pairs, and a cross‑metric consistency protocol that includes a sensitivity matrix, a human study, and LLM‑judge calibration. In a six‑annotator study, PA‑CDM achieves the highest correlation with human judgments (Spearman rho = 0.9535) among seven automatic metrics, approaching the performance of a costly LLM judge while remaining deterministic and cost‑free.
The paper introduces Spatial‑FAD, a few‑shot medical anomaly detection framework that fuses Vision‑Language Model (CLIP) semantics with spatial priors from Vision Foundation Models (DINO). A VFM‑enhanced adapter injects structural affinity into CLIP features, while a sliding‑window aggregation produces high‑resolution embeddings for finer lesion localization. Prototype‑enhanced support memory further improves efficiency and performance, yielding significant gains on Liver CT, Retinal OCT, and Brain MRI datasets, notably an 11.4% Dice improvement in 4‑shot scenarios.
The paper introduces an online sketched Newton method that uses a generalized accelerated sketch-and-project solver (GAS) to approximate Newton directions efficiently. GAS incorporates Nesterov momentum and a flexible projection metric, achieving accelerated convergence and reduced computational cost. The authors prove asymptotic normality and a functional central limit theorem for the averaged iterates, enabling an online inference procedure via random scaling that yields a pivotal test statistic with a parameter‑free limiting distribution.
ExpertHTR is a unified vision‑language framework for handwritten text recognition that tackles the challenge of small, heterogeneous datasets by organizing structural annotations into a common Page‑Region‑Line representation. It defines four related training tasks—complete transcription, physical‑line coverage, text localization, and localized recognition—without extra manual labels. The model combines a jointly trained dense backbone with a sparse Mixture‑of‑Experts architecture, using Sparsegen routing and regularization to adaptively activate experts, achieving state‑of‑the‑art results on the IAM benchmark and outperforming general‑purpose OCR systems on most datasets.
The paper introduces Fundamental Dynamical Units (FDUs), signed three‑node interaction patterns that reduce the combinatorial complexity of interaction architectures in networked dynamical systems. By embedding FDU‑regularized structural inference into a physics‑informed neural ODE, the authors jointly recover interaction structure and perturbation‑resolved trajectories, demonstrating the approach on synthetic benchmarks. This framework enables motif‑prescribed intervention design and mechanistically interpretable inference in complex systems.
The paper introduces a pre‑action verification framework for large language model agents that emit shell commands or code edits. By running deterministic checks before execution, the system can catch invalid commands (95.8% success with a 10.0% false‑positive rate) and prevent silent failures in code edits, achieving high recall while minimizing false positives. The authors provide benchmarks, verifiers, and guards for both action modalities.
The paper introduces Decision-Flow Sampling (DF‑Sample), a training‑free, data‑free inference framework that builds a hierarchical reasoning tree, evaluates entire trajectories, and back‑propagates utilities to guide branching decisions. Unlike local step‑wise sampling, DF‑Sample explicitly assesses global paths, enabling it to recover high‑quality, low‑probability reasoning chains that standard decoding misses. On the GPQA benchmark, DF‑Sample attains 45.6% accuracy, outperforming power sampling (38.9%) and GRPO (39.9%) and consistently surpassing baselines across multiple models and benchmarks, demonstrating significant latent reasoning potential in pretrained LLMs.
ParaRecover is a new process-level benchmark designed to evaluate error localization and recovery in multi-turn parallel tool-use agents. It contains 10,626 instances across two difficulty levels, built on a taxonomy of 14 error types that cover planning dependencies, tool selection, and argument matching. The benchmark introduces the SDE rubric, which assesses structural integrity, diagnostic reasoning, and evolutionary strategy during agent execution, and demonstrates that it can guide improvements in agents’ reflective recovery capabilities.
SNAP‑KG is a framework that assigns new entities to semantic communities in a growing knowledge graph using only raw features, without graph access or retraining at inference time. The authors evaluate it on five multi‑view benchmarks and a large OGB‑WikiKG2 graph, all of which contain at least one homophilous view, and find strong performance. Extending the evaluation to three heterophilous graphs shows that when no homophilous view exists, clustering quality drops sharply for both SNAP‑KG and transductive baselines; the key factor is the homophily of the relation rather than the number of relations, and multi‑view fusion only helps if at least one homophilous relation is present. "whyItMatters":"The study reveals that SNAP‑KG’s effectiveness relies on the homophily assumption, highlighting a limitation for heterophilous knowledge graphs and suggesting a direction for future research on heterophily‑aware models."
The paper introduces a certified AI triage system for ICU alarms, reframing alarm reduction as a three-way decision (retain, suppress, or defer). It demonstrates that, with a 5% budget, the system can suppress 74.8% of false ventricular‑tachycardia alarms while only silencing 1.5% of genuine ones, achieving an AUROC of 0.953 and a Challenge Score of 83.33—comparable to the best existing methods. The study also explores how grid granularity and calibration affect certification, showing that finer grids can certify fewer alarms but with tighter guarantees.