MoI-Bench is a new benchmark designed for on‑device image generation and editing, featuring 64 subtasks and 1150 samples that cover 16 operations and eight visual target categories. It introduces MoI‑Eval, a task‑specific evaluation framework that assesses instruction following, visual quality, and detail preservation, and aggregates results into a two‑dimensional Quality‑Deployability map. The benchmark reveals that current on‑device models excel at efficient generation or editing but need improved editing quality and broader task coverage, especially for text replacement and identity preservation.
By Kailai Feng, Haoxiang Rao, Yuxiang Wei, Bo Chen, Junsong Li, Yang Pan, Chengkun Cai, Jagmohan Chauhan
The paper introduces HiMA-Ecom, a hierarchical multi‑agent e‑commerce benchmark with 22.8K instances, where 17.7K include memory entries. It explores training, evaluation, and management of accumulated context in long‑horizon agentic systems, and proposes HiMA‑R1, a policy optimization method that controls trajectory context budgets and uses reward‑driven memory to accelerate convergence. Experiments show that agents built on 3B and 7B open‑source models outperform DeepSeek‑V3 by an average of 6.0 points, while reward‑driven memory cuts convergence steps by about 55%.
By Junxing Hu, Ai Han, Haolan Zhan, Pu Wei, Zhiqian Zhang, Yuhang Guo, Jiawei Lu, Zhen Chen, Haoran Li, Zicheng Zhang, Ling Li
ScalableRAG introduces a persistent typed‑set workspace that lets an agent construct named document and value sets at query time and combine them using set algebra, typed extraction, aggregation, and diagnostics. The zero‑ingestion variant requires no preprocessing or vector index yet matches leading baselines on standard QA benchmarks and outperforms them on the Hotels‑50 aggregative task. A lightweight‑index version achieves over 80 judge score on 5 M documents with low latency and minimal ingestion time, surpassing other systems in both accuracy and efficiency.
By Hilaf Hasson, Aditya Chakravarty, Jayant Thomas, Krishna Gogineni
The paper introduces SSR, a simple yet effective data selection strategy for million‑scale multi‑modal instruction‑tuning. SSR uses interaction styles as a proxy for structural diversity and decomposes data quality into capability‑specific scores, forming a rich profile for each sample. By casting selection as submodular coverage maximization, SSR consistently outperforms random sampling across two architecture stacks, multiple data pools, and 14 benchmarks, achieving 99.1% of full‑data performance with only 30% of a 2.6M pool.
By Mengyao Lyu, Yan Li, Huasong Zhong, Hui Chen, Jungong Han, Guiguang Ding, Zhenheng Yang
The paper examines whether modern drift‑adaptive malware classifiers provide reliable and stable confidence estimates when faced with distribution shifts. It introduces Aurora, a framework that evaluates confidence quality and operational resilience, and presents metrics that extend beyond point‑in‑time performance to assess stability over time. The study finds that state‑of‑the‑art models exhibit fragility across datasets with varying drift severity, indicating a need to revisit underlying assumptions.
By Alexander Herzog, Aliai Eusebi, Lorenzo Cavallaro
The paper introduces a doubly robust preference optimization algorithm for reinforcement learning from human feedback (RLHF) in large language models. It addresses sensitivity to misspecifications in the preference model, reference policy, or reward function by ensuring consistency when either the preference model or the reference policy is correctly specified. Experiments show that the method outperforms state‑of‑the‑art algorithms both theoretically and empirically.
By Erhan Xu, Kai Ye, Hongyi Zhou, Luhan Zhu, Francesco Quinzan, Chengchun Shi
The paper introduces SELF, an intrinsic weight‑based fingerprinting method for large language models that uses singular value and eigenvalue decomposition of attention weights to generate unique, transformation‑invariant fingerprints. SELF claims to eliminate the need for input data, resist false claim attacks, and reduce false positives compared to existing techniques. Experiments show that SELF achieves high IP infringement detection accuracy and remains robust against quantization, pruning, and fine‑tuning modifications.
By Hanxiu Zhang, Qiang Xu, Yue Zheng
SQuID is a benchmark dataset comprising 2,000 quantitative spatial reasoning questions over 587 satellite images, covering 24 question types across three difficulty tiers. Answers are derived from curated segmentation masks or human consensus, with numeric responses scored against tolerance ranges based on human annotations. The benchmark also provides 75,687 training question-answer pairs and reference masks, enabling modular evaluation and highlighting segmentation as a key error source for current vision‑language models.
By Peter A. Massih, Eric Cosatto, Martin Renqiang Min
SWE-Skills-Bench is a benchmark that evaluates the impact of individual agent skills on real-world software engineering tasks. It uses 98 public skills paired with specific GitHub repositories, generating 3,920 paired evaluations across four agent-model configurations. The study finds that most skills are ineffective, with only a few providing modest gains while some are harmful, raising the overall pass ratio from 76.6% to 76.9%.
By Tingxu Han, Yi Zhang, Wei Song, Chunrong Fang, Zhenyu Chen, Youcheng Sun, Lijie Hu
The study introduces a computationally efficient multi‑section whole slide image analysis framework that captures the entire prostate tumor landscape to predict biochemical recurrence (BCR) after radical prostatectomy. Using a large dataset of 23,451 slides from 789 patients, the authors optimized patch and slide subsampling to reduce computational demands while achieving strong predictive performance for 1‑ and 2‑year BCR, comparable to or better than existing clinical benchmarks. The AI‑derived risk score was an independent prognostic factor, improved prognostic performance when combined with clinical variables, and showed preliminary transferability in an external cohort, with attention‑dispersion analysis confirming that the model leveraged information from multiple sections consistent with prostate cancer’s multifocal nature.
By Yesung Cho, Dongmyung Shin, Sujeong Hong, Jooyeon Lee, Seongmin Park, Geongyu Lee, Won Young Park, Bongjin Park, Jongbae Park, Hong Koo Ha
The paper introduces the Neural Function Minimizer (NFM), a one-shot method that iteratively moves a position across a one‑dimensional domain to locate the global minimum from twenty noisy samples. In a controlled benchmark, the NFM matches Set Transformers on location error but achieves lower regret, especially with mixture heads, and outperforms classical estimators on unseen functions. The authors plan to release the benchmark, models, and harness for further research.
By Qusay Muzaffar, David Levin, Michael Werman
The paper investigates how different scoring rules affect the ranking of large language models (LLMs) on the IndiaFinBench benchmark, which contains 406 expert‑annotated items covering regulatory interpretation, numerical reasoning, contradiction detection, and temporal reasoning. Using twelve LLMs evaluated under strict string‑matching, a cross‑model judge, and a composite regime, the authors find that model rankings vary dramatically across regimes, with no significant rank correlation (Spearman ρ = -0.273). Item‑level analysis reveals that a substantial portion of items are answered identically by all models, while a higher‑discrimination band isolates more challenging cases.
By Rajveer Singh Pall
RePO‑VLA is a recovery‑driven policy optimization framework for Vision‑Language‑Action models that separates success, recovery, and failure trajectories. It introduces Recovery‑Aware Initialization, a Progress‑Aware Semantic Value Function, and Value‑Conditioned Refinement to learn from both successful and failed rollouts, enabling the policy to prefer high‑progress actions without online failure detectors. The authors also present the Failure‑Recovery Benchmark (FRBench) and demonstrate that RePO‑VLA raises adversarial success rates from 20% to 75% on average in simulated and real‑world bimanual tasks.
By Weijia Liufu, Xiaoyu Guo, Ruiyi Chen, Jingzhi Liu, Kaidong Zhang, Xiwen Liang, Jianqi Lin, Dawei Sun, Yuze Wang, Rongtao Xu, Bingqian Lin, Bowen Yang, Tongtong Cao, Bowen Peng, Dongyu Zhang, Guangrun Wang, Min Wang, Liang Lin, Xiaodan Liang
BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.
By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low
The paper investigates whether generative deep learning models can nowcast aerosol particle size distributions (PSDs) from routine PM2.5 measurements and meteorological data. Using data from two Finnish stations—Hyytiälä (boreal forest) and Kumpula (urban)—the authors benchmark six generative models against deterministic and climatological baselines, finding that flow matching, especially its guided variant, achieves the lowest continuous ranked probability score (CRPS). Incorporating a physics-based constraint derived from a reduced aerosol general dynamic equation further improves performance, reducing CRPS by 5.5–11.5% at the training site and improving transferability between sites.
By Zhi-Song Liu, Jenni K\"oykk\"a, Wenqing Peng, Ammar Kheder, Petri Clusius, Michael Boy
H-Glow is a hazard‑guided framework that accelerates graph generation by decomposing Continuous‑Time Markov chain generators into hazards and conditional jump distributions. It plans graph edits and dynamically adjusts sampling time based on graph activity, allowing pretrained graph DFM models to produce higher‑quality graphs with fewer steps. The authors also propose a new graph DFM model that directly predicts hazards and conditional jumps, whose training objective cleanly splits into hazard matching and weighted jump prediction, and demonstrate its effectiveness on molecular and synthetic graph benchmarks.
By Sirine Ayadi, Stephan G\"unnemann
DIGEST is a new method for symbolic regression that combines a lightweight LLM with Genetic Programming. It uses a pretrained decoder block as a symbolic policy to seed GP populations, providing flexible generation of candidate expressions. The approach also feeds high‑quality GP discoveries back into the LLM to refine future generations, leading to improved recovery rates and faster search times on benchmark datasets.
By Kaifeng Jin, Ruizhong Qiu, Zihao Li, Xiao Lin, Dongqi Fu, Yinglong Xia, Hong Li, Hanghang Tong
The paper presents a robust approximate unlearning method for Generalized Learning Vector Quantization (GLVQ) classifiers, enabling the removal of specific data samples from trained models. It evaluates how the size and characteristics of the forget set affect unlearning, proposes an adapted technique to mitigate instabilities, and demonstrates that the method achieves near-zero deviation from a fully retrained model while being up to 10^5 times more efficient on benchmark tabular datasets. The approach is applicable to any GLVQ model without altering its training scheme.
By Sreejita Ghosh, Mohammad Mohammadi
MOFjson is a new representation for metal‑organic frameworks that separates structural data into a raw atomic layer with explicit bond shift vectors and a derived layer that exposes secondary building units and network topology. The authors benchmarked MOFjson against three baselines across four question categories, achieving superior performance (e.g., 100% vs. 18‑47% for nearest periodic copy detection). Ablation of shift labels reduced accuracy dramatically, demonstrating that models rely on the periodic graph rather than surface format. MOFjson provides language models with a structured, topological view of MOFs, enabling more accurate predictions and facilitating agentic tool workflows and constrained MOF generation.
By Anabel Yong, Matthew Hart, Philippe Schwaller
TimeWorld introduces a plug‑in framework for action‑conditioned multimodal world models in time‑series forecasting. It builds on pretrained time‑series foundation models (TSFMs) and adds a Prototype‑Guided Latent Modulation module to embed textual actions into temporal dynamics, along with an Action Contrastive Learning component to sharpen action‑specific response modeling. The authors also present TSWM‑Bench, a cross‑domain benchmark of seven real‑world datasets with textual interventions, showing that simple text‑prefix adaptations outperform conventional multimodal forecasters and that pretrained TSFMs already encode useful world knowledge for action‑conditioned predictions.
By Hang Ni, Weijia Zhang, Hao Liu