Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv Computer Vision
5d ago

FlexReader: Towards Adaptive Decoding Paths for Efficient Arbitrary-Length Scene Text Recognition

FlexReader is a scene text recognition model that supports multiple decoding strides, allowing a single model to balance accuracy and efficiency across various datasets. It introduces a Sub-string Perception module to extract target substrings from visual features and uses parallel decoding blocks to independently decode each character. During inference, an Adaptive Prefix Decoding strategy selects the optimal stride at each step based on confidence, enabling robust recognition of long, curved, or occluded text while speeding up decoding up to 6× compared to autoregressive methods.

By Zhibin Ma, Pengwen Dai, Zhigang Yao, Xugong Qin, Wenqi Ren, Xiaochun Cao
arXiv Machine Learning
5d ago

Leakage-Controlled Structure-Preserving Augmentation and Template-Aware Evaluation for Bangla Symptom-Based Disease Prediction

The study evaluates a leakage‑controlled, structure‑preserving augmentation method—train‑only asymmetric Bernoulli perturbation—on Bangla symptom‑based disease prediction. Using 725 unique profiles across 85 diseases and 172 binary symptoms, the authors compare this augmentation against duplication, oversampling, class weighting, and SMOTEN, finding that boosted‑tree models (XGBoost, LightGBM, CatBoost) gain substantial accuracy improvements while Bernoulli Naive Bayes shows limited benefit after statistical correction. A Hamming‑distance audit reveals that most held‑out profiles are very close to training data, and robustness tests confirm that XGBoost and LightGBM maintain gains even when one‑bit disease profiles are purged.

By Sabah Nushra, Shovan Samanta Turzo, Moinul Hossain, Md Kishor Morol
arXiv Machine Learning
5d ago

Combinatorial Residual Network: a decay-chain-aware classifier without explicit jet assignment

arXiv:2610. 40749v1 Announce Type: cross Abstract: Final states with several jets from the decays of heavy particles pose a combinatorial problem, since the jets must be assigned, at least implicitly, to their parent particles.

By John Alison, Patrick Bryant, Alejandro Gomez Espinosa, Christian Guinto-Brody, Aniket Khanal, Chuyuan Liu, Sindhu Murthy, Guanglei Zhao
arXiv Machine Learning
5d ago

On-Chain Behavioral Signatures for Classifying Centralized Exchange Addresses on Ethereum Using Proof-of-Reserves

The paper presents a classification framework that uses Proof-of-Reserves disclosures to identify centralized exchange (CEX) addresses on Ethereum. It analyzes 3,033,857 addresses from 15 CEXs and five non-CEX categories, extracting 58 on-chain behavioral features and demonstrating that LightGBM outperforms logistic regression across various lookback windows. Key predictive signals include gas behavior, ERC‑20 token activity, and internal transactions, and the authors release a feature dataset for reproducible research.

By Seongchan Park, Hyunho Jeon, Byungsoo Kang, Gaeun Kim, Minseob So, Seongyong Kim
arXiv Machine Learning
5d ago

When Does Context Help? A Systematic Study of Target-Conditional Molecular Property Prediction

The paper systematically investigates when conditioning molecular property predictions on target context is beneficial. Across 10 protein families, 4 fusion architectures, and varying data sizes, the FiLM-based NestDrug model outperforms other conditioning methods, especially in data-scarce scenarios, but can also hurt performance due to distribution mismatch. The study also highlights flaws in standard benchmarks, such as high performance from simple nearest-neighbor methods and data leakage, and demonstrates that a temporal split yields stable generalization to future chemical space.

By Bryan Cheng, Jasper Zhang
arXiv Machine Learning
5d ago

Cyclical Entropy Eruption: Entropy Dynamics in Agent Reinforcement Learning

The paper introduces the phenomenon of cyclical entropy eruption in agent reinforcement learning, where entropy rises sharply and then subsides in recurring cycles, unlike the steady low entropy seen in single‑turn reasoning RL. It breaks this dynamic into three phases, offering theoretical and empirical insights into the mechanisms driving the oscillation. The authors propose SEAL, a lightweight auxiliary loss that separates correct and incorrect trajectories in representation space, which stabilizes training and improves downstream agent performance across various benchmarks and algorithms.

By Wendi Li, Shawn Im, Sharon Li
arXiv Machine Learning
5d ago

Critically-Damped Higher-Order Langevin Dynamics for Generative Modeling

The paper introduces Critically‑Damped Higher‑Order Langevin Dynamics (C‑HOLD) for generative modeling, extending the concept of critical damping to arbitrary‑order Langevin dynamics. It builds on the recent Higher‑Order Langevin Dynamics (HOLD) method by providing an optimal hyperparameter set for the n‑dimensional case and closed‑form expressions for the mean and covariance of the forward process. Experiments on CIFAR‑10 and CelebA‑HQ 256×256 datasets demonstrate the approach’s effectiveness using the FID metric.

By Benjamin Sterling, Chad Gueli, M\'onica F. Bugallo
arXiv Computation and Language
5d ago

Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses

The paper introduces RAGFlow, a retrieval‑augmented generation framework that transforms an undisciplined query loop into a structured pipeline of five stages. It emphasizes knowledge compilation—enriching documents with navigation hierarchies, claim‑evidence pairs, entity graphs, timelines, mind maps, ontologies, wikis, and memory—rather than improving retrieval accuracy. The compiled context reduces query tokens by 91% when evidence suffices, cuts latency from 140 s to 30 s, and provides governance and integration benefits not captured by standard accuracy benchmarks.

By Zhichang Yu, Baining Cao, Sbaaoui Idriss, Qi Wang, Gangqiang Xu, Ling Qin, Xiaoling Jin, An Liu, Shaohao Cai, Xiaoyang Bao, Shengle Chen, Changchang Ding, Pengfei Zhu, Jingxiang Xu, Yifeng Mao, Lu Liu, Hai Jin, Yingfeng Zhang
arXiv Computer Vision
5d ago

Coarse3D: Controllable and Faithful Sparse Structure Generation for Image-to-3D

Coarse3D is a new sparse‑structure generation model that produces orientation‑controllable and structurally faithful voxel geometry from arbitrary unposed images. It introduces a first‑frame anchored canonical space, a dual‑tower DiT with auxiliary reconstruction, and spatial‑prior guided attention to align generation with the input view and ground 2D‑3D correspondence. The model can replace the coarse stage in existing pipelines and, according to experiments, substantially improves sparse structure quality and overall 3D generation performance.

By Gaige Wang, Bowen Pan, Hao Ren, Jiangjing Lyu, Chengfei Lyu
arXiv Computation and Language
5d ago

Aligning MLLMs with the Latent Structure of Human Cognition via Behavior-Derived Semantic Dimensions

The paper introduces a framework that aligns multimodal large language models (MLLMs) with human cognitive structure by leveraging a behavior-derived semantic embedding space built from 4.70 million human Odd-One-Out judgments. It maps each triplet to the most salient decision dimension and generates dimension‑grounded linguistic rationales, enabling MLLMs to learn choices and justifications consistent with human criteria. Experiments show improved consistency with human similarity judgments, competitive benchmark performance, and increased representational alignment with human fMRI data.

By Ning E, Changde Du, Yizhuo Lu, Huiguang He
arXiv Machine Learning
5d ago

Chronos-2: A foundation model for multivariate time series forecasting

Chronos-2 is a foundation model for multivariate time‑series forecasting that captures interactions among multiple series and heterogeneous covariates, enabling zero‑shot probabilistic forecasts without task‑specific tuning. It achieves state‑of‑the‑art accuracy, improving skill scores by 5.2 percentage points and win rates by 10 percentage points over leading baselines, while being computationally efficient—generating forecasts for 100 series in under one second on a single mid‑range GPU. The model demonstrates strong performance across diverse scientific domains such as infectious‑disease forecasting, macroeconomics, carbon‑flux prediction, and runoff simulation, matching or surpassing specialized methods.

By Abdul Fatir Ansari, Oleksandr Shchur, Jaris K\"uken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Florian Saupe, Markus Reichstein, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, Michael Bohlke-Schneider
arXiv Computer Vision
5d ago

DRScaffold: Boosting Dense-Scene Reasoning in Lightweight Vision Language Models

DRScaffold introduces a supervised fine‑tuning framework that decomposes dense‑scene reasoning into four causally ordered stages, enabling lightweight vision‑language models to ground each reasoning step in visual entities and relations. The authors first present DRBench, a benchmark of 14,573 questions over 2,943 images covering five task categories and three reasoning layers. Experiments show that applying DRScaffold to three lightweight VLMs yields significant gains on DRBench while maintaining or improving general‑purpose benchmark performance, with Qwen2.5‑VL‑3B surpassing the frozen Qwen2.5‑VL‑32B under zero‑shot prompting.

By Xinrui Shi, Kai Liu, Ziqing Zhang, Jianze Li, Anqi Li, Yulun Zhang
arXiv Machine Learning
5d ago

Benchmarking Forecasting Agents on Yesterday's Tomorrows

The paper introduces FABench, a scalable backtesting environment designed to evaluate forecasting agents. FABench automatically discovers events, generates questions, and resolves outcomes for any past forecast date, allowing continuous renewal beyond model training cutoffs. In initial tests, retrieval improves Brier scores across eight models, with top agents matching or surpassing market consensus on 54 Polymarket-linked questions, yet analysis shows persistent evidence‑weighting failures that highlight room for improvement.

By Yuxuan Wang, Yishu Wang, Jiaqi Deng
arXiv AI
5d ago

CIRCE Divines LLM Task Complexity from Constraint Structure

CIRCE is a method that estimates the difficulty of a task for a large language model (LLM) by extracting the problem’s constraint structure into a Constraint Intermediate Representation, which is then compiled into a constraint graph. Unlike prior approaches that rely on the problem text or model outputs, CIRCE uses a lightweight LLM to build this representation and feeds it to a graph neural network to predict whether the LLM will solve the problem correctly. Experiments on seven reasoning benchmarks and four LLMs show that even a randomly initialized GNN improves prediction accuracy, lowering negative log‑likelihood from 0.465 to 0.423 and raising AUROC from 0.691 to 0.727 compared to the best baseline.

By Chenhao Hong, Jiakai Xu, Kaushiki Singh, Yusen Zhang, Eugene Wu
arXiv Computation and Language
5d ago

TRIDENT and SHAKTI: A Framework and Benchmark for Hindi AI-Text Detection

The paper introduces SHAKTI, a large-scale Hindi benchmark for detecting AI-generated text, covering multiple genres, LLM families, prompting strategies, and adversarial transformations. It evaluates the benchmark with human annotation and examines linguistic, grammatical, and sentiment differences between human and AI texts. Building on these insights, the authors propose TRIDENT, a round‑trip translation‑based detection framework with three variants that outperforms existing baselines and demonstrates robustness across seven languages and adversarial settings.

By Shifali Agrahari, Sujit Kumar, Sanasam Ranbir Singh
arXiv AI
5d ago

Experts Exhibit Structured Redundancy: Structure-Preserving Expert Pruning for Mixture-of-Experts Large Language Models

The paper introduces a structure‑preserving approach to prune experts in Mixture‑of‑Experts large language models, addressing memory overhead while maintaining performance. It observes that expert outputs exhibit structured redundancy, with stable rank and entropy effective rank far below the nominal number of experts. Two pruning methods—Projection Energy Expert Pruning (PEEP) and Distributional Coverage Expert Pruning (DCEP)—use router‑derived weights to preserve global structural relationships and local importance, achieving superior results on DeepSeek‑V2‑Lite and Qwen3‑30B‑A3B at 75% and 50% expert retention.

By Tian Wang, Yongding Tao, Yihong Dong, Xiaoming Wu, Ying Guo, Ge Li
arXiv Machine Learning
5d ago

Operon-S1: a fast decision controller for long-horizon biomedical agents

Operon‑S1 is a fast decision controller that separates a language‑model agent’s judgment from its execution, posing 13 closed questions at key decision points and using a lightweight decision model to answer them. In seven life‑science benchmarks, the local decision model handled judgments in 0.6 s on median with no measurable impact on agent performance, while the main language model’s involvement accounted for 8–35 % of task cost. The system records every decision and its outcome, allowing retraining to improve accuracy, and is already applied to wet‑lab protocol troubleshooting and germline panel design.

By Genoria AI Team, Yuejie Hou, Yueyuxiao Yang, Weiting Tang, Meng Yang
arXiv Computation and Language
5d ago

The Knowledge Layer Model (KLM): Evidence-Visible AI Output, Routing Diagnostics, and Negative Results

The paper introduces the Knowledge Layer Model (KLM), an open, implementation‑neutral specification for recording inference details, including versioned epistemic assessments, evidence relations, and explicit unknowns. A reference implementation, SEDIM, is provided, and the authors report internal experiments: a routing benchmark achieved 91.3% accuracy across 15 classes, a native‑model weight‑path test failed all 15 cases, and diagnostic metrics such as AUC were below acceptable thresholds. The study notes that evidence visibility alone does not guarantee correctness or compliance, and calls for independent replication and a full public audit package.

By Omer Asim Ozturk, Cihan Aydiner