ScienceJudge introduces Pattern Perturbation Augmentation (PPA) to generate hard negative examples for sentence‑level scientific claim verification by modeling reporting errors as faithfulness‑breaking transformations. PPA applies these patterns to supported claims, creating controlled hard negatives and expanding coverage through pattern‑level, operation‑level, and cross‑domain augmentation. The resulting ScienceJudge system, which ensembles PPA‑trained vision‑language predictors with instruction‑guided LLM predictors, achieves state‑of‑the‑art performance, improving the average score from 0.8124 to 0.8692 and ranking first on the NLPCC 2026 Task 10 Track 1 leaderboard.
arXiv:2609.22767v1 Announce Type: cross
Abstract: The IEEE BigData Cup benchmark combines three prediction problems with different output structures: ordinal suicide-risk classification, multi-label...
By Zirui Li, Yanling Li, Kaolanglang Gao
The paper proposes a calibration-first framework that generates supervision scores without ground-truth labels or a shared annotation space. It aligns subset-specific scorers by mapping them onto a synthetic ordinal reference space built from ordered calibration features, enabling the fusion of otherwise incomparable outputs. Experiments on three benchmark datasets show that this method consistently outperforms uncalibrated averaging and matches or exceeds the best individual scorer, with significant improvements after Bonferroni correction.
By Jackson Eshbaugh, Jorge Silveyra
arXiv:2608.30005v1 Announce Type: new
Abstract: Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific c...
By Fengyu Xie, Yilun Zhao, Bingsen Chen, Arman Cohan, Chen Zhao
arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.
By Alexander Apartsin, Yehudit Aperstein
arXiv:2606. 32034v1 Announce Type: cross Abstract: LLM agents increasingly act over long horizons, where a single trajectory can contain hundreds or thousands of actions.
By Sergio Hern\'andez-Guti\'errez, Matteo Merler, Ilze Amanda Auzina, Joschka Str\"uber, Ameya Prabhu, Matthias Bethge
The paper introduces Test-Time Calibration Learning (TTCL), a label‑free framework that adapts both reasoning accuracy and verbalized confidence of large language models directly on unlabeled target‑task data. TTCL generates self‑supervision signals from multiple model responses, enabling calibration without ground‑truth labels and proving theoretically as a bounded surrogate for the ideal calibration objective. Experiments on mathematical reasoning and factual question answering show consistent improvements, with base models gaining an average 40.13% in accuracy and 70.80% in ECE reduction across eight benchmarks, and further gains under domain shift.
By Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Shixiong Kai, Mingxuan Yuan, Bo Han
arXiv:2607. 16239v1 Announce Type: new Abstract: AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges, tasks, and domains.
By Lei Shi, Anlan Zhang, Rita Lyu, Zhengmian Hu, Tong Yu, David Arbour, Avi Feller, Saayan Mitra, Ritwik Sinha
arXiv:2608. 14649v1 Announce Type: new Abstract: We present dLLM-SetScore, a training-free method that uses discrete masked-diffusion language models for multi-label text classification.
By Pawan Kumar
The paper introduces Chinese Decision, a family of Chinese-language models designed for single-pass probabilistic decision-making. The models generate logits for candidate choices in one forward pass and normalize them without autoregressive generation, supporting Boolean, categorical, and ordinal decisions. A 0.8B model trained on 13.048 million decisions achieves 72.24% general-domain accuracy and 76.98% average accuracy across medicine, law, and finance, while a 2B model shows comparable performance after partial training.
By Qiang Yan (Institute of Computing Technology, Chinese Academy of Sciences)
arXiv:2607. 23575v1 Announce Type: cross Abstract: Ordinal regression is widely used in scenarios where labels are discrete yet inherently ordered.
By Chunlai Dong, Yaojun Hu, Yuyang Xu, Haochao Ying, Jian Wu
arXiv:2609.24303v1 Announce Type: new
Abstract: Post-training often improves task performance but can degrade confidence calibration, leaving post-trained language models (PoLMs) more overconfident t...
By Linhan Luo, Lequan Lin, Dai Shi, Feng Chen, Jos\'e Miguel Hern\'andez-Lobato, Junbin Gao