arXiv AI September 25, 2026 By Xinchen Xiao

UO-FIE: Combining Exact-Label Supervision with Graded Utility for Factivity Inference

UO-FIE is a parameter‑efficient system for the Factivity Inference Evaluation 2026 (FIE2026) that predicts a distribution over nine ordered factivity intervals. It combines hard‑label supervision, utility‑based soft targets, scheduled class weights, and an ordinal loss to balance exact predictions with proximity to the correct interval. Using Qwen3.5‑9B with LoRA, UO‑FIE achieved the highest macro utility (0.8316) in the fine‑tuning track, while a prompt‑based ensemble ranked third in the non‑fine‑tuning track (macro utility 0.8450).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Hugging Face Trending Papers

Oct 29

ScienceJudge: Pattern Perturbation Augmentation for Sentence-Level Scientific Claim Verification

ScienceJudge introduces Pattern Perturbation Augmentation (PPA) to generate hard negative examples for sentence‑level scientific claim verification by modeling reporting errors as faithfulness‑breaking transformations. PPA applies these patterns to supported claims, creating controlled hard negatives and expanding coverage through pattern‑level, operation‑level, and cross‑domain augmentation. The resulting ScienceJudge system, which ensembles PPA‑trained vision‑language predictors with instruction‑guided LLM predictors, achieves state‑of‑the‑art performance, improving the average score from 0.8124 to 0.8692 and ranking first on the NLPCC 2026 Task 10 Track 1 leaderboard.

llms multimodal benchmarks

arXiv Machine Learning

Sep 22

Beyond Final-Token Classification: Heterogeneous Readouts for Evidence-Grounded Suicide Risk Detection

arXiv:2609.22767v1 Announce Type: cross Abstract: The IEEE BigData Cup benchmark combines three prediction problems with different output structures: ordinal suicide-risk classification, multi-label...

By Zirui Li, Yanling Li, Kaolanglang Gao

fine-tuning benchmarks

arXiv Machine Learning

Oct 8

Align Before You Combine: Reference Space Calibration for Supervision Without Ground Truth

The paper proposes a calibration-first framework that generates supervision scores without ground-truth labels or a shared annotation space. It aligns subset-specific scorers by mapping them onto a synthetic ordinal reference space built from ordered calibration features, enabling the fusion of otherwise incomparable outputs. Experiments on three benchmark datasets show that this method consistently outperforms uncalibrated averaging and matches or exceeds the best individual scorer, with significant improvements after Bonferroni correction.

By Jackson Eshbaugh, Jorge Silveyra

benchmarks

arXiv Computation and Language

Sep 1

Small Language Models as Judges for Rubric-Based Reinforcement Learning

arXiv:2608.30005v1 Announce Type: new Abstract: Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific c...

By Fengyu Xie, Yilun Zhao, Bingsen Chen, Arman Cohan, Chen Zhao

llms reinforcement-learning

arXiv AI

Jun 3

CoEval: Ranking Language Models for Custom Tasks Without Labeled Data or Trustworthy Benchmarks

arXiv:2606. 03650v1 Announce Type: cross Abstract: Choosing or ranking language models for a specific application is hardest when no task-specific labeled data exists, and standard public benchmarks cannot be trusted, their items having likely leaked into pretraining, so scores reflect memorization rather than fitness.

By Alexander Apartsin, Yehudit Aperstein

llms benchmarks safety

arXiv AI

Jul 1

QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents

arXiv:2606. 32034v1 Announce Type: cross Abstract: LLM agents increasingly act over long horizons, where a single trajectory can contain hundreds or thousands of actions.

By Sergio Hern\'andez-Guti\'errez, Matteo Merler, Ilze Amanda Auzina, Joschka Str\"uber, Ameya Prabhu, Matthias Bethge

llms rag agents efficiency benchmarks