arXiv Machine Learning

Multi-stage Dynamic Selection for Cross-Project Defect Prediction

arXiv:2607. 20151v1 Announce Type: cross Abstract: Cross-Project Defect Prediction (CPDP) involves building models using data from external projects, called training projects, to predict modules from the target project.

arXiv Machine Learning
Aug 19

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
arXiv AI
Jul 22

Automated Data Engineering and Feature Selection for the Case Study of Warpage Detection in Fused Deposition Modeling

arXiv:2607. 18515v1 Announce Type: cross Abstract: This study contributes toward development of an Automated Data Processing (ADP) framework designed to evaluate and reinforce optimal machine learning model-feature combinations for predictive tasks in fused deposition modeling (FDM) process datasets.

By Saleh Valizadeh Sotubadi, Nazanin Mahjourian, Vinh Nguyen
arXiv Machine Learning
4d ago

Simulating Classification Models for Ex-Ante Evaluation of Predict-Then-Optimize Methods

The paper extends ex‑ante evaluation of Predict‑Then‑Optimize methods from binary to multiclass classification by simulating predictions at specified performance levels and mapping prediction errors to decision regret. It introduces a first‑order approximation that estimates regret from individual misclassifications, reducing computational effort. Experiments show the simulation accurately reproduces target performance and that the approximation is close for some problems, though it falters when simultaneous misclassifications interact significantly.

By Pieter Smet
arXiv Machine Learning
Jul 31

What Is The Performance Ceiling of My Classifier? Utilizing Category-Wise Influence Functions for Pareto Frontier Analysis

arXiv:2510. 03950v2 Announce Type: replace Abstract: Data-centric learning seeks to improve model performance from the perspective of data quality, and has been drawing increasing attention in the machine learning community.

By Shahriar Kabir Nahin, Wenxiao Xiao, Joshua Liu, Anshuman Chhabra, Hongfu Liu
arXiv AI
Aug 26

Evaluating Multiple LLM Generations with Validated Task Coverage

The paper introduces VTC-Bench, a five‑domain benchmark designed to evaluate multiple outputs from large language models (LLMs) by measuring Validated Task Coverage (VTC). VTC quantifies how many distinct, useful results are produced within a set number of attempts, using real‑data tasks that allow automatic, reproducible checks of output quality and task‑relevant distinctness without relying on model‑based judges. Experiments show that models which perform best on single‑draw quality do not always achieve the highest coverage, and simple output‑variation metrics fail to capture task‑relevant diversity, highlighting the importance of evaluating finite candidate sets directly.

By Florian Le Bronnec, Rio Yokota
arXiv AI
Aug 17

Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability -- Diverse Evaluation Is Required

arXiv:2608. 13566v1 Announce Type: cross Abstract: Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.

By Egor Shibaev, Vera Kudrevskaia, Timur Galimzyanov, Mikhail Evtikhiev, Ana Terna, Rastislav Rabatin, Timur Kudashev, Timofey Bryksin, Arina Puchkova, Patrik Bartak, Egor Bogomolov, Sergey Titov