The paper introduces Pairwise Difference Learning (PDL) for classification, extending a meta‑learning technique originally designed for regression. PDL learns a function that predicts the difference between outcomes of two instances; predictions for a new instance are obtained by averaging differences with all training examples. Experiments show that the resulting PDL classifier surpasses state‑of‑the‑art methods, and the authors release a user‑friendly Python implementation.
By Mohamed Karim Belaid, Maximilian Rabus, Eyke H\"ullermeier
arXiv:2610.27669v1 Announce Type: cross
Abstract: Deep neural networks often fail at small-sample regression, collapsing to low-variance predictions that miss individual differences. Here, we introdu...
By Robert Bain, Marjan Rashidi, Elizabeth Chrastil
arXiv:2403. 05532v2 Announce Type: replace Abstract: We introduce Tune without Validation (Twin), a simple and effective pipeline for tuning learning rate and weight decay of homogeneous classifiers without validation sets, eliminating the need to hold out data and avoiding the two-step process.
By Lorenzo Brigato, Stavroula Mougiakakou
arXiv:2607. 10391v1 Announce Type: cross Abstract: Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks.
By Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig
The paper introduces Backward Kernel Herding, an algorithm that iteratively removes data points to create representative subsets for kernel learning, achieving performance comparable to state‑of‑the‑art methods while speeding up subsampling when the reduced size is less than half the original dataset. It also proposes Flexible Kernel Thinning, an extension that allows construction of subsets of any size, not just successive halvings, and demonstrates that this method often yields the best predictive performance. Experiments on Gaussian Processes and Kernel Support Vector Machines show that Backward Kernel Herding excels in training‑time efficiency, while Flexible Kernel Thinning offers superior predictive accuracy and competitive memory usage, emphasizing the need to choose a reduction strategy based on the desired trade‑off between performance, cost, and memory.
By Blanca Cano-Camarero, Yago R. Aguado-Carrillo-de-Albornoz, \'Angela Fern\'andez-Pascual, Jos\'e R. Dorronsoro
The paper refactors and expands the scikit-rebate Python package, adding new Relief‑Based Algorithm (RBA) variants such as SWRF*, mu‑Relief, and five novel methods that use alternative neighbor selection and feature scoring strategies. Benchmarking across diverse genomic simulations shows that most RBAs, except mu‑Relief, effectively detect 2‑way interactions in noisy data, with far‑scoring variants like MultiSWRFDB* excelling at interaction detection but being less sensitive to main effects. The refactored package achieves 10‑ to 35‑fold runtime reductions, and the new RBAs maintain strong performance for both main effects and 2‑way epistatic interactions, preserving predictive signals for downstream modeling.
By Kia Kazemi-Nia, Harsh Bandhey, Philip J. Freda, Ryan J. Urbanowicz
arXiv:2609.24126v1 Announce Type: cross
Abstract: Black-box machine learning models increasingly deliver strong predictions, but extracting useful information from them, such as a set of important fe...
By Xuhui Liu, Lili Zheng
The paper introduces LIDFS, an unsupervised feature selection technique that uses Local Intrinsic Dimensionality (LID) to rank features based on their local discriminability. By treating each feature as a point and estimating its LID via maximum likelihood from distances to its k nearest neighboring features, LIDFS assigns higher importance to features with lower LID and refines the ranking to remove locally redundant features. Experiments on 23 high‑dimensional datasets show that LIDFS outperforms both classic and state‑of‑the‑art methods in supervised and clustering evaluations while running faster than its closest competitor.
By Oussama Chelly, Michael E. Houle, Ken-ichi Kawarabayashi, Muhammad Rajabinasab, Arthur Zimek
The paper introduces a novel Siamese network framework that integrates Group‑Lasso for feature selection, aiming to identify compact feature subsets that preserve intrinsic data representations while maintaining high classification performance. Two variants are proposed—one using Euclidean distance and another employing a modified Jaccard similarity for real‑valued vectors—to better handle high‑dimensional sparse data. Experiments on 12 benchmark datasets with feature dimensions up to 7,070 demonstrate that the method consistently selects discriminative features and achieves competitive or superior classification results across eight classifiers, with embedding quality evaluated via UMAP and t‑SNE.
By Nisargo Ranjan Pal, Pradeep Singh
arXiv:2604. 05635v2 Announce Type: replace Abstract: Numerical preprocessing remains a critical component of tabular deep learning, as the representation of continuous features can strongly affect downstream performance.
By Manish Kumar, Anton Frederik Thielmann, Christoph Weisser, Benjamin S\"afken
Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.
By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
arXiv:2610.09863v1 Announce Type: cross
Abstract: Standard information retrieval metrics, such as mean Average Precision (mAP), assess performance one query at a time, based on how the similarities b...
By Bill Psomas, Mohammad Mahdi, Michalis Thomas, Danda Pani Paudel, Giorgos Tolias, Giorgos Kordopatis-Zilos