arXiv:2606. 18307v1 Announce Type: cross Abstract: Optimizing the training data distribution for Supervised Fine-Tuning (SFT) dictates the capability of Large Language Models (LLMs).
By Zefan Wang, Lincheng Li, Tianyu Yu, Yuan Yao
The paper introduces a control‑variable framework for deep neural networks to mitigate omitted variable bias, particularly shortcut learning where covariates like demographics influence predictions. It refits the final layer of a pre‑trained network using cross‑fitting with ridge penalisation, orthogonalises covariate effects, and marginalises predictions over covariate distributions to achieve unbiased, interpretable results. Experiments on simulated images and neuroimaging data show consistent estimation of true effects and performance close to models trained on unconfounded data.
By Manuel Pfeuffer, Roshan Prakash Rane, Kerstin Ritter, Sonja Greven
arXiv:2511. 22823v2 Announce Type: replace-cross Abstract: Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire.
By Miao Zhang, Junpeng Li, Changchun Hua, Yana Yang
arXiv:2608. 13133v1 Announce Type: cross Abstract: Distributional shifts arise when the target deployment environment differs from the source environment that generated the training data.
By Zhiyi Li, Xiaojie Mao, Yunbei Xu, Ruohan Zhan
The paper introduces Inverse Knowledge Distillation (IKD), an attack‑agnostic technique that enhances adversarial transferability by maximizing the discrepancy between benign and adversarial prediction distributions on a surrogate model. IKD employs a CE/KL‑equivalent soft‑label objective to push adversarial predictions away from a fixed benign anchor, leveraging Fisher‑sensitive surrogate directions. The authors provide theoretical analysis showing CE and KL induce identical gradients, derive a lower bound on Fisher‑subspace overlap, and demonstrate through extensive ImageNet experiments that IKD consistently improves black‑box attack performance across CNN, ViT, and defended models.
By Wenyuan Wu, Yuan Sun, Yingke Chen, Chao Su, Xi Peng, Dezhong Peng, Xu Wang
arXiv:2605. 21422v3 Announce Type: replace Abstract: As LLMs continue to scale up, improving training efficiency heavily relies on effective data utilization.
By Qihao Lin, Guanxu Chen, Dongrui Liu, Jing Shao