arXiv AI By Denys Herasymuk, Anastasiia Mozghova, Nazar Protsiv, Vladyslav Sydorak, Julia Stoyanovich

VirnyFlow: Optimizing ML Pipelines for Accuracy, Fairness, and Stability at Scale

Read the original on arXiv AI →

arXiv:2506. 01584v2 Announce Type: replace-cross Abstract: Developing machine learning (ML) systems for real-world deployment requires navigating context-dependent trade-offs among accuracy, fairness, stability, and other objectives.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Jun 10

Piper: A Programmable Distributed Training System

arXiv:2606. 11169v1 Announce Type: cross Abstract: Large-scale model training increasingly relies on composing multiple parallelism strategies, such as data, pipeline, and expert parallelism, together with memory-saving optimizations like ZeRO.

By Megan Frisella, Shubham Tiwari, Andy Ruan, Yi Pan, Parker Gustafson, Mat Jacob, Gilbert Bernstein, Stephanie Wang
arXiv Machine Learning
Aug 18

Evolving Executable Pipeline Programs for AutoML with Language Models

arXiv:2608. 16416v1 Announce Type: new Abstract: Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space.

By Sofoklis Kitharidis, Cor J. Veenman, Jan N. van Rijn, Thomas B\"ack, Niki van Stein
arXiv Machine Learning
Sep 14

SAGE-Loop: Reliable Closed-Loop LLM-Driven AutoML with Trial-and-Correction and Adaptive Ensembling

SAGE-Loop is a new closed‑loop, self‑adaptive AutoML framework that uses large language models to generate and validate machine learning pipelines in multiple rounds, allowing trial‑and‑repair and adaptive ensemble selection for both supervised and unsupervised tasks. It addresses the lack of instant feedback and correction in existing AutoML by enabling process‑level recovery from failures and dynamic use of model diversity. Experiments on 20 public datasets show consistent improvements in performance and stability across classification, regression, and clustering, and demonstrate the system’s ability to recover from execution failures.

By Junquan Gu, Shibo Cui, Xiangfeng Luo, Hang Yu