arXiv Machine Learning

MLLM-DataEngine: Closing the Loop of Multimodal Instruction Tuning Data Generation

arXiv:2607. 15299v1 Announce Type: cross Abstract: In this paper, we propose MLLM-DataEngine, a novel closed-loop system that bridges data generation, model training, and evaluation.

arXiv AI

Jun 9

Exploring Autonomous Agentic Data Engineering for Model Specialization

arXiv:2605. 30407v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance on general tasks, while often struggling to adapt to specialized domains without high-quality domain-specific data.

By Yujie Luo, Xiangyuan Ru, Jingsheng Zheng, Jingjing Wang, Yuqi Zhu, Jintian Zhang, Runnan Fang, Kewei Xu, Ye Liu, Zheng Wei, Jiang Bian, Zang Li, Shumin Deng

llms agents data-engineering

arXiv AI

Jul 1

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

arXiv:2606. 31270v1 Announce Type: cross Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility.

By Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy, Yuhui Zhang

llms agents fine-tuning multimodal benchmarks

arXiv AI

1d ago

ML-AutoResearch: Training Machine Learning Research Agents with Automatically Generated Environments

arXiv:2603. 17216v2 Announce Type: replace Abstract: With the advent of AI agents, automated scientific discovery is becoming an increasingly plausible goal.

By Ziyang Cai, Amir Saeidi, Harkirat Behl

agents fine-tuning benchmarks

arXiv AI

Jul 10

Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs

arXiv:2503. 12999v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have demonstrated exceptional performance in various multi-modal tasks.

By Ruichuan An, Kai Zeng, Ming Lu, Sihan Yang, Renrui Zhang, Huitong Ji, Hao Liang, Wentao Zhang

llms fine-tuning multimodal benchmarks data-engineering

arXiv AI

Jun 8

DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving

arXiv:2606. 07001v1 Announce Type: cross Abstract: High-quality training data is essential to large language models (LLMs) and typically requires extensive and costly manual curation.

By Chao Deng, Shaolei Zhang, Ju Fan, Xiaoyong Du

llms agents benchmarks data-engineering

arXiv AI

Jun 16

LLM4RTL: Tool-Assisted LLM for RTL Generation

arXiv:2606. 15500v1 Announce Type: cross Abstract: Large language models (LLMs) have facilitated impressive progress in software engineering, code generation, tooling, and systems.

By Jing Jin, Robert Chu, Ning Yan, Masood S. Mortazavi

llms fine-tuning benchmarks

arXiv AI

Aug 7

PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis

arXiv:2608. 05249v1 Announce Type: cross Abstract: Real-world multimodal instructions often bundle multiple requirements with unequal importance, yet most multimodal training data still reduce instruction following to answering one self-contained question.

By Xiaomin He, Dongling Xiao, Jiahao Xie, Ruiqi Lu, Qianle Wang, Zhongbin Guo, Wanxuan Sun

multimodal benchmarks

arXiv Machine Learning

Jun 30

DataComp-VLM: Improved Open Datasets for Vision-Language Models

arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.

By Matteo Farina, Vishaal Udandarao, Thao Nguyen, Selim Kuzucu, Maximilian B\"other, Andreas Hochlehnert, Adhiraj Ghosh, Marianna Nezhurina, Karsten Roth, Joschka Struber, Yuhui Zhang, Sebastian Dziadzio, Elaine Sui, Soumya Jahagirdar, Dhruba Ghosh, Hasan Hammoud, Thomas De Min, Simone Caldarella, Jehanzeb Mirza, Sedrick Keh, Mehdi Cherti, Hilde Kuehne, Bernt Schiele, Serena Yeung-Levy, Muhammad Ferjad Naeem, Federico Tombari, Ana Klimovic, Elisa Ricci, Matthias Bethge, Sewoong Oh, Ameya Prabhu, Alessio Tonioni, Jenia Jitsev, Massimiliano Mancini, Ludwig Schmidt, Nikhil Parthasarathy

llms multimodal benchmarks

arXiv Machine Learning

Jun 9

Towards Automated Kernel Generation in the Era of LLMs

arXiv:2601. 15727v3 Announce Type: replace Abstract: The performance of modern AI systems is fundamentally constrained by the quality of their underlying GPU kernels, which translate high-level algorithmic semantics into low-level hardware operations.

By Yang Yu, Peiyu Zang, Chi Hsu Tsai, Haiming Wu, Yixin Shen, Jialing Zhang, Haoyu Wang, Zhiyou Xiao, Jingze Shi, Yuyu Luo, Wentao Zhang, Chunlei Men, Guang Liu, Yonghua Lin

llms agents benchmarks

arXiv AI

Jul 24

PlanE: Meta Planning of Data, Tuning, and Inference for Extractive-based LLMs

arXiv:2607. 20470v1 Announce Type: new Abstract: Enhancing the task-specific capabilities of Large Language Models (LLMs) primarily requires substantial instruction-tuning datasets.

By Jiacheng Wang, Weiyan Zhang, Guangya Yu

llms fine-tuning

arXiv AI

Jul 28

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

arXiv:2607. 22769v1 Announce Type: cross Abstract: The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data.

By He Zhang

llms fine-tuning benchmarks data-engineering

arXiv AI

Jul 16

Advancing Multimodal Judge Models through a Capability-Oriented Benchmark and MCTS-Driven Data Generation

arXiv:2603. 00546v2 Announce Type: replace Abstract: Using Multimodal Large Language Models (MLLMs) as judges to achieve precise and consistent evaluations has gradually become an emerging paradigm across various domains.

By Zeyu Chen, Huanjin Yao, Ziwang Zhao, Min Yang

llms multimodal benchmarks safety

Related stories