Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

3,360 stories · RSS feed

arXiv AI

Aug 11

Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

arXiv:2608. 00326v2 Announce Type: replace Abstract: Tool calling allows large language models (LLMs) to invoke external computation during problem solving, a useful capability in various fields including AI for mathematics.

By Bohan Chen, Shivam N. Patel, Richard Hoffmann, Sam Looi, Tony Yue Yu

llms agents fine-tuning

arXiv Machine Learning

Aug 11

OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories

arXiv:2608. 08557v1 Announce Type: cross Abstract: Visual tool use has emerged as a fundamental capability for multimodal agents to actively acquire evidence beyond a fixed image encoding.

By Changhao Xiang, Shilin Zhang, Zheng Ma, Kanzhi Cheng, Ruize Ma, Yi Feng, Jianbing Zhang, Zhi Wang, Zhen Wu, Xinyu Dai, Lewei Lu

agents fine-tuning multimodal benchmarks

arXiv AI

Aug 11

SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs

arXiv:2603. 20253v3 Announce Type: replace-cross Abstract: Evaluating LLM agents for scientific tasks has focused on token costs while ignoring tool-use costs like simulation time and experimental resources.

By Yadi Cao, Sicheng Lai, Jiahe Huang, Yang Zhang, Zach Lawrence, Rohan Bhakta, Izzy F. Thomas, Mingyun Cao, Chung-Hao Tsai, Zihao Zhou, Yidong Zhao, Hao Liu, Alessandro Marinoni, Alexey Arefiev, Rose Yu

llms agents fine-tuning benchmarks

arXiv AI

Aug 11

Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models

arXiv:2608. 09227v1 Announce Type: new Abstract: Omnimodal language models (OLMs) enable unified audio-visual understanding, but processing long joint token sequences makes inference computationally prohibitive.

By Puneet Mathur, Manan Suri, Dinesh Manocha

llms nlp fine-tuning efficiency multimodal benchmarks

arXiv AI

Aug 11

Expert-Guided Multimodal Fusion for Unified Emotion and Sentiment Analysis

arXiv:2601. 07565v2 Announce Type: replace-cross Abstract: Multimodal emotion understanding requires the integration of heterogeneous data sources, including text, audio, and visual modalities, while simultaneously addressing discrete emotion recognition and continuous sentiment analysis.

By Jiaqi Qiao, Xinran Li, Yifan Lyu, Xiujuan Xu, Liu Yu

llms nlp fine-tuning multimodal benchmarks

arXiv AI

Aug 11

Build it, Break it, Repeat: Benchmarking and improving LLM-manipulated disinformation detection in social media posts

arXiv:2608. 09510v1 Announce Type: cross Abstract: Detecting machine-generated disinformation on social media is increasingly difficult as large language models (LLMs) make it easier to generate and rewrite misleading content at scale.

By Kevin Thomas, Milosz Kasprzyk, Reuel C Igbokwe Onuigbo, Elliott Pert, Cameron Tovey, Jo\~ao A. Leite, Olesya Razuvayevskaya, Carolina Scarton

llms fine-tuning benchmarks safety

arXiv Machine Learning

Aug 11

Label Granularity Skew in Federated Learning with Hierarchical Image Classification

arXiv:2608. 09236v1 Announce Type: new Abstract: Federated learning enables privacy-preserving collaboration across distributed devices without centralizing local data.

By Jaeheon Kim, Hokeun Kim, Bong Jun Choi

computer-vision fine-tuning safety

arXiv AI

Aug 11

SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

arXiv:2608. 09006v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks.

By Shiwei Gan, Xiao Liu, Yafeng Yin, Zhiwei Jiang, Bowen Guo, Lie Xie, Sanglu Lu, Hongkai Wen

llms fine-tuning efficiency

arXiv Machine Learning

Aug 11

LoRA-based Adaptation Alone Is Not Enough: Understanding the Limits of Foundation Models for Face Presentation Attack Detection

arXiv:2608. 09633v1 Announce Type: cross Abstract: Face presentation attack detection (PAD) aims to reliably detect a wide range of presentation attacks.

By Peter Lorenz, Anjith George, Marcel S\'ebastien

llms fine-tuning benchmarks

arXiv AI

Aug 11

How Far Do Foundation Models Transfer to Infant Signals? A Cross-Dataset Transfer Audit with a Unified Need Ontology

arXiv:2608. 08989v1 Announce Type: cross Abstract: Public infant cry corpora are small, label-incompatible, and almost always evaluated one corpus at a time.

By Wu Hangyu

rag fine-tuning

arXiv AI

Aug 11

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

arXiv:2608. 08907v1 Announce Type: cross Abstract: Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code.

By Delin Mao, Chenghao Sun, Jingwei Song, Chishui Chen, Linfeng Zhang

agents reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Aug 11

VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging

arXiv:2511. 18121v2 Announce Type: replace-cross Abstract: While Multimodal Large Language Models (MLLMs) excel on benchmarks, their processing paradigm differs from the human ability to integrate visual information.

By Ming Zhong, Yuanlei Wang, Liuzhou Zhang, Ruichuan An, Renrui Zhang, Hao Liang, Ming Lu, Ying Shen, Wentao Zhang

llms fine-tuning multimodal benchmarks

arXiv AI

Aug 11

Scaling Audio Models Efficiently: A Joint Study of Compute Constraints and Optimization Behavior

arXiv:2606. 22790v2 Announce Type: replace-cross Abstract: In this paper, we investigate the tradeoffs between compute allocation and model performance for two speech processing tasks: Automatic Speech Recognition (ASR) and Speech Emotion Recognition (SER).

By Vyom Agarwal, Mokshda Gangrade, Siddharth Pal, Jerry Wu

fine-tuning multimodal

arXiv AI

Aug 11

Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge

arXiv:2608. 07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures.

By Shibo Chu, Yuze Liu, Tiehua Zhang, Zhishu Shen, Lianghua He, Haofen Wang, Zhijun Ding

llms fine-tuning benchmarks safety

arXiv AI

Aug 11

CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecasting

arXiv:2608. 07693v1 Announce Type: cross Abstract: Generative traffic video forecasting aims to synthesize long-horizon, temporally coherent future videos of traffic scenes from a short observation history and textual descriptions.

By Quang Minh Dinh, Tuan Kiet Doan

llms fine-tuning benchmarks safety

arXiv AI

Aug 11

MathShikkha: A Controlled Study of Answer-Only and Chain-of-Thought Supervision for Bangla Mathematical Reasoning in Small Language Models

arXiv:2608. 08503v1 Announce Type: new Abstract: Mathematical reasoning remains challenging in low-resource languages such as Bangla.

By Rahma Simin Ali, Jawad Hossain

llms fine-tuning benchmarks

arXiv AI

Aug 11

Visual Distortion Detection in UGC Images Using Large Multimodal Models

arXiv:2608. 09122v1 Announce Type: cross Abstract: The localized depiction of perceptual quality has long been a crucial, yet underexplored, challenge in image quality assessment (IQA).

By Ziheng Jia, Yingji Liang, Jiaying Qian, Xiongkuo Min

llms fine-tuning multimodal

arXiv AI

Aug 11

Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models

arXiv:2512. 11614v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) relies on retrieved context to guide large language models (LLM), yet treats the retrieval as a heuristic rather than verifiable evidence -- leading to unsupported answers, hallucinations, and reliance on spurious context.

By Bj\"orn Deiseroth, Max Henning H\"oth, Kristian Kersting, Letitia Parcalabescu

llms rag agents fine-tuning benchmarks safety

arXiv AI

Aug 11

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games

arXiv:2604. 11741v2 Announce Type: replace Abstract: Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer game settings with imperfect and deceptive information.

By Keyang Zhong, Junlin Xie, Hefeng Wu, Haofeng Li, Guanbin Li

llms agents reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Aug 11

FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks

arXiv:2604. 10015v3 Announce Type: replace Abstract: Recent studies demonstrate that tool-calling capability enables large language models (LLMs) to interact with external environments for long-horizon financial tasks.

By Yupeng Cao, Haohang Li, Weijin Liu, Wenbo Cao, Anke Xu, Lingfei Qian, Xueqing Peng, Minxue Tang, Zhiyuan Yao, Jimin Huang, K. P. Subbalakshmi, Zining Zhu, Jordan W. Suchow, Yangyang Yu

llms agents reinforcement-learning fine-tuning benchmarks