arXiv AI

Capturing LLM Capabilities via Evidence-Calibrated Query Clustering

arXiv:2605. 17110v2 Announce Type: replace Abstract: Query clustering organizes queries into groups that reflect shared latent capability demands, enabling capability-aware LLM evaluation.

arXiv AI

2d ago

RepBench: Compiling Benchmarks into Capability Representations for Large Language Models

arXiv:2607. 28008v2 Announce Type: replace-cross Abstract: Representation engineering reads and steers capability directions in large language models, yet methods are typically evaluated on paper-specific synthetic data.

By Yanshi Li, Xueru Bai, Shuman Liu, Long Zhang

llms benchmarks

arXiv AI

Jun 26

The Capability Frontier: Benchmarks Miss 82% of Model Performance

arXiv:2606. 26836v1 Announce Type: new Abstract: Existing benchmarks typically report accuracy for a single model on a single run.

By Bradley Fowler, Ryan Smith, Daniel Thi Graviet, William Myers, Joshua Greaves, Narmeen Fatimah Oozeer, Ant\'ia Garc\'ia, Philip Quirke, Amirali Abdullah, Fazl Barez, Shriyash Kaustubh Upadhyay

llms agents benchmarks

arXiv AI

Aug 11

SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests

arXiv:2608. 08640v1 Announce Type: new Abstract: Large language model agents increasingly rely on reusable skills to extend their capabilities beyond parametric knowl- edge.

By Donghong Jiang, Endian Lin, Luoping Cui, Hanqing Liu, Mingjie Liu, Fan Yang, Hong Wang, Zhao Yang, Chuang Zhu

llms agents benchmarks safety

arXiv AI

Jul 20

CRAFT: Clustering Rubrics to Diagnose Weak LLM Capabilities and Generate Targeted Fine-Tuning Data

arXiv:2607. 16122v1 Announce Type: new Abstract: Evaluations should do more than measure a models current performance.

By Vipul Gupta, Zihao Wang, Razvan-Gabriel Dumitru, MohammadHossein Rezaei, Aakash Sabharwal, Yunzhong He

llms fine-tuning benchmarks

Hugging Face Trending Papers

Aug 9

SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests

Large language model agents increasingly rely on reusable skills to extend their capabilities beyond parametric knowl- edge. However, retrieving the appropriate skill from a large- scale library remains challenging because realistic user re- quests are often concise and underspecified, stating only the task goal while leaving the required capabilities and execu- tion steps implicit.

llms agents benchmarks safety

arXiv AI

Jun 30

SemJoin: Semantic Join Optimization

arXiv:2606. 29532v1 Announce Type: cross Abstract: Integrating unstructured data into relational database systems is increasingly important as demand grows for natural language querying and analysis.

By Christopher Gou, Aditya Banerjee, Jiaxuan Wang, Chunwei Liu

llms rag agents

arXiv AI

Aug 10

Semantic Adapter Routing with Fine-Tuning Task Embeddings

arXiv:2606. 19079v2 Announce Type: replace Abstract: Parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters.

By Enrico Cassano, Micha{\l} Brzozowski, Paolo Mandica, Zuzanna Dubanowska, Neo Christopher Chung

llms rag diffusion fine-tuning

arXiv AI

Jun 12

ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs

arXiv:2606. 12451v1 Announce Type: new Abstract: Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck.

By Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber, Sahil Bansal

llms rag agents fine-tuning benchmarks

arXiv Machine Learning

Jun 8

From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing

arXiv:2606. 06924v1 Announce Type: new Abstract: Existing LLM routing methods typically treat a model's single response to a query as its capability label for training routers.

By Guannan Lai, Haoran Hu, Long Chen, Zhenguo Li, Han-Jia Ye

llms

arXiv Machine Learning

Jun 9

A Unified Structured Query Understanding Framework for Industrial Semantic Search

arXiv:2605. 27441v2 Announce Type: replace-cross Abstract: Query understanding in large-scale industrial search systems is typically implemented as a cascade of disparate, task-specific components.

By Ping Liu, Qianqi Shen, Jianqiang Shen, Chunnan Yao, Kevin Kao, Rajat Arora, Dan Xu, Baofen Zheng, Yunxiang Ren, Benjamin Le, Ali Hooshmand, Igor Lapchuk, Juan Bottaro, Raghavan Muthuregunathan, Caleb Johnson, Liangjie Hong, Jingwei Wu, Wenjing Zhang

llms rag efficiency

arXiv Machine Learning

Jul 2

FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

arXiv:2507. 10540v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has created a diverse landscape of models, each excelling at different tasks.

By Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 28

TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs

arXiv:2607. 22639v1 Announce Type: new Abstract: Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search.

By Sai Shruthi Sistla, Ashutosh Hathidara, Christopher Toukmaji, Mayank Shrivastava, Karthikeyan Asokkumar

llms rag fine-tuning

Related stories