arXiv AI

CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases

arXiv:2408. 03910v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval and MBPP, but struggle with handling entire code repositories.

arXiv AI

5d ago

Unified Multi-Dimensional Benchmark for Complex Graph Reasoning in Large Language Models

arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.

By Fali Wang, Ali Al-Lawati, Iliyas Bektas, Jinxuan Fang, Alek Melenski, Tianxiang Zhao, Yao Ma, Suhang Wang

llms rag fine-tuning benchmarks

arXiv AI

Jun 3

Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisition

arXiv:2606. 03657v1 Announce Type: new Abstract: Large language models for code generation often need to use APIs that are absent from their pretraining data.

By Jinnuo Liu, Yue Peng, Jinhan Niu, Hongyi Wen

llms agents fine-tuning benchmarks

arXiv AI

Aug 6

Text2GraphQuery-Bench: A Text to Graph Query Benchmark

arXiv:2602. 11745v2 Announce Type: replace Abstract: Graph models are fundamental to data analysis in domains rich with complex relationships.

By Songlin Lyu, Lujie Ban, Zihang Wu, Tianqi Luo, Jirong Liu, Ayoub Moussaid, Oskar van Rest, Heng Lin, Chenhao Ma, Nan Tang, Shipeng Qi, Yongchao Liu, Zhan Qiu, Juelu Zhang, Jiajun Zheng

llms fine-tuning benchmarks data-engineering

arXiv AI

Jun 2

MemGraphRAG: Memory-based Multi-Agent System for Graph Retrieval-Augmented Generation

arXiv:2606. 00610v1 Announce Type: cross Abstract: Retrieval-Augmented Generation (RAG) has become an essential method for mitigating hallucinations in Large Language Models (LLMs) by leveraging external knowledge.

By Chuanjie Wu, Zhishang Xiang, Yunbo Tang, Zerui Chen, Qinggang Zhang, Jinsong Su

llms rag agents benchmarks safety

arXiv AI

Jul 7

Query2Diagram: Answering Developer Queries with UML Diagrams

arXiv:2604. 23816v2 Announce Type: replace-cross Abstract: Software documentation frequently becomes outdated or fails to exist entirely, yet developers need focused views of their codebase to understand complex systems.

By Oleg Baryshnikov, Anton M. Alekseev, Sergey I. Nikolenko

llms fine-tuning benchmarks

arXiv AI

Jun 12

ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs

arXiv:2606. 12451v1 Announce Type: new Abstract: Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck.

By Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber, Sahil Bansal

llms rag agents fine-tuning benchmarks

arXiv AI

Jul 28

KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering

arXiv:2607. 22652v1 Announce Type: new Abstract: Recent research has explored the integration of knowledge graphs (KGs) with large language models (LLMs) to enhance their performance on downstream knowledge-intensive tasks, particularly knowledge graph question answering (KGQA).

By Yike Wu, Nan Hu, Guilin Qi, Guohui Xiao, Chen Jiang, Xinchun Zou, Yuchen Lu, Songlin Zhai, Yongrui Chen, Yuyang Zhang, Xiaoguang Li, Lifeng Shang, Jiaoyan Chen, Jeff Z. Pan

llms rag agents nlp safety

arXiv AI

5d ago

CangjieBench: Benchmarking LLMs on a Low-Resource General-Purpose Programming Language

arXiv:2603. 14501v2 Announce Type: replace-cross Abstract: Large Language Models excel in high-resource programming languages but struggle with low-resource ones.

By Junhang Cheng, Fang Liu, Jia Li, Chengru Wu, Nanxiang Jiang, Li Zhang

llms rag agents benchmarks

arXiv AI

Jun 16

AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion

arXiv:2601. 19697v2 Announce Type: replace-cross Abstract: Repository-level code completion remains a challenging task for existing code large language models (code LLMs) due to their limited understanding of repository-specific context and domain knowledge.

By Tianyue Jiang, Yanli Wang, Yanlin Wang, Daya Guo, Ensheng Shi, Yuchi Ma, Jiachi Chen, Zibin Zheng

llms rag reinforcement-learning benchmarks

arXiv AI

Jul 3

XSearch: Explainable Code Search via Concept-to-Code Alignment

arXiv:2605. 16046v2 Announce Type: replace-cross Abstract: Semantic code search has been widely adopted in both academia and industry.

By Yiming Liu, Ruofan Liu, Yun Lin, Zicong Zhang, Weiyu Kong, Pengnian Qi, Xiao Cheng, Weinan Zhang, Qianxiang Wang, Linpeng Huang

rag benchmarks safety

arXiv AI

Jul 10

ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation

arXiv:2607. 08691v1 Announce Type: cross Abstract: Repository-level code generation requires implementing target functions while accounting for complex cross-file dependencies and project-specific conventions.

By QiHong Chen, Aaron Imani, Iftekhar Ahmed

agents

arXiv AI

Jul 24

Demonstrating GenDB: Instance-Optimized and Customized Query Processing Code Generation via LLM Agents

arXiv:2607. 20630v1 Announce Type: cross Abstract: Traditional query processing engines require continuous development and extensions to support new techniques and user requirements, and in some cases, entirely new systems must be built from scratch.

By Jiale Lao, Immanuel Trummer

llms agents benchmarks data-engineering

Related stories