arXiv AI

NL2SHACL-Bench: A Benchmark Suite for Natural Language to SHACL Translation

arXiv:2608. 07530v1 Announce Type: new Abstract: SHACL is a core technology for validating the conformance of RDF knowledge graphs (KGs).

arXiv AI

Jun 3

ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models

arXiv:2510. 09711v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches.

By Wenbin Guo, Xin Wang, Jiaoyan Chen, Lingbing Guo, Zhao Li, Zirui Chen

llms rag nlp efficiency benchmarks

arXiv AI

Jul 24

Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc

arXiv:2607. 20456v1 Announce Type: cross Abstract: Large language models excel at code generation for mainstream programming languages but struggle with rare, domain-specific languages such as MiniZinc, a constraint modeling language for combinatorial problems.

By Serdar Kadioglu, Karthik Uppuluri

llms fine-tuning

arXiv AI

Jul 21

GRIP: In-Parameter Graph Reasoning through Fine-Tuning Large Language Models

arXiv:2511. 07457v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in modeling sequential textual data and generalizing across diverse tasks.

By Jiarui Feng, Donghong Cai, Yixin Chen, Muhan Zhang

llms fine-tuning safety

arXiv AI

2d ago

Rewrite Once, Validate Anywhere: Producing OWL-Aware SHACL Constraints (Extended Version)

arXiv:2608. 14104v1 Announce Type: cross Abstract: The Shapes Constraint Language (SHACL) is a W3C recommendation to express syntactic constraints, called shapes, on RDF graphs.

By Anouk Oudshoorn, Piotr Gorczyca, D\"orthe Arndt

benchmarks

arXiv AI

Jun 24

Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules

arXiv:2508. 10971v2 Announce Type: replace-cross Abstract: Knowledge graphs (KGs) can be enhanced through rule mining; however, the resulting logical rules are often difficult for humans to interpret due to their inherent complexity and the idiosyncratic labeling conventions of individual KGs.

By Nasim Shirvani-Mahdavi, Chengkai Li

llms fine-tuning

arXiv AI

5d ago

Unified Multi-Dimensional Benchmark for Complex Graph Reasoning in Large Language Models

arXiv:2608. 12391v1 Announce Type: cross Abstract: Graph reasoning provides a promising testbed for evaluating the reasoning ability of large language models (LLMs), as graph instances can be programmatically generated, structurally controlled, and naturally scaled to long-input settings.

By Fali Wang, Ali Al-Lawati, Iliyas Bektas, Jinxuan Fang, Alek Melenski, Tianxiang Zhao, Yao Ma, Suhang Wang

llms rag fine-tuning benchmarks

arXiv AI

Jun 30

A Large-Scale Dataset for Molecular Structure-Language Description via a Rule-Regularized Method

arXiv:2602. 02320v4 Announce Type: replace-cross Abstract: Molecular function is largely determined by structure.

By Feiyang Cai, Guijuan He, Yi Hu, Jingjing Wang, Joshua Luo, Tianyu Zhu, Srikanth Pilla, Gang Li, Ling Liu, Feng Luo

llms safety

arXiv AI

2d ago

Jais 2: A Family of Arabic-Centric Open Large Language Models

arXiv:2608. 13580v1 Announce Type: cross Abstract: Jais 2 is a family of Arabic-centric large language models developed jointly by MBZUAI, Cerebras, and Inception, designed to advance Arabic-centric language modeling, with strong performance across the Arabic and culturally grounded benchmarks evaluated in this report.

By Mohamed Anwar, Abed Alhakim Freihat, George Ibrahim, Mostafa Awad, Abdelrahman Sadallah, Gurpreet Gosal, Gokulakrishnan Ramakrishnan, Sarath Chandran, Biswajit Mishra, Rituraj Joshi, Ahmed Frikha, Etienne Goffinet, Abhishek Maiti, Ali El Filali, Sarah AlBarri, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Awantika Shukla, Sajid siddiki, Samta Kamboj, Onkar Pandit, Sunil Kumar Sahu, AbdelRahman Elbadawy, Amr Mohamed, Ahmad Chamma, Evan Dufraisse, Abdelaziz Bounhar, Dani Bouch, Hadi Abdine, Guokan Shang, Fajri Koto, Yuxia Wang, Zhuohan Xie, Ali Mekky, Rania Elbadry, Sarfraz Ahmad, Momina Ahsan, Omar El Herraoui, Daniil Orel, Hasan Iqbal, Kareem Elzeky, Mervat Abassy, Kareem Elozeiri, Saadeldine Eletter, Farah Atif, Nurdaulet Mukhituly, Haonan Li, Xudong Han, Aaryamonvikram Singh, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Larry Murray, Avraham Sheinin, Joel Hestness, Natalia Vassilieva, Hector Xuguang Ren, Zhengzhong Liu, Michalis Vazirgiannis, Preslav Nakov

llms nlp efficiency benchmarks

arXiv AI

Jul 28

KG2Code: Bridging Knowledge Graphs and Large Language Models via Executable Code for Question Answering

arXiv:2607. 22652v1 Announce Type: new Abstract: Recent research has explored the integration of knowledge graphs (KGs) with large language models (LLMs) to enhance their performance on downstream knowledge-intensive tasks, particularly knowledge graph question answering (KGQA).

By Yike Wu, Nan Hu, Guilin Qi, Guohui Xiao, Chen Jiang, Xinchun Zou, Yuchen Lu, Songlin Zhai, Yongrui Chen, Yuyang Zhang, Xiaoguang Li, Lifeng Shang, Jiaoyan Chen, Jeff Z. Pan

llms rag agents nlp safety

arXiv AI

Aug 7

MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation

arXiv:2511. 14967v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown great promise in generating structured diagrams from natural language descriptions, particularly Mermaid sequence diagrams for software engineering.

By Basel Shbita, Farhan Ahmed, Chad DeLuca

llms benchmarks

arXiv AI

Jun 15

Achieving Precise Text-To-Cypher Via Grounded Knowledge Graph Data Generation

arXiv:2606. 14325v1 Announce Type: cross Abstract: Property Graphs are rapidly being adopted as database frameworks for representing heterogeneous data sources.

By Francesco Cazzaro, Jessica Lennon, Ariadna Quattoni

llms fine-tuning benchmarks

arXiv AI

Jun 12

SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation

arXiv:2606. 13647v1 Announce Type: cross Abstract: We introduce SkMTEB, the first comprehensive MTEB-style text embedding benchmark for Slovak, a low-resource West Slavic language, comprising 31 datasets across 7 task types -- nearly 4$\times$ the depth of existing multilingual benchmark coverage for Slovak.

By Marek \v{S}uppa, Andrej Ridzik, Daniel Hl\'adek, Nat\'alia K\v{n}a\v{z}ekov\'a, Vikt\'oria Ondrejov\'a

rag fine-tuning benchmarks

Related stories