arXiv AI

Machine-Readable Operator Manuals over the ISO 11783 Network for Offline, Multilingual LLM Assistance

The paper investigates how to synchronize operator manuals with tractor-side assistants over ISO 11783 CAN buses, enabling offline, multilingual LLM assistance. It benchmarks data transfer using Markdown, JSON, and XML, finding that Markdown files transfer fastest and that retrieval‑augmented generation works well for English but less so for other languages. The study shows that a new parameter group can be added to existing ISO 11783 mechanisms, suggesting a path toward standardization.

arXiv AI

Oct 7

Tool-calling retrieval versus vector RAG for a small Greek--English knowledge base: accuracy and robustness to how users type Greek

The study compares two methods for grounding assistants in a small Greek–English agricultural knowledge base: tool‑calling retrieval via a live data interface and vector retrieval‑augmented generation (RAG). Using the KyGround benchmark of 198 questions, vector RAG achieved 95.3% accuracy on canonical Greek questions, outperforming the tool agent’s 71.6% and revealing that the tool agent’s failures stem from literal searches that miss non‑verbatim matches. The results show that search tolerance to user typing variations—such as accents, capitalization, and Greeklish transliterations—is crucial for reliable community knowledge interfaces.

By Nikolaos D. Tantaroudas, Ilias Karachalios, Andrew J. McCracken

llms rag agents benchmarks

arXiv Machine Learning

Sep 24

Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints

arXiv:2609.28007v1 Announce Type: cross Abstract: Most Turkish-capable large language models (LLMs) are evaluated using general-purpose benchmarks rather than long, structurally complex domain docume...

By Imtiaz Ul Hassan, \"Oyk\"u Akbulut, Onur Kaya, Ardhendu Behera, Swagat Kumar, Peter Matthew, Yonghuai Liu

llms nlp benchmarks

arXiv AI

Oct 15

Frustratingly Easy Agentic RAG via Portable Skills: A Case Study in Customer Support

arXiv:2610.14428v1 Announce Type: new Abstract: Enterprise workflow automation typically requires retrieving a complete and accurate set of relevant information from large operational document corpor...

By Istok Bako, Bonan Min, Tiffany Shao, Claire Cheng, Yashar Mehdad

llms rag agents

Hugging Face Trending Papers

Sep 23

Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints

Most Turkish-capable large language models (LLMs) are evaluated using general-purpose benchmarks rather than long, structurally complex domain documents. This paper evaluates five open-weight 7B-8B mo...

llms nlp benchmarks

arXiv Computation and Language

Oct 15

Plant Memory: Technical Language Processing and a Verified Maintenance Ontology for Agent-Accessible Failure Knowledge from SAP PM Work Orders

arXiv:2610.19131v1 Announce Type: cross Abstract: Maintenance work orders are the largest written record of how industrial equipment fails, yet in most plants they are close to unusable for analysis....

By Akash Raj

llms agents

Hugging Face Trending Papers

Jul 21

RAGAL: A Frugal, Fully Local Retrieval-Augmented Assistant for Technical Support at a Government Agency

Public institutions hold large volumes of sensitive documents and support tickets that cannot leave the premises, ruling out cloud-hosted language models entirely. We report on RAGAL, a retrieval-augmented assistant for the technical-support team of AFIR, the Romanian Agency for Financing Rural Investments, built and operated under three hard constraints: zero data egress (no external API calls, even for synthetic data), a read-only mandate (the assistant drafts, humans execute), and a single 8 GB consumer laptop as the only development and training machine.

llms rag fine-tuning efficiency safety data-engineering

arXiv Computation and Language

2d ago

INVAR-Agri: Detecting Discrepancies in Multilingual Operator Manuals of Agricultural Implements with a Fine-Tuned LLM

arXiv:2610.35423v1 Announce Type: new Abstract: Operator manuals of agricultural implements are delivered in many languages, and the standards require that competent persons review every translation,...

By Timo Oksanen

llms fine-tuning

arXiv Computation and Language

Sep 24

Automated Extraction of Records of Processing Activities (RoPA) Using Hybrid RAG and Locally Deployed Large Language Models

The paper introduces RoPA Manager, a system that automates the extraction of Records of Processing Activities (RoPA) required by Vietnam’s new Personal Data Protection Law. It combines hybrid retrieval techniques—lexical ranking, dense‑vector search, and Reciprocal Rank Fusion—with locally deployed large language models to avoid data‑sovereignty issues. A Vietnamese RoPA benchmark of 32 organizations and 77 processing activities was created, and the system achieved robust scorer performance (F1 ≈ 0.95) and moderate end‑to‑end token coverage (≈ 50‑55%).

By To Duy Hinh, Nguyen Le Quoc Anh, Phan Van Tri, Khuong Nguyen-An

llms rag benchmarks

arXiv AI

Sep 15

Natural Language Knowledge Graph Query Execution: Leveraging Controlled Semantics in the LLM Context Window

arXiv:2609.14652v1 Announce Type: cross Abstract: Large Language Model (LLM) applications often transfer domain concepts into the model's context informally, through prompt prose, schema dumps, and e...

By Blake G. Fitch

llms benchmarks

arXiv AI

Aug 18

LLMs Get Smarter from Targeted Synthetic Multilingual Data

arXiv:2608. 15964v1 Announce Type: cross Abstract: Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt.

By Ishika Agarwal, Arkajyoti Charaborty, Tanner Sorensen, Neha Gupta, Andreas Stolcke

llms fine-tuning

arXiv Computation and Language

Aug 25

Industrial-Instruction: An End-to-End Framework for Building Instruction-Tuning and Benchmark Datasets from Industrial Technical Reports

arXiv:2608.22817v1 Announce Type: new Abstract: Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering, but their heterogeneous structure...

By Parsa Bakhtiari, Hassan Bashiri, Alireza Khalilipour, Masoud Nasiripour, Moharram Challenger

llms fine-tuning benchmarks

arXiv AI

Oct 2

Build2SPARQL: A Large-Scale Text-to-SPARQL Benchmark Dataset for Building Knowledge Graph Querying

Build2SPARQL is a large-scale benchmark dataset for translating natural-language questions into SPARQL queries over building knowledge graphs. The dataset is generated by a KG‑grounded pipeline that produces 6,136 executable SPARQL queries and 30,680 corresponding natural-language questions across six query-pattern families and five vocabulary registers, covering 201 building KGs. Human validation shows high semantic fidelity, naturalness, and operational plausibility, and retrieval‑augmented evaluation demonstrates significant accuracy gains for open‑weight language models.

By Wooyoung Jung

llms rag agents benchmarks

Related stories