arXiv AI

Semantic Layer Induction from Raw Telemetry via Hierarchical LLM and RAG Abstraction

The paper introduces an end‑to‑end framework that automatically builds a business semantic layer from raw application logs. It uses a two‑stage abstraction: first, large language models identify high‑level business features with industry knowledge, and second, a structured pipeline refines data, retrieves relevant information, filters, clusters semantically, and assigns canonical names. Evaluation on production‑scale telemetry shows significant gains in semantic quality, noise reduction, and maintenance effort, with a 0.87 Cohen’s kappa in an LLM‑as‑Judge assessment.

arXiv AI

Sep 2

Automated Event Log Generation from Unstructured Text Using Finetuned LLMs

The paper introduces a scalable framework that uses finetuned large language models (LLMs) to translate unstructured textual resources into structured event logs for process mining. By creating a new text-to-log dataset and finetuning LLMs on it, the authors demonstrate that the resulting models produce high‑fidelity event logs, outperforming few‑shot or zero‑shot prompting methods. This approach enables previously unused organizational data, such as incident tickets and manuals, to be incorporated into process mining workflows.

By Maximilian Seeth, Gabriel Marques Tavares, Daniel Schuster

llms fine-tuning

Hugging Face Trending Papers

Sep 1

Automated Event Log Generation from Unstructured Text Using Finetuned LLMs

Process mining (PM) provides a powerful framework for discovering and optimizing operational processes from event data. However, the efficacy of PM techniques is strictly predicated on the availabilit...

llms fine-tuning

arXiv AI

Jul 3

AgenticDataBench: A Comprehensive Benchmark for Data Agents

arXiv:2607. 01647v1 Announce Type: cross Abstract: Data science aims to derive actionable insights from heterogeneous raw data, unlocking the value of the massive amounts of data generated in modern society.

By Zhaoyan Sun, Shan Zhong, Daizhou Wen, Jiaxing Han, Guoliang Li, Ying Yan, Peng Zhang, Yu Su, Xiang Qi, Baolin Sun, Chengyuan Yang, Tao Fang, Huaiyu Ruan

llms agents benchmarks

arXiv AI

Jul 29

Unified Semantic Modeling Framework for Large-Scale Job Understanding at LinkedIn

arXiv:2607. 24783v1 Announce Type: new Abstract: Job understanding is critical to LinkedIn's mission of connecting talent with opportunity.

By Dan Xu, Baofen Zheng, Jianqiang Shen, Qi Xiao, Benjamin Hoan Le, Wen Pu, Saurabh Gupta, Ran Zhou, Neha Saraf, Alice Leung, Qianqi Shen, Liangjie Hong, Jingwei Wu, Wenjing Zhang

llms fine-tuning

arXiv AI

Jul 28

Retrieval-Augmented Generation of Ontologies from Relational Databases

arXiv:2506. 01232v2 Announce Type: replace-cross Abstract: Deriving OWL ontologies from relational database schemas supports semantic interoperability and downstream tasks such as knowledge graph population, ontology-based data access, graph-based learning, and automated reasoning.

By Nadeen Fathallah, Mojtaba Nayyeri, Athish A Yogi, Ratan Bahadur Thapa, Hans-Michael Tautenhahn, Anton Schnurpel, Steffen Staab

llms rag

arXiv AI

Sep 10

Hypergraph Enterprise Agentic Reasoner over Heterogeneous Business Systems

arXiv:2605.14259v3 Announce Type: replace Abstract: Applying Large Language Models (LLMs) to heterogeneous enterprise systems is hindered by hallucinations and failures in multi-hop, n-ary reasoning....

By Ling Wang, Xin Liu, Songnan Liu, Jianan Wang, Cheng Cheng, Yihan Zhu, Enyu Li, Yu Xiao, Jiangyong Xie, Duogong Yan, Jiangyi Chen

llms agents safety

arXiv AI

Jun 15

Abstracting Cross-Domain Action Sequences into Interpretable Workflows

arXiv:2606. 14654v1 Announce Type: new Abstract: Sequential or time-stamped interaction logs provide objective records of digital application usage, yet their granularity and noise often obscure meaningful insights into people's work.

By Gaurav Verma, Scott Counts

llms safety

arXiv Machine Learning

Jun 15

SemPiper: Interactive Code Synthesis for Semantic Operators in Machine Learning Pipelines

arXiv:2606. 14361v1 Announce Type: new Abstract: Machine learning (ML) pipelines require extensive data preparation, feature engineering, and integration across heterogeneous sources, making them tedious and error-prone to develop.

By Olga Ovcharenko, Luciano Duarte, Sebastian Schelter

llms

arXiv AI

Sep 10

DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents

DI-Bench is a pipeline that automatically creates realistic data intelligence benchmarks for enterprise agents by linking data tables, dimensions, metrics, and documents into an artifact graph. It generates questions that combine structured data queries with knowledge retrieval, validates answers via query execution and LLM-generated questions, and has produced a 731-task benchmark covering knowledge retrieval, analytical computation, and rule‑grounded reasoning. Evaluation of four models on this benchmark shows that only 32% accuracy is achieved on computational tasks that involve business rules modifying the computation.

By Jiangyun Zhang, Kristen Surrao, Torpong Nitayanont, Yupei Zhang, Roopali Singh, Zhiyu Chen, Julia Huang, Zhou Tang, Shayan Ali Akbar, Omar Alonso, Erwin Cornejo, Yuan Li, Yi Zhang

llms agents benchmarks

arXiv AI

Sep 3

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

The paper investigates hallucination detection in black‑box large language models by leveraging two accessible signals: semantic entropy, which captures disagreement among sampled response meanings, and token‑level uncertainty derived from log‑probabilities. It introduces a TopK aggregation technique, a hybrid CoCoA method combining uncertainty with semantic dissimilarity, and two supervised approaches—Gated and Stacked—that integrate token and semantic features. Across seven benchmarks and four language models, the supervised Stacked method performs best in many cases, while TopK and CoCoA remain competitive without labeled data, though all methods require careful threshold calibration.

By Urja Pawar, Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Abhishek Mandal, Houssem Chatbri, Christopher Martin

llms multimodal benchmarks safety

arXiv Machine Learning

Jul 21

FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection

arXiv:2605. 22779v2 Announce Type: replace-cross Abstract: Production systems generate millions of log lines daily, yet most anomaly detectors operate at the session or window-level, flagging groups of lines rather than identifying the specific message responsible.

By Huanchi Wang, Zihang Huang, Yifang Tian, Kristina Dzeparoska, Hans-Arno Jacobsen, Alberto Leon-Garcia

llms

arXiv AI

Jul 16

How Far Can Root Cause Analysis Go on Real-World Telemetry Data?

arXiv:2607. 13548v1 Announce Type: new Abstract: Identifying root causes in production microservice failures requires reasoning over large-scale, multimodal telemetry spanning metrics, logs, and traces, a problem that has proved resistant to both classical and LLM-based approaches.

By Athira Gopal, Ashwanth Krishnan

llms agents multimodal benchmarks

Related stories