arXiv Computation and Language
Sep 24

UniDataAgent: An Ontology-Grounded Agent for Enterprise Question-to-Report Automation

UniDataAgent (UniDataAgent) is an ontology‑grounded system designed to automate enterprise question‑to‑report tasks while preserving organization‑specific semantics. It separates semantic acquisition from online execution, with an Ontology Acquisition and Validation (OAV) stage that builds versioned ontologies from metadata, business knowledge, and expert input, and a Question‑to‑Report Execution (QRE) stage that retrieves semantic contracts, coordinates skills and data tools, validates results, and produces evidence‑linked reports. In a deployment across 27 enterprise tables and thousands of metric types, ontology construction took a few hours versus a week manually, and report generation took minutes versus several working days, achieving 95.0% strict accuracy on real business questions compared to 72.5% for document RAG.

By Yutai Duan, Yahui Zhao, Zhangti Li, Yu Ma, Zhenfeng Qi, Shaoyang Yuan, Jing Fan, Jie Liu
arXiv AI
Jun 9

Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting

arXiv:2606. 09809v1 Announce Type: new Abstract: AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs.

By Avijit Ghosh, Anka Reuel, Jenny Chim, Wm. Matthew Kennedy, Srishti Yadav, Jennifer Mickel, Yanan Long, Andrew Tran, Anastassia Kornilova, Damian Stachura, Kevin Klyman, Felix Friedrich, Jeba Sania, Max Lamparth, Jan Batzner, Anoop Mishra, Eliya Habba, Yixiong Hao, Nathan Heath, Shalaleh Rismani, Usman Gohar, Andrea Loehr, David Manheim, Ruchira Dhar, Sree Harsha Nelaturu, Aarush Sinha, Leshem Choshen, Drishti Sharma, Ishan Khire, Amit Saha, Subramanyam Sahoo, Michael Hardy, Michael Alexander Riegler, Kabir Manghnani, Michelle Lin, Yanan Jiang, Yilin Huang, Asaf Yehudai, Jessica Ji, Aris Hofmann, Mubashara Akhtar, Nuno Moniz, Yacine Jernite, Stella Biderman, Zeerak Talat, Sanmi Koyejo, Mykel Kochenderfer, Irene Solaiman
arXiv AI
Sep 12

Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation

Benchmark Radar is a living database and search engine that aggregates AI benchmark papers, datasets, code, and score histories. It automatically discovers new benchmark resources from 37 sources, maintains a catalog of 1,283 records with 12,916 numeric observations, and provides tools such as a web dashboard, CLI, and downloadable evidence for researchers. The system also offers visualizations like a Pareto frontier and trend views to help users assess benchmark saturation and adoption.

By Koutian Wu, Junjie Zhou, Ergan Shang, Jiayu Wang, Pengqian Han, Junkai Wang, Wanghan Xu
OpenAI Blog
Sep 17

Introducing Astra for Law

OpenAI for Law brings frontier intelligence for law, custom firm workflows, connected legal data sources, and legal-grade controls for confidential client work.

arXiv AI
Jun 29

POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation

arXiv:2606. 23533v2 Announce Type: replace Abstract: Recent large language models (LLMs) are good at general text generation, but it is still hard to use them for domain-specific data generation because the output must follow strict formatting and structural rules.

By Hung Phan, Aniroop Naladala, Dubey Avanindra, Supryia Chinthavali, Lunga Dalton, Ali Jannesari