arXiv Machine Learning August 28, 2026 By Meiwei Zhang, Eduardo Miranda, Bruce Baynes, Suvigya Jain, Wanlong Chen, Tao He, Sergey Borodavkin

Beyond Capability Benchmarks: Learning Operational Fingerprints of LLM Cloud Services from Production Incident Metadata

Read the original on arXiv Machine Learning →

The paper introduces Operational Embedding (OpEmbed), a framework that learns compact operational fingerprints of large language model (LLM) cloud services from structured, privacy‑preserving support‑case metadata, without using the case text. OpEmbed aggregates model–time windows into an eight‑channel operational signature and uses temporal contrastive learning, cross‑view reconstruction, and generational‑ordinality regularization to produce a low‑dimensional representation. Evaluated on over 33,000 production support cases across seven LLM families at Google Cloud, OpEmbed recovers interpretable family‑ and version‑level structure, improves leave‑one‑model‑out operational forecasting, remains effective with limited early‑window data, and supports cross‑model fault‑type transfer, providing practical insights for model onboarding, support readiness assessment, and operational monitoring.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 15

A Machine Learning API for Earth Observation Data Cubes Based on openEO

arXiv:2609.13453v1 Announce Type: new Abstract: Earth Observation (EO) data are increasingly organized as spatio-temporal data cubes, while machine learning (ML) methods operate on tabular feature ma...

By Brian Pondi, Jonas Hurst, Rolf Simoes, Jonas Starke, Marius Appel, Edzer Pebesma

arXiv AI

Aug 25

On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

arXiv:2608.22089v1 Announce Type: cross Abstract: Modern software systems require earlier and more scalable vulnerability severity assessment to reduce exposure to high-impact security flaws. Securit...

By Daniel Rodriguez-Cardenas, David Nader Palacio, Anna Schmedding, Yiyang Lu, Aadil Mallick, Bill Hudson, Chris Gourley, Michael Roytman, Chris Shenefiel, Evgenia Smirni, Denys Poshyvanyk

llms safety

arXiv Computation and Language

Sep 3

LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization

The paper presents a systematic framework for large language model (LLM) watermarking as a provenance tool in big data ecosystems. It categorizes existing watermarking methods along four deployment dimensions—insertion point, verification authority, operational state, and transformation threat model—and aligns them with the big data principles of Volume, Velocity, Variety, Veracity, and Value. The authors introduce a readiness framework that maps four key workloads—online generation, streaming detection, transformation pipelines, and ecosystem governance—to system-level requirements such as throughput, false-positive control, robustness, cross-domain reliability, governance, and downstream utility, while highlighting gaps between benchmark performance and real-world deployment readiness.

By Huy Phan, Kieu Dang, Ojaswi Dulal, Aiham AL Shukairi, Abby Shine, Chase Garner, Phung Lai

llms rag efficiency benchmarks

arXiv AI

Jun 9

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

arXiv:2606. 07524v1 Announce Type: cross Abstract: The explosive growth of large language models (LLMs) has created a heterogeneous and poorly documented ecosystem, making systematic model comparison increasingly important for provenance auditing, security analysis, and model selection.

By Zirui Wang, Yusen Hou, Shaofeng Liang, Bowen Tian, Yanlin Zhang, Wenshuo Chen, Yutao Yue

llms rag nlp benchmarks safety

arXiv AI

Aug 25

LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...

By Xiaogang Xu, Jiaqi Tang, Jianmin Chen, Yingying Yan, Zhenchao Tang, Xiangxin Zhou, Xiaobin Hu, Wei Wei, Jinfeng Wu, Qifeng Chen, Lu Zhou, Jiafei Wu, Zhe Liu, Jianwei Yin, Weimin Zheng

llms rag agents benchmarks

arXiv Machine Learning

Jun 4

It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks

arXiv:2602. 12147v4 Announce Type: replace Abstract: Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.

By Zhongzheng Qiao, Sheng Pan, Anni Wang, Viktoriya Zhukova, Yong Liu, Xudong Jiang, Qingsong Wen, Mingsheng Long, Ming Jin, Chenghao Liu

llms benchmarks

One email a morning, machine-written

Related stories

A Machine Learning API for Earth Observation Data Cubes Based on openEO

On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

LLM Watermarking as Big Data Provenance: A Deployment-Oriented Systematization

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks