The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
arXiv:2610.19291v1 Announce Type: new Abstract: In so-called "Industry 4.0," integrated sensors, analytics tools, and other digital technologies supplement industrial workflows, potentiating enhanced...
arXiv:2606. 17904v1 Announce Type: new Abstract: Language models increasingly serve as advisory systems in maintenance operations.
DiagnosticIQ is a benchmark that tests large language models (LLMs) on translating engineer‑written symbolic rules into maintenance actions for industrial assets. It contains 6,690 multiple‑choice questions derived from 118 expert rule‑action pairs across 16 asset types, and includes five variants that probe different failure modes. The benchmark evaluates 29 LLMs and four embedding baselines, showing that while top models perform near the frontier, they exhibit brittleness under distractor expansion and structural perturbations.
ToolRobustBench is a stage-wise diagnostic benchmark designed to evaluate and diagnose failures in tool‑calling agents, which are large language models that select tools, provide structured arguments, and interpret tool feedback. The benchmark aligns four perturbation families—tool‑interface, user‑intent, tool‑output/observation, and runtime‑environment—with the tool‑use pipeline, attributing failures to specific stages such as tool selection, schema grounding, argument binding, and feedback handling. Experiments across 15,456 instances, 7 models, and 16 local tools reveal that while overall performance is high, robustness degrades significantly, especially under tool‑output/observation perturbations, and mixed‑family perturbations produce non‑additive failure patterns.
arXiv:2608.30685v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed in user-facing services that require iterative tool use under dynamic business conditions....
arXiv:2508. 00923v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against.