arXiv Machine Learning
LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
1d ago
Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap
arXiv:2609.23742v1 Announce Type: new Abstract: Small open-source large language models (LLMs) in the 0.6B-4B parameter range are increasingly deployed for structured output generation (JSON, functio...
arXiv AI
Aug 24
PromptResponse: Optimizing Prompts for LLM Coding Tasks
arXiv:2608.21074v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to in...
arXiv AI
Aug 11
TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair
arXiv:2608. 07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatched environments, broken imports, or package conflicts.
arXiv AI
Aug 28
A Multi-Framework Comparison of Outline Stages in Long-Form Generation with LLMs
The paper presents a benchmark that compares seven long‑form generation frameworks across three granularities—single chapter, multi‑chapter, and whole book—using an anchor‑based LLM‑as‑a‑judge protocol to evaluate outlines directly. Results show no single framework dominates across all settings; performance depends on how well a framework’s output form matches the target granularity, with SuperWriter excelling in length‑constrained single‑chapter mode but losing advantage in whole‑book mode. The study finds only moderate correlation between outline and writing quality, supporting the idea that these two stages should be evaluated separately.
arXiv AI
Aug 26
ToolRobustBench: Stage-Wise Perturbation Evaluation and Failure Diagnosis for Tool-Calling Agents
ToolRobustBench is a stage-wise diagnostic benchmark designed to evaluate and diagnose failures in tool‑calling agents, which are large language models that select tools, provide structured arguments, and interpret tool feedback. The benchmark aligns four perturbation families—tool‑interface, user‑intent, tool‑output/observation, and runtime‑environment—with the tool‑use pipeline, attributing failures to specific stages such as tool selection, schema grounding, argument binding, and feedback handling. Experiments across 15,456 instances, 7 models, and 16 local tools reveal that while overall performance is high, robustness degrades significantly, especially under tool‑output/observation perturbations, and mixed‑family perturbations produce non‑additive failure patterns.
Hugging Face Trending Papers
Aug 10
Failure-Aware Long-Form Translation: Design and Implementation of a Recoverable LLM Translation System
A long-form translation request can succeed at the API layer and still produce an unusable result. The output may be empty, truncated, filtered, dominated by source or prompt material, or interrupted after producing text worth keeping.