arXiv:2609.23742v1 Announce Type: new Abstract: Small open-source large language models (LLMs) in the 0.6B-4B parameter range are increasingly deployed for structured output generation (JSON, functio...
arXiv:2609.40322v1 Announce Type: cross Abstract: Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented lang...
arXiv:2608.21074v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to in...
The paper introduces ELCD, a latent conflict detector that verifies LLM outputs after generation to catch instruction conflicts that static input checks miss. ELCD builds a hidden-state representation from the final-token embedding and the mean-pooled response embedding, then trains a pairwise margin ranking objective to distinguish compliant from drifting responses. Experiments on five large language models show ELCD outperforms baselines, boosting PR-AUC for Llama‑2‑7B by ~30 percentage points and cutting FPR95 for Mistral‑7B to 2.67%.
arXiv:2608. 07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatched environments, broken imports, or package conflicts.
The paper presents a benchmark that compares seven long‑form generation frameworks across three granularities—single chapter, multi‑chapter, and whole book—using an anchor‑based LLM‑as‑a‑judge protocol to evaluate outlines directly. Results show no single framework dominates across all settings; performance depends on how well a framework’s output form matches the target granularity, with SuperWriter excelling in length‑constrained single‑chapter mode but losing advantage in whole‑book mode. The study finds only moderate correlation between outline and writing quality, supporting the idea that these two stages should be evaluated separately.
The paper introduces Brittlebench, a framework and evaluation pipeline that measures how sensitive large language models (LLMs) are to prompt variations. By applying semantics‑preserving perturbations to popular benchmarks, the authors show that model performance can drop by up to 12% and that such perturbations can change model rankings in 63% of cases. The study finds that these input variations can explain up to half of the performance variance for a given model, underscoring the importance of robust evaluation methods.
ToolRobustBench is a stage-wise diagnostic benchmark designed to evaluate and diagnose failures in tool‑calling agents, which are large language models that select tools, provide structured arguments, and interpret tool feedback. The benchmark aligns four perturbation families—tool‑interface, user‑intent, tool‑output/observation, and runtime‑environment—with the tool‑use pipeline, attributing failures to specific stages such as tool selection, schema grounding, argument binding, and feedback handling. Experiments across 15,456 instances, 7 models, and 16 local tools reveal that while overall performance is high, robustness degrades significantly, especially under tool‑output/observation perturbations, and mixed‑family perturbations produce non‑additive failure patterns.
A long-form translation request can succeed at the API layer and still produce an unusable result. The output may be empty, truncated, filtered, dominated by source or prompt material, or interrupted after producing text worth keeping.
arXiv:2601. 04498v2 Announce Type: replace Abstract: Infographics are composite visual artifacts that combine data visualizations with textual and illustrative elements to communicate information.
arXiv:2608. 13547v1 Announce Type: new Abstract: LLM coding agents issue Bash commands through interfaces that may serialize, wrap, and reparse model output.
arXiv:2610.35179v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for codebook-based content analysis in computational social science, where codebooks translate theor...