Hugging Face Blog
Nov 19, 2024
LAVE: Zero-shot VQA Evaluation on Docmatix with LLMs - Do We Still Need Fine-Tuning?
Related stories
arXiv AI
Aug 5
MMLongBench-Doc-V2: A Corrected-Annotation, Semantics-Aware Revision of MMLongBench-Doc
arXiv:2608. 03397v1 Announce Type: new Abstract: MMLongBench-Doc is a long-document QA benchmark of 1,082 questions over 135 PDFs.
arXiv AI
Jun 30
LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents
arXiv:2606. 29399v1 Announce Type: new Abstract: Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters.
arXiv AI
Jul 10
When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability
arXiv:2607. 08535v1 Announce Type: cross Abstract: An LLM-as-judge score can move even when the candidate responses stay fixed, simply because the evaluator has changed.
arXiv AI
Jun 19
VERITAS: Verifier-Guided Proof Search for Zero-Shot Formal Theorem Proving
arXiv:2606. 19399v1 Announce Type: cross Abstract: LLM-based formal provers often collapse rich verifier signals (syntax errors, type mismatches, partial goal progress) into a binary pass/fail bit.
arXiv AI
Jun 6
Evaluation of LLMs for Mathematical Formalization in Lean
arXiv:2606. 05632v1 Announce Type: new Abstract: Within the past few years, the ability of Large Language Models (LLMs) to generate formal mathematical proofs has improved drastically.
arXiv AI
Jun 9
Decision-Aware Memory Cards: Counterfactual-Inspired Context Selection and Compression for Tool-Using LLM Agents
arXiv:2606. 08151v1 Announce Type: new Abstract: Tool-using LLM agents often fail not because relevant text is absent, but because decisive evidence is not selected, compressed, or surfaced at action time.
arXiv AI
Jun 29
Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA
arXiv:2606. 28050v1 Announce Type: cross Abstract: LLM-as-a-Judge and self-evaluation pipelines implicitly assume that evaluation is easier than generation.
arXiv AI
Jun 11
PoQ-Judge: A Multi-Architecture Evaluation Framework for Cost-Aware Proof-of-Quality in Decentralized LLM Inference
arXiv:2606. 11196v1 Announce Type: cross Abstract: Decentralized LLM inference networks need lightweight, reference-free quality evaluation for Proof of Quality (PoQ).
arXiv Machine Learning
Jun 19
Probe-and-Refine Tuning of Repository Guidance for Coding Agents
arXiv:2606. 20512v1 Announce Type: cross Abstract: LLM-based coding agents need higher-level operational knowledge about a repository (which files house which subsystems, how to run the test suite, which workflows have historically led to wrong fixes) that does not exist in the code itself.
arXiv AI
Jul 13
Test-Time Scaling for Small VLMs on Multilingual Visual MCQ
arXiv:2607. 09438v1 Announce Type: cross Abstract: Test-time scaling (TTS) reliably improves reasoning in large language models, but whether it transfers to small open vision-language models remains unclear.
arXiv AI
Jul 22
HALLMARK: Diagnosing Three Failure Modes in LLM Citation Verifiers
arXiv:2607. 18360v1 Announce Type: cross Abstract: Large language models (LLMs) now routinely draft literature reviews and assist with academic writing, which means a higher risk of fabricated references: GPTZero found 53 papers with hallucinated citations among NeurIPS 2025's accepted set.