Hugging Face Trending Papers
kw-bench: Breaking Frontier Harnesses on Real-World Tasks
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Oct 28
kw-bench: Breaking Frontier Harnesses on Real-World Tasks
The paper introduces kw-bench, a benchmark that evaluates coding-agent harnesses on investigative knowledge work, specifically a five‑year VAT reconstruction task across multiple SAP systems. It contrasts generative benchmarks with this real‑world, evidence‑heavy task, showing that model‑agnostic harnesses performed well while lab harnesses failed. The study demonstrates how frontier models can dramatically reduce time and cost for complex audit‑style engagements.
arXiv AI
Aug 6
ORCA-bench: How Ready Are Language Model Agents for Oncall?
arXiv:2607. 28545v2 Announce Type: replace-cross Abstract: Large language models can write, patch, and search code, but oncall root cause analysis (RCA) demands something different: reasoning over noisy metrics, logs, traces, and source code, starting from ambiguous user-facing reports, often hours after the incident began.
arXiv Machine Learning
Aug 18
The Working Set of a Coding Agent: Coherence Debt in Repository-Scale Tasks
arXiv:2608. 16630v1 Announce Type: cross Abstract: Repository-scale coding requires an agent to keep tests, imports, configuration, and migration rules consistent within a bounded context window.
arXiv AI
Oct 2
K-Dense BYOK: An Open-Source AI Research Assistant That Runs Locally and Keeps a Hash-Chained Lab Notebook
K‑Dense BYOK is a free, open‑source AI research assistant that runs locally on a researcher’s own computer. It provides a structured environment with scientific procedures, workflow templates, and a living lab notebook that logs all actions without allowing the agent to alter the record. The system emphasizes reproducibility by recording the software environment and offering commands to regenerate results, outperforming managed platforms on interdisciplinary research prompts.
arXiv AI
23h ago
The Last Mile to the Real World: Agent-Native Primitives as Reusable Building Blocks for Solving Real-World Tasks
arXiv:2611.07461v1 Announce Type: cross Abstract: How can an agent handle a real-world task end to end? Frontier models with their own harnesses still fall short on real-world benchmarks: on Terminal...
arXiv AI
Jun 4
Parthenon Law: A Self-Evolving Legal-Agent Framework
arXiv:2606. 04602v1 Announce Type: new Abstract: As agents grow more capable, legal-domain LLM agents promise to turn document-heavy matters into reviewable work products -- yet reliable deployment faces three obstacles: no large-scale evidence on how today's strongest model-and-harness combinations behave on end-to-end legal matters; no agent architecture adapted to the legal vertical, only general-purpose harnesses; and, in a setting that keeps shifting with new facts, authorities, and deadlines, no mechanism for systems to learn from their own outcomes.