The paper investigates how a large language model can act as a corpus debugger for the Slovak portion of the multilingual FineWeb2 dataset. By using a tool‑using agent that explores the corpus via a read‑only, budgeted API, the authors identified seven issues across four phenomena, including a large machine‑generated content farm that slipped through the pipeline’s filters. The agent’s compiled test suite can scan the entire 26.5 M‑document corpus in 0.7 s without a GPU, providing precise prevalence estimates and actionable insights into pipeline failures.
arXiv:2604. 16706v2 Announce Type: replace Abstract: Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, yet this assumption is rarely validated against human annotation.
By Bhaskar Gurram
arXiv:2607. 06411v1 Announce Type: cross Abstract: Developers increasingly delegate real maintenance work to product-grade coding agents, and many state tasks in their native language, in the style of a customer request rather than a curated English issue.
By Evgeny Shilov (Independent Researcher)
When a tool-using agent is given the same task in a different language, does it still take the same steps? Multilingual evaluation rarely asks: it compares final answers and discards the actions.
The paper introduces BRIA (BRanch-Inspect-Adopt), an agent that isolates tool‑candidate investigations into separate branches, executing each candidate independently and then inspecting the execution evidence to adopt only useful results. BRIA dynamically retrieves additional tools as new needs arise during task execution and is evaluated on PlanBench‑XL and MCP‑Atlas with a library of 1,897 tools, outperforming or matching the strongest retrieval baselines. On MCP‑Atlas, BRIA achieves an average claim score of 89.77 versus 75.40 for MCP‑Zero, and it lies on the Pareto frontier for task performance and input‑token cost.
By Anna Mazhar, Seiji Maekawa, Nikita Bhutani
arXiv:2610.27205v1 Announce Type: cross
Abstract: Web agents are typically evaluated on English-language interfaces, leaving their ability to operate across the world's multilingual websites largely...
By Ocean Monjur, Daud Ahmad Nisar, Shahriar Kabir Nahin, Tadesse Destaw Belay, Sushruti Mishra, Israel Abebe Azime, Anshuman Chhabra