arXiv Computation and Language
BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
Hugging Face Trending Papers
Sep 1
WorldBench: Culturally Grounded Benchmark for Multilingual Agents
Despite the growing use of LLM-powered agents to solve multi-step tasks in complex environments, existing benchmarks rarely test state preservation, performance across languages, and application to re...
arXiv AI
Sep 2
WorldBench: Culturally Grounded Benchmark for Multilingual Agents
WorldBench is a new multilingual benchmark that tests large language model agents on culturally grounded everyday workflows, offering 1,600 tasks in seven languages and eight cultures. The benchmark evaluates agents through structured sandbox actions and introduces Constrained Task Success (CTS), a metric that assesses task completion, minimal modification, and other complementary aspects via deterministic and LLM-as-a-Judge evaluations. Experiments show that even leading models achieve only 49.2% CTS, revealing significant gaps in correctness and state preservation across languages and cultures.
arXiv AI
Aug 18
PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows
arXiv:2607. 06008v3 Announce Type: replace Abstract: While Large Language Model (LLM) agents excel at monolingual long-horizon planning and tool use, enterprise workflows inherently require processing multilingual resources across extended trajectories.
arXiv AI
Sep 15
Translating the Translator: Decomposing the Cost of English-Forced Inter-Agent Communication
arXiv:2609.15079v1 Announce Type: cross Abstract: Multi-agent LLM architectures, such as LangChain and AutoGen, largely assume English as the lingua franca for internal inter-agent communication, eve...
Hugging Face Trending Papers
Aug 9
OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents
Agentic benchmarks aim to measure how well AI agents plan, search, execute, and recover within realistic multi-tool environments, but they are almost exclusively in English. As AI agents are globally deployed to a linguistically diverse user base, whether agentic competence measured in English transfers to other languages remains an open question.
arXiv AI
Jul 8
PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents
arXiv:2607. 06008v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments.