arXiv AI
Jun 3
Cross-Lingual Token Arbitrage: Optimizing Code Agent Context Windows via Local LLM Preprocessing
arXiv:2606. 03618v1 Announce Type: new Abstract: AI-assisted coding agents are bottlenecked by input-token cost.
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
arXiv:2601. 05366v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls.
Related stories
arXiv AI
1d ago
PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows
arXiv:2607. 06008v3 Announce Type: replace Abstract: While Large Language Model (LLM) agents excel at monolingual long-horizon planning and tool use, enterprise workflows inherently require processing multilingual resources across extended trajectories.
arXiv AI
Jun 9
Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs
arXiv:2606. 08840v1 Announce Type: new Abstract: Code generation models are typically compared using compact execution benchmarks and aggregate pass rates, but such summaries obscure how performance varies across programming languages, problem families, and failure modes.
Hugging Face Trending Papers
Jul 16
Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality
Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and programming conventions remains largely unexplored.
arXiv AI
Jul 17
Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality
arXiv:2607. 14816v1 Announce Type: cross Abstract: Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias.
arXiv AI
Jul 8
PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents
arXiv:2607. 06008v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external environments.
arXiv AI
Jul 1
Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions
arXiv:2606. 30790v1 Announce Type: cross Abstract: Romanized Code Mixing (RCM), where bilingual speakers fluidly blend local languages with English in Roman script, has emerged as the dominant form of communication across multilingual communities.
arXiv AI
6d ago
When the API Speaks the Wrong Language: Revisiting Post-Training for Multilingual Tool Use
arXiv:2608. 11715v1 Announce Type: cross Abstract: The reliability of Large Language Models (LLMs) for API calling degrades in multilingual settings.
arXiv AI
Jul 21
JOR-Bench: Japanese Operations Research Benchmarks for Large Language Models
arXiv:2607. 16777v1 Announce Type: cross Abstract: We present JOR-Bench, a collection of five Japanese-language benchmarks for evaluating the ability of large language models (LLMs) to formulate and solve operations research (OR) problems.
Hugging Face Trending Papers
Aug 9
OmnilingualGAIA2: Evaluating the Multilingual Gap in Frontier AI Agents
Agentic benchmarks aim to measure how well AI agents plan, search, execute, and recover within realistic multi-tool environments, but they are almost exclusively in English. As AI agents are globally deployed to a linguistically diverse user base, whether agentic competence measured in English transfers to other languages remains an open question.
arXiv AI
Jun 30
SEATauBench: Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages
arXiv:2606. 28715v1 Announce Type: cross Abstract: While AI development and evaluation for Southeast Asia (SEA) has grown rapidly, agent capabilities in regional languages are still poorly understood despite its importance to sovereign AI.
arXiv AI
Aug 11
PluginEval: A Diagnostic Benchmark for Fine-Grained Error Attribution in Function Calling
arXiv:2608. 08700v1 Announce Type: new Abstract: Reliable evaluation of tool routing is critical as Large Language Models increasingly operate as autonomous agents.