arXiv Computation and Language
KiranaBench: Can a Language Model Run a Neighbourhood Shop?
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
1d ago
{\tau}-bench-LILT: Evaluating Conversational Agents on Realistic Multilingual Tool-Use Tasks
The paper introduces ${ au}$-bench-LILT, a multilingual extension of the ${ au^3}$-bench benchmark that evaluates conversational agents on realistic tool‑use tasks in German, Korean, Turkish, and Farsi across airline, retail, telecom, and banking domains. It localizes every layer of the benchmark, including culturally appropriate customer databases, native‑language user‑simulator prompts refined by contrastive annotation, and tasks tailored to realistic regional scenarios. Evaluation of eight agent models reveals the most significant performance drops in the Korean and Farsi retail domains, with reductions of up to 61 points in pass^4.
arXiv Computation and Language
Aug 27
Sell More, Play Less: Benchmarking LLM Realistic Selling Skill
The paper introduces SalesLLM, a bilingual (Chinese/English) benchmark for evaluating large language models (LLMs) in realistic sales dialogues. It comprises 30,074 scripted configurations and 1,805 curated multi‑turn scenarios from Financial Services and Consumer Goods, with controllable difficulty and personas. An automatic evaluation pipeline uses an LLM judge for sales‑process progress and fine‑tuned BERT classifiers for end‑of‑dialogue buying intent, while a user model, CustomerLM, is trained to improve simulation fidelity. SalesLLM scores correlate strongly with human ratings (Pearson r = 0.86) and reveal that top Chinese LLMs match junior‑to‑intermediate human salespeople but not experts, with cross‑lingual consistency remaining poor.
arXiv AI
Sep 15
Translating the Translator: Decomposing the Cost of English-Forced Inter-Agent Communication
arXiv:2609.15079v1 Announce Type: cross Abstract: Multi-agent LLM architectures, such as LangChain and AutoGen, largely assume English as the lingua franca for internal inter-agent communication, eve...
arXiv AI
Oct 21
LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents
arXiv:2610.27205v1 Announce Type: cross Abstract: Web agents are typically evaluated on English-language interfaces, leaving their ability to operate across the world's multilingual websites largely...
arXiv AI
Jun 16
RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environments
arXiv:2606. 15862v1 Announce Type: new Abstract: Large language model (LLM) agents have made rapid progress on short-horizon, well-scoped tasks, yet their ability to sustain coherent decisions in dynamic long-horizon environments remains uncertain.
arXiv Computation and Language
Sep 18
Compass-v3: Scaling Domain-Specific LLMs for Multilingual E-Commerce in Southeast Asia
Compass‑v3 is a 245B‑parameter Mixture‑of‑Experts language model tailored for Southeast Asian e‑commerce, featuring 71B active parameters per token and hardware‑efficient expert parallelism. It is trained on 12 trillion multilingual tokens and synthetic e‑commerce instructions, and incorporates Optimal‑Transport Direct Preference Optimization to improve instruction adherence. Benchmarks show it outperforms GPT‑4, DeepSeek‑V3.1, and Qwen3‑235B, and it is already deployed at scale on Shopee, handling over 70% of the platform’s LLM traffic.