{\tau}-bench-LILT: Evaluating Conversational Agents on Realistic Multilingual Tool-Use Tasks

The paper introduces ${ au}$-bench-LILT, a multilingual extension of the ${ au^3}$-bench benchmark that evaluates conversational agents on realistic tool‑use tasks in German, Korean, Turkish, and Farsi across airline, retail, telecom, and banking domains. It localizes every layer of the benchmark, including culturally appropriate customer databases, native‑language user‑simulator prompts refined by contrastive annotation, and tasks tailored to realistic regional scenarios. Evaluation of eight agent models reveals the most significant performance drops in the Korean and Farsi retail domains, with reductions of up to 61 points in pass^4.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• agents
• benchmarks