FiMI Banking presents a controlled Indian retail‑banking environment designed to train conversational agents that can answer product questions, handle account requests, and comply with strict operational and regulatory constraints. The system is built from vetted banking documents, structured ground truth, synthetic customer backgrounds, and banking tools. Evaluation of two post‑training methods—preference optimization and reinforcement learning with verifiable rewards—shows that preference optimization significantly increases safe behavior (out‑of‑scope refusal from 52% to 80%), while reinforcement learning improves edge‑case and order‑sensitive task performance and reduces token usage.
TauIndianBankBench is a new benchmark that evaluates conversational agents on 1,000 Indian retail‑banking tasks by checking not only the final records but also whether every required procedural step was performed and no unrequested changes were made. It highlights that models can appear successful when judged only on final records, yet may skip necessary steps or alter records incorrectly. The benchmark provides a more accurate assessment of language model performance in the Indian banking domain.
By Krishanu Adhikary, Prashant Devadiga
IndicBankBench is a 799‑case benchmark designed to evaluate the safety and reliability of language model assistants in Indian retail banking. It covers five operational domains, a capability/refusal domain, and twenty primary axes, assessing each case at four stages: safety, action and tool use, response adequacy, and advisory quality. The benchmark uses deterministic safety checks, a narrow resolver for ambiguous confirmation‑before‑write scenarios, and an LLM judge for semantic response adequacy, reporting strict pass rates that reveal a gap between strict reliability (43.7%–58.2%) and at‑least‑once success (60%–74%).
By Suvradip Paul, Chandra Bhushan, Harsh Sharma, Nitin Kukreja, Yatharth Dedhia, Keyur Doshi, Prashant Devadiga
Dialogue systems in e-commerce scenarios often need to satisfy multiple objectives: accurately reasoning over user profiles (e. g.
RL-ADA introduces a co‑evolutionary training framework that replaces costly human annotations with world‑feedback rewards derived from interaction outcomes. In this system, a large Customer Support Agent and an Adversarial Customer Agent train together, guided by an automated judge that rewards successful resolution and realistic intent‑concealing utterances, respectively. Applied to a banking support proof of concept, the method eliminates routing errors and doubles the end‑to‑end PASS rate over five cycles, while also revealing a new adversarial strategy called Contextual Camouflage.
By Ram Narayanan, Harshit Rajgarhia, Abhishek Mukherji
ProfileSim is a profile‑driven user simulation framework designed for financial marketing outbound calls. It jointly models user demographics, Big Five personality traits, and business intents reverse‑engineered from real call logs, allowing fine‑grained control over simulated behavior. The system is fine‑tuned on real voice dialogue data and further refined with Group Relative Policy Optimization using hierarchical rewards, achieving high colloquial naturalness, strong user profile adherence, and low agent scores on long‑tail intents while producing shorter user utterances.
By Juan Li, Fangshu Chang, Xinyi Jiang, Ling Huang, Yunzhang Chen