ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model

Supercharged Customer Service with Machine Learning

CEO-Bench: Can Agents Play the Long Game?

arXiv:2606. 18543v1 Announce Type: new Abstract: Language model agents are becoming proficient executors at isolated, short-horizon tasks such as software engineering and customer service.

SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking

arXiv:2607. 20655v1 Announce Type: cross Abstract: Lead ranking in Customer Relationship Management (CRM) systems faces a persistent challenge: models achieving high offline accuracy often underperform in production.

Introducing the Open Chain of Thought Leaderboard

HiMA-Ecom: A Long-Horizon Memory Benchmark for Hierarchical Multi-Agent E-commerce Assistants

The paper introduces HiMA-Ecom, a hierarchical multi‑agent e‑commerce benchmark with 22.8K instances, where 17.7K include memory entries. It explores training, evaluation, and management of accumulated context in long‑horizon agentic systems, and proposes HiMA‑R1, a policy optimization method that controls trajectory context budgets and uses reward‑driven memory to accelerate convergence. Experiments show that agents built on 3B and 7B open‑source models outperform DeepSeek‑V3 by an average of 6.0 points, while reward‑driven memory cuts convergence steps by about 55%.

MemRerank: Preference Memory for Personalized Product Reranking

arXiv:2603. 29247v3 Announce Type: replace-cross Abstract: LLM-based shopping agents increasingly rely on long purchase histories and multi-turn interactions for personalization, yet naively appending raw history to prompts is often ineffective due to noise, length, and relevance mismatch.

Introducing the Enterprise Scenarios Leaderboard: a Leaderboard for Real World Use Cases

Introducing the Open FinLLM Leaderboard

Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration

arXiv:2607. 16848v1 Announce Type: cross Abstract: Long-term memory is becoming a core component of LLM agents, but most memory benchmarks evaluate conversations or compact summaries, while research agents need to restore evidence from full scientific papers.

Sell More, Play Less: Benchmarking LLM Realistic Selling Skill

The paper introduces SalesLLM, a bilingual (Chinese/English) benchmark for evaluating large language models (LLMs) in realistic sales dialogues. It comprises 30,074 scripted configurations and 1,805 curated multi‑turn scenarios from Financial Services and Consumer Goods, with controllable difficulty and personas. An automatic evaluation pipeline uses an LLM judge for sales‑process progress and fine‑tuned BERT classifiers for end‑of‑dialogue buying intent, while a user model, CustomerLM, is trained to improve simulation fidelity. SalesLLM scores correlate strongly with human ratings (Pearson r = 0.86) and reveal that top Chinese LLMs match junior‑to‑intermediate human salespeople but not experts, with cross‑lingual consistency remaining poor.

Evaluating Memory Structure in LLM Agents

The paper introduces StructMemEval, a benchmark designed to assess how well large language model (LLM) agents can organize their long‑term memory rather than merely recall facts. It compiles tasks that humans typically solve by structuring knowledge—such as transaction ledgers, to‑do lists, and trees—and evaluates agents on these. Experiments show that simple retrieval‑augmented LLMs struggle with such organization tasks, while memory‑augmented agents perform better when explicitly prompted to structure their memory, yet many modern LLMs still fail to recognize memory structures without prompting.

DolphinBench: Mapping the Pareto Frontier of Agent Memory

arXiv:2609.24971v1 Announce Type: new Abstract: Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are bu...