arXiv AI
FinLifeBench: Exhaustive Life-Event History and Financial-State Reconstruction from Longitudinal Banking Dialogue
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 6
FinPerMA: A Theory-Informed, Event-Grounded Personalized-Memory Benchmark for LLM Agents
arXiv:2608. 04095v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly used as personalized assistants in high-stakes domains such as financial advising, yet it remains unclear whether they can maintain and update an individualized user model over long horizons.
arXiv AI
Sep 16
LSREP: A Longitudinal State-Replay Protocol for Evaluating Conversational Memory, with ICE v2 as an Audited Local-First Architecture
The paper introduces LSREP, a Longitudinal State‑Replay Evaluation Protocol designed to assess how conversational memory evolves over time, incorporating ordered replay, lifecycle schedules, repeated probes, evolving reference answers, and mechanism‑fidelity checks. It applies LSREP to ICE v2, a local‑first memory middleware, and reports that on three ordinary‑density datasets ICE v2 achieves near‑zero mean quality difference from vector‑RAG while using fewer fragments but slightly more prompt tokens, yet fails catastrophically on a dense dataset. In a public diagnostic, ICE v2 underperforms pure vector‑RAG on LongMemEval, revealing significant multi‑session and temporal failures and a quality‑cost trade‑off rather than superior efficiency.
arXiv AI
Jul 15
MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
arXiv:2607. 12893v1 Announce Type: new Abstract: Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions.
arXiv Machine Learning
Jul 22
HindsightBench: A Black-Box Behavioral Audit Protocol for Parametric Hindsight in Time-Indexed LLM Decision Tasks
arXiv:2607. 18867v1 Announce Type: new Abstract: Large language models leak parametric knowledge of realized outcomes into historical financial decision tasks.
arXiv AI
Sep 30
Memory Is a Derivation: The Distributed-Evidence Paradox in Long-Term Agents
The paper introduces the Distributed‑Evidence Paradox, where long‑running LLM agents compress past interactions into persistent memories that may not be fully supported by the interaction history. It defines three key requirements—evidence scope, compositional validity, and admission reliability—and proposes DerivAudit, a framework that checks whether a memory is truly supported by the available history. Experiments on two memory corpora show that expanding the evidence base can recover support for many memories, yet many remain unsupported, and broader evidence alone does not guarantee reliable admission.
arXiv AI
Aug 5
LeanMem: Simple and Efficient Long-Term Memory for LLM Agents
arXiv:2608. 03463v1 Announce Type: new Abstract: Long-term memory is essential for LLM-based agents to sustain interactions and reliably leverage distant history.