The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
LifeVal is a new benchmark that evaluates AI model performance on personalized agentic consumer tasks, covering over 1.7 trillion US dollars in annual willingness-to-pay across 148 task categories such as managing finances and finding childcare. The benchmark was built using a survey of more than 3,800 consumers and cross‑referenced with national time‑use studies. Task creators, who met strict expertise thresholds, designed AI evaluation tasks personalized with real ChatGPT conversations, memories, reference files, and personal context, and all tasks underwent at least three rounds of human quality control before inclusion.
Large language model (LLM) agents are increasingly deployed as personal assistants. Existing evaluations, however, mostly use short, self-contained requests in static environments.
arXiv:2608. 10875v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed as personal assistants.
The paper presents a systematic study of token consumption in AI agents performing coding tasks. It finds that agentic tasks are far more expensive—about 1000 times more tokens than code reasoning or chat—primarily due to input tokens, and that token usage varies wildly, with accuracy peaking at moderate costs. Models differ significantly in efficiency, and current frontier models cannot reliably predict their own token usage, often underestimating it.
The paper introduces Active Preference Acquisition (APA) for long‑term personalized web agents, defining the Prospective Personalization Value (PPV) of a missing preference as its expected cumulative utility over future tasks minus the cost of asking. APA selects questions within a per‑interaction budget, prioritizing preferences that may not benefit the current task but can be reused across many future tasks. Experiments on the ActiveWebPref benchmark show APA acquiring 80.3% of amortization‑dependent preferences and achieving higher net utility than baseline methods, while limiting the number of questions asked.
The paper introduces Active Preference Acquisition (APA), a method for web agents to decide which unknown user preferences to ask about by evaluating the Prospective Personalization Value (PPV)—the expected cumulative benefit across future tasks minus the cost of asking. APA calculates counterfactual gains for each candidate preference using task prototypes and aggregates them by expected reuse, enabling agents to prioritize questions that may not help immediately but improve long‑term performance. Experiments on the ActiveWebPref benchmark show APA can acquire a large share of valuable preferences and achieve higher net utility than baseline strategies, demonstrating that considering future reuse is crucial for effective personalization.