Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

17,750 stories · RSS feed

arXiv AI
Jul 7

S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval

arXiv:2607. 02689v1 Announce Type: cross Abstract: As wearable devices enable continuous first-person recording, AI assistants must reason across long time horizons to recall past experiences-a capability known as episodic memory.

By Xiaodong Wang, Xuanyi Zhao, Pedro Rodriguez, Devendra Singh Sachan, Barlas Oguz, Seungwhan Moon, Shang-Wen Li, Gargi Ghosh, Xin Dong, Wen-Tau Yih
arXiv Machine Learning
Jul 7

Social Networks of LLM Agents

arXiv:2607. 03695v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed in interacting populations, raising the question of what such populations come to believe collectively.

By Kaixuan Liu, Guojun Xiong, Weinan Zhang, Shengpu Tang
arXiv Machine Learning
Jul 7

Granger Causality in Extremes

arXiv:2407. 09632v3 Announce Type: replace-cross Abstract: We introduce a rigorous mathematical framework for Granger causality in extremes, designed to identify causal links from extreme events in time series.

By Juraj Bodik, Olivier C. Pasche
arXiv AI
Jul 7

STELLA: Efficient Sensor-to-LLM Translation for On-Device Human Activity Recognition

arXiv:2607. 03089v1 Announce Type: cross Abstract: HAR is increasingly expected to run continuously on edge devices, yet recent LLM-based methods remain hard to deploy: raw sensor prompts are long, cloud inference adds latency and privacy risk, and fine-tuned LLM pipelines turn general-purpose models into task-specific classifiers.

By Nirhoshan Sivaroopan, Albert Zomaya, Kanchana Thilakarathna
arXiv Machine Learning
Jul 7

Adversarial LassoNet: Robust Feature Selection via Stability-Driven Sparse Learning

arXiv:2607. 03839v1 Announce Type: new Abstract: Sparse feature selection is critical for high-dimensional machine learning, yet traditional $\ell_1$-regularized methods are often brittle under observational noise and spurious correlations, leading to unstable feature supports and degraded generalization.

By Zhen Huang, Peicheng Xu, Junbiao Pang, Yulong Zheng
arXiv Machine Learning
Jul 7

Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQL

arXiv:2607. 03991v1 Announce Type: new Abstract: Repeated LLM calls are the standard way to estimate how trustworthy a Text-to-SQL result is: run the pipeline multiple times, judge each SQL execution, and use the consistency of the verdicts as a confidence signal.

By Yaron Anavi, Mor Aisenberg, Nadav Nesher, Elena Khabibullina, Isabella Cattinelli
arXiv Machine Learning
Jul 7

Functional Bilevel Optimization for Predictive Fairness

arXiv:2607. 05098v1 Announce Type: new Abstract: When sensitive attributes are continuous and high-dimensional $-$ demographic score vectors, posteriors over attributes, age or income profiles $-$ enforcing full statistical independence is often too restrictive, and existing relaxations rely on indirect dependence penalties or adversarial schemes that do not directly target the fairness-accuracy trade-off.

By Ieva Petrulionyte, Julien Mairal, Michael Arbel