arXiv:2608. 07508v1 Announce Type: cross Abstract: Large language models are already advisors to millions of people of faith who bring them real decisions.
arXiv:2609.13634v1 Announce Type: cross Abstract: Do AI assistants help believers reason about moral dilemmas consistently with their faith? We present FaithfulBench, the first benchmark to score AI...
Ansari is a retrieval‑grounded Islamic AI assistant that has handled over 140,000 conversations in more than 25 languages since June 2023. It uses an agentic retrieval loop where a language model searches authenticated Islamic corpora—including the Qur’an, hadith collections, fiqh encyclopedias, and tafsir sources—and answers only based on retrieved content, providing citations for verification. The paper details Ansari’s architecture, multi‑platform deployment, evaluation results (including top performance on the IslamicMMLU leaderboard and strong resistance to false premises), and lessons for faith‑sensitive LLM deployments.
arXiv:2608. 12324v1 Announce Type: cross Abstract: People increasingly ask large language models (LLMs) for counsel on questions of faith, doctrine, and pastoral care.
AgentAudit is an open, extensible framework that evaluates the full lifecycle of AI agents, assessing planning, tool selection, execution, memory, and reasoning across ten dimensions such as instruction integrity, security, and alignment. Unlike existing benchmarks that focus on single aspects, AgentAudit analyzes the entire execution trace to attribute failures to specific stages. The framework was tested on five large language models, revealing significant differences in trustworthiness even among models with similar task‑completion performance.
arXiv:2606. 07810v1 Announce Type: cross Abstract: Large language models (LLMs) are widely used as judges for evaluating model outputs, but their high cost, latency, and opacity limit scalability.
LLM companions are deployed at scale in personally consequential settings, yet poorly evaluated. Existing benchmarks use hand-authored scenarios and prompted simulators, aggregate empathy into one score, and overlook judge biases such as same-family favoritism and scale drift.
arXiv:2607. 18064v1 Announce Type: cross Abstract: Coding agents can now be left alone to improve software against a score.
arXiv:2608. 13787v1 Announce Type: new Abstract: AI agents increasingly act on their users' behalf, handling tasks such as scheduling meetings, comparing offers, and haggling over prices.
arXiv:2608. 02046v2 Announce Type: replace-cross Abstract: LLM companions are deployed at scale in personally consequential settings, yet poorly evaluated.
arXiv:2509.08494v2 Announce Type: replace-cross Abstract: As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures....
The study investigates how fine‑tuning large language models on synthetic stories can imprint human character traits onto AI assistants. Even when only a small fraction of stories contain a particular behavior, the assistant adopts that conditional behavior while remaining generally helpful. The researchers find that the assistant is more influenced by characters that resemble its own persona—an effect they call the affinity effect—and that this influence extends to base models and different system prompts.
AgentJudgeBench is a new benchmark that evaluates the reliability of large language model (LLM) judges on agentic tool‑calling tasks involving workflow directed acyclic graphs (DAGs). It contains 3,808 instances across six DAG topologies and three difficulty tiers, tested with five generators (3B–70B open‑weight models and GPT‑5.4) and six judges (20B to frontier scale) under both paired‑with‑and‑without‑ground‑truth conditions. The study finds that judge alignment degrades with task difficulty, ground‑truth exposure can sometimes hurt alignment, and structured evaluation rubrics provide modest improvements, revealing a structural ceiling that model capacity alone cannot surpass.