arXiv AI By Yunke Zhao, Isobel Voysey, Alastair van Heerden, Rob Hughes, Jun Zhao

A Human-Centred Approach to Benchmarking LLMs for Parenting Advice

Read the original on arXiv AI →

arXiv:2608. 14622v1 Announce Type: new Abstract: People are increasingly using large language models (LLMs) to seek advice, including for parenting.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Sep 4

The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models

The paper introduces KIDBench, a benchmark designed to evaluate the safety of large language models (LLMs) for children aged 7-11. It includes realistic child queries across ten categories, single- and multi-turn prompts, and compares different prompting strategies—no cues, implicit cues, and explicit age instructions—showing that cueing improves safety scores. The study also reveals uneven safety performance across languages and cultures, and presents KIDGuardLlama, a child-safety evaluator, and KIDLlama, a child-safe response model.

By Samee Arif, Angana Borah, Rada Mihalcea
arXiv AI
Aug 19

Beyond BFI: The CSI for Enhanced Reliability and Validity in Evaluating LLM Personality Traits

The paper introduces the Core Sentiment Inventory (CSI), a new personality trait evaluation tool for large language models (LLMs) that addresses reliability and validity issues found in existing methods like the Big Five Inventory (BFI). CSI is designed specifically for LLMs, supports both English and Chinese, and provides detailed psychological portraits of model behavior. Experiments show that CSI captures nuanced behavioral patterns, improves reliability, and correlates strongly (above 0.85) with real-world LLM outputs.

By Huanhuan Ma, Haisong Gong, Xiaoyuan Yi, Xing Xie, Philip S. Yu, Dongkuan Xu
arXiv Computation and Language
Sep 22

LLJ Cards: Best practices for the Use of LLMs as Judges

arXiv:2609.24516v1 Announce Type: new Abstract: In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJs), these s...

By Khaoula Chehbouni, Melina Medjdoub, Florian Carichon, Golnoosh Farnadi, Jackie Chi Kit Cheung
arXiv Computation and Language
Aug 24

Using Human-LLM Disagreement to Improve Checklist-Based Quality Appraisal

arXiv:2608.20385v1 Announce Type: new Abstract: Systematic reviews rely on quality appraisal of included studies, a process that is time-consuming and sensitive to ambiguity in checklist criteria. Al...

By Timo van der Kuil (Methodology and Statistics Utrecht University), Bruno Messina Coimbra (Methodology and Statistics Utrecht University), Mirjam van Zuiden (Clinical Psychology Utrecht University), Robert A. Bagheri (Methodology and Statistics Utrecht University), Rens van de Schoot (Methodology and Statistics Utrecht University), Klaas Dieleman (Methodology and Statistics Utrecht University), Berend Greijn (Methodology and Statistics Utrecht University), Stefan Houkes (Methodology and Statistics Utrecht University), Sebastiaan Rodenhuis (Methodology and Statistics Utrecht University), Elizabeth M. Grandfield (Methodology and Statistics Utrecht University)