The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper introduces the Profiling, Investigation, and Judgment (PIJ) benchmark, which contains 2,500 real homicide cases from five countries to evaluate large language models (LLMs) on pre‑arrest criminal investigation tasks. It assesses LLMs across criminal profiling, crime process reconstruction, and sentence prediction, revealing that performance drops as tasks require more implicit reasoning about unknown suspect profiles. The study finds that LLMs lag behind human experts, especially on inferential tasks like motivation and victim‑offender relationships, and exhibit biases in gender, age, and motive attribution.
The paper introduces the Profiling, Investigation, and Judgment (PIJ) benchmark, which contains 2,500 real homicide cases from five countries to evaluate large language models (LLMs) on pre‑arrest criminal investigation tasks. It assesses LLMs across criminal profiling, crime process reconstruction, and sentence prediction, revealing that performance drops as tasks require more implicit reasoning about unknown suspect profiles. The study finds that LLMs lag behind human experts, especially in inferential categories like motivation and victim‑offender relationships, and exhibit biases in gender, age, and motive attribution.
arXiv:2607. 29064v1 Announce Type: cross Abstract: Police crash narratives contain information that may supplement structured crash databases, but manual review is labor-intensive and it remains unclear how well large language models (LLMs) reproduce official crash coding.
The paper presents a decision‑aware framework for predicting dementia‑related crash severity that emphasizes auditability and selective deferral. Using 4,781 Texas crash records, the authors evaluate several models—including structured, narrative, fusion, calibrated fusion, BERT‑family, and local large‑language‑model baselines—under a stratified 70/15/15 split. The leakage‑controlled Gemma model achieves the highest macro‑F1 of 0.545, while a calibrated fusion model reaches 0.522 macro‑F1 with an expected calibration error of 0.033; selective deferral further improves performance, raising macro‑F1 to 0.573 at 70% coverage and reducing severity cost to 0.577.
The paper introduces a reusable Semantic Web framework that aggregates fragmented evidence needed for Fundamental Rights Impact Assessments under the EU AI Act, focusing on high‑risk public sector categories such as employment and worker management and access to essential public services. A curated 150‑record corpus is annotated across four axes and serialized into a SPARQL‑queryable knowledge graph of 1,351 RDF triples, enabling five demonstration scenarios that retrieve 103 records (68.7% coverage). Evaluation against a 69‑record gold standard shows that LLM‑assisted classification in the employment domain yields a low κ of 0.045, highlighting challenges in automated fairness‑related evidence retrieval, while all artefacts are released openly for regulators, authorities, and SMEs.
The paper introduces Expert-Committee Audit Screening (ECAS), a framework that evaluates the credibility of synthetic minority samples for predicting crash injury severity in automated driving systems. Using real incident data from the NHTSA, ECAS filters generated samples based on label support, boundary separation, committee agreement, and local plausibility, then selects accepted samples via within‑class percentile normalization and Pareto non‑dominated sorting. The best ECAS configuration, combined with normalizing flow augmentation and a TabPFN classifier, outperformed other evidence settings in balanced accuracy, macro‑F1, and minor‑injury recall, and analysis showed ECAS‑accepted samples were better supported by nearby real crashes.