Hugging Face Trending Papers

Auditing Recorded Predictive Lead Service-Line Classifications Against Physical Verification: A Statewide Study of New York

Under the US Lead and Copper Rule Revisions, a utility may determine a service line's material with a predictive model instead of inspecting it. New York State publishes, per address, which method was used.

arXiv Computation and Language
3d ago

Humanized but Still True? Auditing Facts and Detection in AI-Text Humanizers

The study audits AI "humanizers"—tools that rewrite machine‑generated text to evade detection—by checking whether factual units such as numbers, dates, names, and quoted spans survive each rewrite. Using a rule‑based checker validated by AI annotators, the authors found that in the 2026 public leaderboard, 35–65% of documents had at least one factual unit altered or dropped, with quoted spans most affected. An open 12B humanizer retained most numbers and names but reworded 67–78% of quotes, and a zero‑shot detector (Jev) could distinguish human originals from humanized rewrites with an AUROC of 0.864, outperforming several existing detectors.

By Ziyuan Wang, Genliang Zhu, Xiaobo Chen
arXiv AI
Oct 26

Spatial disparities in simulated patrol detection: a reproducible generative-model audit across Baltimore and Chicago

The study presents a reproducible spatial simulation audit that combines a generative adversarial network for patrol allocation with a probabilistic detection model to assess patrol detection disparities in Baltimore and Chicago. Using archived crime records and census data, the authors compare GAN-based allocation to empirical and uniform deployment across multiple random seeds, finding detection-rate differences between Black-majority and White-majority areas ranging from –2.03 to 0.75 percentage points and coverage between 1.78% and 7.50%. The audit also evaluates a conditional tabular generative model for synthetic replacement, noting that geographic consistency fails in many cases, and highlights how simulated disparities depend on observation processes, allocation policies, training budgets, and randomness.

By Pronob Kumar Barman, Pronoy Kumar Barman
arXiv Machine Learning
Oct 5

Validated Data Onboarding for AI Demand Forecasting on U.S. Building Meter Data: Design, Controlled Evaluation, and a Corrected Negative Result

The paper introduces a data‑onboarding pipeline that detects and repairs defects in building meter data before training machine‑learning demand‑forecasting models. Controlled experiments on hourly electricity data from twelve U.S. buildings show that the pipeline restores forecast accuracy to clean‑data levels, even when defects affect up to 1.6% of training rows, and works across multiple models and horizons. The study also documents a first‑version over‑cleaning issue, its cause, and a per‑building calibration that corrects it, with all results reproducible from publicly pinned inputs.

By Yixuan Liang
arXiv Machine Learning
Jul 9

Prior-matched evaluation of operational Earth-observation classifiers: a three-number reporting method demonstrated on Sentinel-1 internal-wave detection

arXiv:2607. 07146v1 Announce Type: new Abstract: The Internal Waves Service screens the Sentinel-1 Wave-mode archive for internal solitary waves, routing detections to experts whose adjudication time is the resource the effort exists to conserve.

By Joao Pinelo, Joao Goncalves, Arun Shukla, Adriana Santos-Ferreira
arXiv AI
Sep 1

Benchmark Contamination: A Taxonomy Organized by Defeated Mitigation

The paper introduces a new taxonomy for benchmark contamination that categorizes leakage by the mitigation it defeats—direct, derivative, temporal, distributional, and acquired—covering both training‑time and evaluation‑time scenarios. It proposes a four‑field disclosure protocol to record contamination status alongside benchmark scores, and provides a JSON schema, validator, and examples. An empirical study of 41 documents using a pre‑registered instrument shows limited reporting of contamination types and variable reliability, highlighting gaps in current disclosure practices.

By Johanna Angulo, V\'ictor Yeste, Hector Espinos-Morato
arXiv Machine Learning
Jul 21

STRATA: A Name-and-Geography Race Inference Model for Fair Lending and Housing Equity Applications

arXiv:2504. 21259v2 Announce Type: replace-cross Abstract: Accurate imputation of race and ethnicity (R&E) is essential for fair lending compliance under ECOA, HMDA, and the Community Reinvestment Act, where up to 15% of mortgage applications carry missing race data and regulated institutions bear responsibility for identifying disparities on those records.

By S. Chalavadi, A. Pastor, T. Leitch
arXiv Machine Learning
5d ago

What Standard Evaluations Hide About Detectors of Hijacked LLM Agents: Rare Attacks, Unseen Tools and Benchmark Transfer

The paper investigates how standard evaluations of detectors for hijacked large language model (LLM) agents may conceal important weaknesses. By auditing eleven evaluations on the AgentDrift and AgentDojo benchmarks, the authors show that detectors often perform poorly when faced with rare attacks, unseen tools, or data from a different benchmark, with precision dropping dramatically at low attack rates and rankings reversing as tool coverage falls. They also find that a released LLM action monitor, despite high AUROC on its own data, would still generate many false alerts when applied to a new benchmark. "whyItMatters":"The study highlights that current evaluation practices can give a misleading sense of detector reliability, underscoring the need for more robust, cross‑benchmark testing to ensure real‑world safety of LLM agents."

By Mahmoud Abbasi, Juan M. N\'u\~nez V., Amin Shahraki