arXiv AI

A Framework for Identifying, Categorizing, and Explaining Bias in AI-Generated Code

The paper presents a taxonomy-driven framework for identifying, categorizing, and explaining bias in AI-generated Python code. By extending an existing dataset and manually annotating bias categories and justifications, the authors evaluate both proprietary and open-source large language models (LLMs) for automated bias detection and explanation. Results show that models such as Gemini and Qwen3-coder achieve high classification accuracy and produce justification and code identification similarities that closely match human-authored reasoning.

arXiv AI
3d ago

Code as Explanations: Executable and Verifiable LLM Explanations

The paper introduces "code explanations"—programs that describe how large language models (LLMs) generate outputs across a dataset, enabling automatic verification through execution. It defines a Consistency metric to assess how often a program reproduces the LLM’s outputs and evaluates this approach on three open‑source code‑capable LLMs across eight tasks, finding that code explanations are significantly shorter than chain‑of‑thought explanations and often better align with model behavior. However, consistency drops on tasks with complex linguistic components, and even with consistency feedback and execution, LLMs struggle to produce reliable explanations for these harder tasks.

By Nhi Nguyen, Shauli Ravfogel, Rajesh Ranganath
arXiv AI
Oct 28

LLM Code Smells: A Taxonomy and Detection Approach

The paper "LLM Code Smells: A Taxonomy and Detection Approach" presents a refined taxonomy of nine LLM code smells and introduces SpecDetect4LLM, a static analysis tool for detecting them. Extensive empirical evaluation across 692 open‑source projects shows that 73.5% of systems contain LLM code smells, with the tool achieving 91.3% precision and 71.8% recall.

By Zacharie Chenail-Larcher, Brahim Mahmoudi, Naouel Moha, Quentin Sti\'evenart, Florent Avellaneda
arXiv AI
Jun 11

Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study

arXiv:2606. 12231v1 Announce Type: cross Abstract: The adoption of AI-powered Integrated Development Environments (AI IDEs) has introduced "Rules" as a novel software artifact, allowing developers to persistently inject project-specific constraints and architectural guidelines into the context of Large Language Models (LLMs).

By Guangzong Cai, Ruiyin Li, Peng Liang, Zengyang Li, Mojtaba Shahin
arXiv AI
Jun 12

HalluJudge: A Reference-Free Hallucination Detection for Context Misalignment in Code Review Automation

arXiv:2601. 19072v3 Announce Type: replace-cross Abstract: Large Language models (LLMs) have shown strong capabilities in code review automation, such as review comment generation, yet they suffer from hallucinations -- where the generated review comments are ungrounded in the actual code -- poses a significant challenge to the adoption of LLMs in code review workflows.

By Kla Tantithamthavorn, Hong Yi Lin, Patanamon Thongtanunam, Wachiraphan Charoenwet, Minwoo Jeong, Ming Wu
arXiv Machine Learning
Jun 5

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606. 05792v1 Announce Type: cross Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language still requires time and expertise, which limits adoption.

By Arslan Bisharat, Brian Ortiz, Eric Spencer, Khushboo Bhadauria, TaiNing Wang, George K. Thiruvathukal, Konstantin Laufer, Mohammed Abuhamad
arXiv AI
Sep 11

XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?

XAI-Arena proposes using large language models (LLMs) as judges to evaluate the quality of explainable AI (XAI) explanations, aiming for reproducibility, scalability, and multidimensional assessment. The framework assesses dimensions such as simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability across different datasets, models, and stakeholder personas. Human validation shows a strong positive correlation between LLM-generated and human ratings (Spearman's rho = .693, p < .001), supporting the viability of LLM-based evaluations.

By Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
Hugging Face Trending Papers
Jun 22

UnBias-Plus: Detect, Explain, and Rewrite Bias

Bias in natural language remains a persistent challenge in both human-written and AI-generated content, affecting domains such as journalism, education, and AI research. Most existing detection methods identify only the presence of bias, with limited support for granular detection, interpretable explanations, neutral rewriting, and openly available trained models.