The paper presents ESSE, a self‑explanation tutor that uses a large language model to give immediate feedback on students’ line‑by‑line explanations of introductory programming worked examples. It evaluates the LLM’s judgments against a domain expert and a crowd of non‑experts, finding that the model is reliable enough to serve as the tutor’s assessment engine. In an introductory Java course, the tutor’s feedback encourages students to persist, improves the completeness and conceptual depth of their explanations, and shows evidence of learning.
By Arun-Balajiee Lekshmi-Narayanan, Mohammad Hassany, Kamil Akhuseyinoglu, Rully Hendrawan, Peter Brusilovsky
DisasterInsight is a building‑centric benchmark designed to evaluate vision‑language models (VLMs) for disaster response. Built on the xBD satellite dataset, it adds OpenStreetMap‑derived functional labels to 134,108 building instances and offers 15 task types, including instance assessment, scene counting, multi‑instance reasoning, and structured report generation. Experiments show that VLMs excel at visible damage detection but struggle with building function, multi‑instance reasoning, counting, and grounded reporting, and instruction tuning only partially mitigates these gaps.
By Sara Tehrani, Yonghao Xu, Leif Haglund, Amanda Berg, Gulnaz Zhambulova, Michael Felsberg
arXiv:2609.21996v1 Announce Type: new
Abstract: Large language models can hold knowledge they do not report. A model may sandbag on a capability evaluation, or answer against what it internally knows...
By Hiskias Dingeto
arXiv:2609.21190v1 Announce Type: cross
Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras
The article examines how Large Language Models can aid in creating Entity-Relationship diagrams from natural language requirements. It tests three LLMs with three prompting strategies—Zero-Shot, Chain of Thought, and Chain of Thought + Verifier—on scenarios of increasing complexity. Findings show that while LLMs perform adequately on simpler tasks, their reliability drops with more complex requirements, leading to inconsistencies, ambiguities, and constraint representation failures.
By Arthur F. Siqueira, Carlos D. S. Nogueira, Eduarda Farias, Claudio E. C. Campelo, J\'ulia Menezes
Fine‑tuning reshapes internal representations of large language models, affecting attention patterns and layer‑wise activations. The study shows that components identified by EAP as important for task performance cluster in specific layers, yet these layers do not align with those undergoing the largest representational changes. Additionally, overlapping EAP components across different tasks do not guarantee cross‑task transfer and can even degrade performance when tasks differ in nature.
By Lingfang Li, Procheta Sen, Shubham Das, Danushka Bollegala
The paper investigates Japanese past‑tense verb inflection in neural morphological generation systems, focusing on a rare irregular subtype where stems end in /e/ and require gemination before the suffix. Despite overall high accuracy (>97%), this <1% subclass accounts for 30–43% of residual errors, amplifying its impact by 34–48 times its frequency. Ablation experiments show that removing this specific subtype yields larger accuracy gains than removing all irregular verbs, highlighting the role of low‑frequency patterns and orthographic processes in error concentration.
By Wen Zhang
The Spoken Wikipedia Presentation Corpus extends the existing Spoken Wikipedia Corpora by adding LLM-generated slide decks for multimodal automatic speech recognition (ASR). Slides are produced through a hybrid pipeline that combines LLM-based content planning with rule-based design, generating titles, bullet points, takeaway messages, and visual descriptions for illustrations. The corpus is evaluated with multiple ASR and spoken language models, achieving a best micro-WER of 10.23% and micro-CER of 6.48% on audio-only inputs, with English performing best and lower-resource languages showing higher error rates.
By Thomas Ranzenberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer
The paper presents a bi‑modal speech‑level transformer that eliminates segment‑level labeling and introduces a hierarchical attention interpretation method. By using gradient‑weighted attention maps from all attention layers, the model provides both speech‑level and sentence‑level explanations of depression detection. Experimental results show the transformer outperforms a segment‑level model (p=0.854 vs. 0.732, r=0.947 vs. 0.808, F1=0.897 vs. 0.768).
By Qingkun Deng, Saturnino Luz, Sofia de la Fuente Garcia
The paper introduces a clinician‑grounded evaluation platform called InterviewPlayground, which uses a memory‑augmented patient simulator to assess AI‑assisted psychiatric intake systems. It supports comparison across different interviewing styles, reduces clinician workload, and measures clinically relevant performance. In a pilot study, a GPT‑based intake interviewer captured more relevant items but made more unfounded inferences and missed safety concerns compared to clinicians.
By King Shi, Amanda Li, Jonathan Ivey, Synthia Qia Wang, Guan Gui, Hyunseo Kim, Peter Zandi, Jason Straub, Jacob Taylor, Ananya Joshi
Large Language Model agents are being explored for chip design, but most methods work directly at RTL. This study compares Direct RTL Design, Agent-based HLS Design, Post-Compiler HLS Refinement, and Post-HLS RTL Refinement, and proposes a combined workflow called Agent-based HLS with RTL Refinement (AHRR). Using FPGAs for evaluation, AHRR achieves a 2.6× geometric‑mean speedup over Direct RTL Design across an 11‑task benchmark suite, demonstrating that higher‑level abstractions and subsequent RTL refinement can improve chip design efficiency.
By Zijian Ding, Yang Zou, Yizhou Sun, Jason Cong
MENASpeechBank is a new reference voice bank that provides about 18,000 high‑quality utterances from 124 speakers across multiple MENA countries, covering English, Modern Standard Arabic, and regional Arabic varieties. The dataset is built through a controllable pipeline that creates persona profiles inspired by the World Values Survey, defines a taxonomy of roughly 5,000 conversational scenarios, matches personas to scenarios via semantic similarity, and generates around 417,000 role‑play conversations using an LLM. Synthetic speaker‑conditioned user‑turn audio is produced from reference recordings to maintain speaker diversity, and both synthetic and human‑recorded conversations are evaluated and analyzed for quality.
By Zien Sheikh Ali, Hunzalah Hassan Bhatti, Rabindra Nath Nandi, Shammur Absar Chowdhury, Firoj Alam
SpecOpt is a new molecular design task that optimizes the binding specificity of existing drugs by making constrained structural modifications. The method uses an agentic framework that docks a compound against its intended target and known off‑targets, compares residue‑aware atom‑protein contacts, and feeds the differential interactions to a large language model to propose changes. On a benchmark of 915 compounds, SpecOpt increased the target‑off‑target binding gap for 84.8% of cases while preserving drug‑like properties and structural similarity.
By Thao Nguyen, Heng Ji
CogGym is a scalable, unified framework that standardizes diverse cognitive experiments into a task‑agnostic Experiment Markup Language (EML) for systematic comparison of human and AI behavior. The initial release curates 258 experiments from 100 papers focused on human commonsense reasoning and evaluates 50 large language models, revealing a scaling trend where larger models better reproduce human judgments but still lag far behind human split‑half reliability. The framework aims to continually incorporate new cognitive science experiments to track where model behavior aligns with or diverges from human cognition as models evolve.
By Lance Ying, Jinzhou Wu, Yingshan Susan Wang, Shivam Aarya, Luca M. Schulze Buschoff, Harry Chen, Katherine M. Collins, Andrea de Varda, Shuhao Fu, Sean Dae Houlihan, Akshay K. Jagadish, Guangyuan Jiang, Samuel Kiegeland, Tetsu Kurumisawa, Rongzhi Liu, Ryan Liu, Ningshan Ma, Kathryn McGregor, Younes Strittmatter, Polina Tsvilodub, Jacob Hoover Vigly, Sarah Wu, Enjie Xu, Yiling Yun, Kelsey Allen, Tyler Brooke-Wilson, Brian Christian, Evelina Fedorenko, Michael C. Frank, Michael Franke, Tao Gao, Samuel J. Gershman, Robert D. Hawkins, Jennifer Hu, Julian Jara-Ettinger, Max Kleiman-Weiner, Sydney Levine, Tal Linzen, Hongjing Lu, Timothy O'Donnell, Desmond C. Ong, Steven T. Piantadosi, Rebecca Saxe, Eric Schulz, Tianmin Shu, Felix A. Sosa, Ilia Sucholutsky, Tan Zhi-Xuan, Tomer Ullman, Fei Xu, Ilker Yildirim, Jian-Qiao Zhu, Thomas L. Griffiths, Tobias Gerstenberg, Kevin Smith, Joshua B. Tenenbaum
MT‑WAM enhances the Fast‑WAM framework by adding complementary supervision for future 2‑D point trajectories and visual features while keeping the original training objectives. A lightweight dual‑stream branch and structured attention mask isolate motion‑specific processing, and motion‑stream tokens provide additional dynamics cues to the action expert. During inference, MT‑WAM skips future‑video prediction, using cached video and motion information to achieve higher success rates on LIBERO, LIBERO‑Plus, RoboTwin 2.0 Clean2Rand, and several real‑world tasks.
By Yiguang Yang, Jiankun Peng, Xiaoming Wang, Yiran Zhang, Zhibo Fang
The paper introduces a Chinese competitive debating dataset and benchmark, comprising 148 fully adjudicated matches with 2,698 stages and 20,542 exchange units, all annotated by 120 professional judges using a shared rubric. It offers manually verified transcripts, segmentation, and preserved scores, votes, and rationales, enabling evaluation of large language models on winner-tendency prediction, stage-score prediction, and best-debater prediction. Zero‑shot experiments show the best models achieve 66.2% winner‑prediction accuracy, a 0.250 Pearson correlation for stage scores, and 56.8% best‑debater accuracy.
By Zongrui Yang, Haoyuan Li, Zhongsheng Wang, Zhirui Zeng, Pengqian Han, Yi Zhou, Yuting Wang, Jiamou Liu
Samsone is a family of small audio language models (SALMs) designed for on‑device inference, with the flagship Samsone‑134M setting a new state‑of‑the‑art for its size class across multiple benchmarks. The paper also presents Samsone‑99M and Samsone‑356M to study scaling laws, showing that these compact models achieve performance competitive with much larger counterparts. The authors train the models on publicly available data and release training code, weights, mobile‑optimized checkpoints, and an open‑source Android app for real‑time inference.
By Piotr Masztalski, Micha{\l} K. Grzeszczyk, Olaf Sikorski
CIPL (Channel Inversion for Privacy Leakage) is a channel-aware framework designed to evaluate black-box privacy leakage in large language model agents. It models the leakage process through stages of sensitive source, selection, assembly, execution, observation, and extraction, assessing how selected sensitive units become attacker-recoverable outputs. Experiments across memory, retrieval, and tool-mediated targets, plus a live-agent case study, reveal that recoverability depends on factors beyond storage labels, such as observation surface, prompt alignment, retrieval depth, and provider behavior, and that a semantic audit can uncover disclosures missed by exact matching.
By Tao Huang, Guosen Wu, Guolong Zheng, Jiayang Meng, Chen Hou, Xu Yang, Xuechao Yang, Feng Xia
The paper introduces a new multi-draft speculative sampling algorithm that uses Poisson processes to improve inference efficiency and output provenance for large language models. It achieves strong sampling efficiency while allowing an unbiased watermark to be embedded without reducing speculative acceptance. The method relies on an exact list‑coupling‑without‑communication scheme, giving a drafter‑invariant property that benefits both sampling and watermarking, and the authors experimentally confirm its effectiveness.
By Yanxiao Liu, Sicheng Wan, Zhan Gao, Deniz G\"und\"uz
ProTracer is a training‑free framework that uses Vision‑Language Models (VLMs) together with proprioceptive signals to analyze robot manipulation failures. It performs binary failure detection, categorization, explanation generation, and introduces failure onset localization—identifying the earliest moment a robot deviates from a valid trajectory leading to failure. The method leverages proprioceptive dynamics to pinpoint informative action boundaries and converts robot‑state signals into natural‑language descriptions for joint multimodal reasoning, achieving strong performance on both conventional failure diagnosis and the new failure onset localization task.
By Chang Dong, Mehdi Hosseinzadeh, King Hang Wong, Lingqiao Liu, Francois Fraysse, Feras Dayoub, Minh Hoai Nguyen