The paper investigates why retrieval‑based open‑ended evaluation fails in medical fact verification. By creating two detailed taxonomies—one for retrieval‑stage errors across five quality dimensions and another for verifier‑reasoning errors across six steps—the authors automatically label evidence quality and reasoning errors using an LLM‑as‑Judge pipeline. Their large‑scale stress tests across multiple retrieval methods and verifier models show that increasing model size, reasoning effort, source breadth, or medical fine‑tuning does not eliminate these failure modes, indicating fundamental limits of the retrieve‑then‑verify paradigm in open‑ended medical contexts.
By Heyuan Huang, Jirui Dai, Alexandra DeLucia, Sonal Joshi, Mahsa Yarmohammadi, Jie Gao, Bernal Jim\'enez Guti\'errez, Mark Dredze
The paper introduces an LLM-as-a-judge method to quantify perturbation strength for assessing self-consistency in LLM-generated explanations. It evaluates explanations from various LLMs under controlled perturbation strengths, comparing input and chain-of-thought (CoT) perturbations. Results show the proposed measure outperforms embedding- and probability-based approaches, and that input perturbations impact LLMs more than CoT perturbations, implying fair self-consistency judgments require consistent perturbation types.
By Phuong Q. Le, Kemal Kurniawan, Jey Han Lau
The paper introduces an interaction‑centric framework that unifies representations for two‑finger gripper manipulation across different robot embodiments. By using a parameterized universal gripper abstraction and a canonical gripper‑frame representation, the system infers sub‑tasks from language and RGB‑D inputs, grounds interaction triplets, and employs hybrid features and a Flow‑Matching Transformer to generate smooth 7‑DoF action sequences. Experiments in both simulation and real‑world settings show that this approach achieves competitive benchmark performance while enabling extreme cross‑embodiment and cross‑viewpoint zero‑shot sim‑to‑real transfer to heterogeneous robot platforms.
By Guanlin Li, Shifeng Bao, Yihan Zhao, Haitao Shen, Haoyang Li, Chen Zhao, Tong Yang, Jie Tang, Jing Zhang
The paper presents a two‑stage framework for prenatal congenital heart disease (CHD) screening that operates directly on whole fetal ultrasound studies. It first learns transferable frame representations via self‑supervised masked‑autoencoder pre‑training, then identifies cardiac frames with a disease‑robust module and aggregates them using a transformer‑based multiple instance learning model to produce a case‑level diagnosis. The approach achieves high performance (AUC 0.985, specificity 0.990) on an internal test set and, after label‑free CORAL adaptation, improves to an AUC of 0.944 on an external cohort, outperforming existing baselines.
By Mohamed Azzam, Ruobing Liu, Esther C. Ugwueke, Ziyang Xu, Shibiao Wan, Alex Foy, Abraham Zabih, Jason Christensen, Neil Hamill, Ling Li, Jieqiong Wang
The paper introduces SAGE, a multi‑agent framework that uses specialized analyzers to diagnose univariate time‑series anomalies by examining point, structural, seasonal, and pattern deviations. Each analyzer produces numerical evidence and visual diagnostics, which a Detector consolidates into intervals, candidate types, and confidence scores, and a Supervisor converts these into analyst‑friendly reports. Experiments on Yahoo S5, KPI, and WSD datasets show SAGE achieving the highest average Point‑F1 score (66.26) and receiving higher usefulness ratings in a blind human study.
By Hyeongwon Kang, Jeongseob Kim, Jinwoo Park, Pilsung Kang
The paper introduces HiCoMER, a framework that manages hierarchical collaborative memory and performs validity-aware retrieval for large language model agents. HiCoMER distinguishes between team and individual memories, updates their validity, and retrieves only those that remain valid, rather than treating all memories as a flat pool. Experiments on two new collaborative QA datasets show that HiCoMER reduces outdated retrieval, preserves current team consensus, and improves downstream question‑answering quality.
By Yufei Shi, Rujing Yao, Ang Li, Yang Wu, Zhuoren Jiang, Xiaozhong Liu
The paper introduces SRBench, a benchmark dataset comprising 45,064 labeled entries from 32 curated secondary studies, designed to evaluate large language model performance in systematic review screening while addressing class imbalance. It also presents PromptSR, a tool that facilitates prompt experimentation, experiment management, and result analysis for LLM-based screening. A use case demonstrates the practical application of both SRBench and PromptSR.
By Gauransh Kumar, Luciano Marchezan, Guillaume Genois, K\'evin Delcourt, Eugene Syriani
The paper introduces a new target‑speaker unlearning task for automatic speech recognition (TSU‑ASR) that allows certain speakers to opt out of transcription while still indicating their presence. A lightweight Enrollment‑Conditioned Gating (ECG) module is added to a frozen dual‑stream speech LLM, enabling dynamic unlearning of new opt‑out speakers during inference. Experiments on AMI and AliMeeting datasets show significant drops in transcription accuracy for opt‑out speakers while preserving performance for retained speakers.
By Bo Su, Yueru Yan, Thai Le
The paper investigates whether training small decoder-only transformers on code‑switched text can induce cross‑lingual alignment. Using two 100‑million‑word multilingual corpora—one a mix of English, Dutch, and Chinese BabyBabelLM data, and another generated by inserting word‑ and sentence‑level code‑switching via an LLM—the authors find that code‑switched training aligns representations of parallel text, especially across different scripts, and that this alignment persists when later training on monolingual documents. A curriculum that progresses from word‑level code‑switching to sentence‑level code‑switching and finally to monolingual data yields models that outperform baselines on the BabyLM evaluation suite, demonstrating that code‑switching curriculum learning is an effective data augmentation strategy for multilingual pretraining.
By Dries Rooryck, Alex Cai, Yonatan Belinkov, David Alvarez-Melis, Kiant\'e Brantley
REALMS is a conversational system that provides real‑time, exact audience sizing for digital marketers. It uses embedding‑based vector search to retrieve relevant categorical attributes, an LLM‑powered NL2SQL pipeline for accurate query generation over complex nested schemas, and schema standardization for industry‑agnostic deployment. Evaluations on real enterprise data show high recall, accurate SQL execution, and low latency, enabling interactive audience insights that previously required hours.
By Haixu Ma, Aditya Bansal, Shubham Lohiya, Sumit Ranjan
The study investigates how different transcript compression methods affect large language model (LLM) detection of medical misinformation in Japanese YouTube videos. Four input designs were compared: full transcripts, LLM-generated summaries, RAPTOR-based retrieval‑augmented generation (RAG), and a Screening approach that extracts candidate medical sentences. Results show that full transcripts yield the best classification accuracy, while all compressed inputs increase false negatives, with summaries causing the largest performance drop and Screening performing best among compressed methods yet still missing many relevant sentences. Linguistic analysis indicates that compression reduces affective, social, temporal, cognitive, and conversational cues, and increases the prominence of institutional and technical terms, thereby making fake videos appear more coherent and authoritative.
By Yuya Wake, Sho Tsugawa, Toshiyuki Amagasa
The paper proposes a new way to evaluate language models on cultural understanding by focusing on interpretive depth rather than just factual recall. It argues that literary interpretation, where scholars can disagree yet still assess the quality of evidence, provides a useful framework for testing how models handle cultural references, reuse, and transformation across texts. The authors suggest linking evidence-centered benchmarks, preserving scholarly disagreement, and conducting model-development experiments on literary data, with Danish literature as a starting point for broader applications.
By Daniel Hershcovich, Alexander Conroy, Jens Bjerring-Hansen
The paper introduces a training‑free speech‑and‑text‑to‑pronunciation (ST2P) pipeline that combines lexical candidates from G2P tools with acoustic rescoring using frozen pretrained S2P models. By performing a left‑to‑right greedy search over whole‑sequence negative log‑likelihoods, the method achieves a dramatic reduction in character error rate on Japanese corpora, outperforming both baseline G2P/S2P approaches and commercial multimodal LLMs. The approach is also significantly faster—3–3.5× faster than beam search and twice as fast as direct decoding—while maintaining high accuracy across multiple languages.
By Hikaru Asano, Yotaro Kubo, So Kuroki
The study investigates sycophancy in Chinese large language models (LLMs) by analyzing 364,941 responses from DeepSeek, Qwen, and Doubao to 12,165 yes/no factual questions derived from real-world search queries. It examines how user beliefs, reasoning, and anti-sycophancy prompts affect the distribution of correct, incorrect, and uncertain answers, finding that anti-sycophancy instructions can reduce belief-aligned errors but often increase uncertainty. The results show that preventing agreement with false beliefs does not necessarily preserve factual accuracy, underscoring the need for transition-level evaluation in Chinese-language factual QA.
By Geng Liu, Feng Li, Mengxiao Zhu, Francesco Pierri
The study explores whether instruction-tuned large language models (LLMs) can analyze collaborative student sensemaking without task-specific training, and whether adding structured knowledge-state information enhances this analysis. Two mid-size LLMs were evaluated on 23 expert-labeled episodes under various prompting conditions, showing that reasoning-enabled prompts better detect unsuccessful sensemaking and that knowledge-state diagnostics improve agreement with experts. No single configuration outperformed others across all sensemaking dimensions, highlighting the task’s multidimensional nature.
By \"Ozge Alacam, Z\"ubeyde Demet Kirbulut G\"une\c{s}, Funda Ekici, Nurcan Turan-Oluk, Dilay Din\c{c}demir, Hakk{\i} Kaday{\i}f\c{c}{\i}, Sevin\c{c} Nihal Ye\c{s}ilo\u{g}lu, Burcu I\c{s}{\i}k, Halil T\"umay, Sinem Gencer
CG-Probes are a method for extracting guardrail directions—Medical Urgency, Psychological Urgency, and Topic Sensitivity—from patient query embeddings. By treating each risk axis as a linear direction in the normalized embedding space, the approach uses difference‑in‑means probing and contrastive pairs generated from clustered Czech oncology search queries. Evaluated on 200 queries graded by oncologists, the probes perform comparably to open‑weight and frontier LLMs while offering lower latency and requiring only search logs, axis definitions, and black‑box embedding access.
By Marko \v{R}eh\'a\v{c}ek, V\'it\v{e}zslav Du\v{s}ek, Martin Rusinko, V\'it Nov\'a\v{c}ek
RupeeBias is a new benchmark that audits demographic bias in large language models (LLMs) when they provide economic guidance in India. It contains 39,150 prompts across four use cases—salary estimation, salary increment estimation, counter‑offer recommendation, and service pricing recommendation—varying 87 India‑specific demographic identifiers such as caste, religion, regional identity, gender, disability, and urban‑rural location. Evaluations of nine LLMs show that outputs differ by an average of 20.2% when only the demographic identifier changes, revealing systematic disparities across all six axes.
By Pavithra P M Nair, Bhavik Talaviya, Shourya Bhushan, Rahul Pankajakshan, Seema Guruvadoo, Avinash Agarwal, Gilad Gressel, Krishnashree Achuthan
The paper introduces "stale‑document poisoning," a temporal alignment failure where outdated retrieval evidence causes retrieval‑augmented generation models to produce incorrect answers even when the model could answer correctly without retrieval. A benchmark of 317 verified knowledge reversals across medicine, law, software, and platform policy shows that outdated evidence flips 30–37% of Llama and Qwen answers, rising to 66–75% when models are explicitly instructed to trust the document. The study demonstrates that providing explicit validity information and a recency‑aware re‑ranker can substantially reduce poisoning, highlighting the need for models to assess whether retrieved evidence remains applicable.
By Md Shamim Ahmed, Lukas Galke Poech, Richard R\"{o}ttger
The paper introduces two new sampling techniques—GROOT and Verbalized Sampling—to create strategically diverse training data for self-training large language models. By focusing on substantive variation in problem-solving approaches rather than just correctness, the authors demonstrate that models trained on this data outperform those trained on IID samples across competitive programming and Next‑Chapter Prediction tasks. Notably, self‑training with strategically diverse but incorrect traces from Qwen3‑4B surpasses IID distillation from a 235B teacher, challenging assumptions about the importance of correctness and teacher scale.
By Alexander Gurung, Esmeralda S. Whitammer, Mirella Lapata
The paper evaluates whether music‑text models truly capture fine‑grained musical meaning by introducing attribute‑swap perturbations that exchange properties such as timbre or order between instruments in a caption. Four contrastive models and one large audio‑language model were tested to see if they would score higher on the original caption than on the perturbed one. The results show that none of the contrastive models reliably distinguish the captions, and the audio‑language model’s advantage stems mainly from language priors, indicating that CLAP scores behave like a bag‑of‑words and fail to reflect attribute bindings.
By Yuan-Chiao Cheng, Alexander Lerch