The paper presents a bilingual (English/Arabic) benchmark for evaluating how well open-weight, locally deployable language models can accurately recall specific Hadith texts. Five models—Mistral‑7B, Gemma3‑4B, Jais‑7B, ALLaM‑7B, and Phi4‑mini‑3.8B—were tested on 3,000 Hadiths per language across six canonical Sunni collections, with each model given only a bibliographic reference and asked to reproduce the exact text. None of the 30,000 responses achieved both high lexical and semantic fidelity, and Arabic outputs performed uniformly worse than English, indicating that these models, without retrieval augmentation, cannot reliably provide verifiable religious scripture.
By Dayam Nadeem, Safdar Mustafa, Ameen Ur Rehman, Shahab Saquib Sohail, Mohammad Nadeem, Usman Naseem
The paper investigates how a language model’s first answer to a yes/no question influences its subsequent answer to a different question about the same entity, a phenomenon termed self‑commitment. By fixing the initial answer and measuring changes in the second across 3,600 question pairs for 22 open‑weight models (0.5B–72B parameters), the authors find that each answer writes a commitment direction into the residual stream, and the similarity of these directions predicts answer coupling. They show that ablating a shared direction reduces coupling significantly, while the reading circuit for this direction varies with model size, instruction tuning, and prompts, and that sparing the attention sink during ablation preserves low perplexity cost in most models.
By Marcus Armstrong
MultiInpaintCOCO extends the English InpaintCOCO dataset to 13 typologically diverse languages, creating a multilingual benchmark of 953 minimal-pair examples focused on fine-grained visual details. The study evaluates state-of-the-art multilingual multimodal retrieval models, revealing that performance drops significantly for lower-resource languages and that many models exhibit a systematic bias toward English captions. The benchmark and code are publicly available at https://github.com/ufal/multi-inpaint-coco.
By Andrei-Alexandru Manea, Jind\v{r}ich Libovick\'y, Philipp J. R\"osch
VeriHarm-Bench is a new benchmark that measures harmful success in large language model coding agents by automatically generating verifiable tests that detect unintended harm while preserving the original task objective. The framework minimally perturbs coding tasks to elicit harmful behavior and synthesizes executable harm tests alongside task‑correctness tests. In evaluations, frontier agents such as GPT‑5.6‑Sol and Claude‑Opus‑5 achieve high task accuracy (≈95%) but only a 37% safe success rate, showing that task success can coexist with significant unintended harm.
By Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Yu-Min Tseng, Tzu-Han Lin, Yu Meng
The paper introduces SHIFT, a method for unlearning in large language models that avoids explicit forgetting targets by shrinking a fine‑tuned model toward its pre‑fine‑tuning weights and briefly relearning the retain set. In the linearized regime, this shrink‑and‑heal lookahead moves the model a fixed fraction toward the retrained model, and SHIFT extrapolates these per‑token changes into distillation targets for iterative refinement. Evaluated against eleven baselines under the OpenUnlearning protocol, SHIFT achieves the highest aggregate score across all TOFU settings, aligns 3.7× closer to the oracle than the next best method, and preserves the most retain knowledge on MUSE, maintaining proximity to the oracle even under larger optimization budgets and relearning attacks.
By Hoang-Chau Luong, Lingwei Chen
The paper introduces TAILOR, a framework that uses Conditional Value at Risk (CVaR) to guide large language model (LLM) synthesis of heuristics for computer systems. By optimizing the paired CVaR of cost differences relative to a baseline, TAILOR ensures improvements in both average performance and tail optimality gaps without needing to compute the unknown optimum. Experiments on Azure GPU batch admission, Azure VM packing, and server placement demonstrate that TAILOR achieves the best mean–tail trade‑off and outperforms robust, human‑designed alternatives in the tail while maintaining comparable average gains.
By Pantea Karimi, Roozbeh Bostandoost, Dany Rouhana, Siva Kesava Reddy Kakarla, Pooria Namyar, Mohammad Alizadeh, Venkat Arun, Behnaz Arzani
The paper introduces Orthogonal Drift Correction (ODC), an inference-time guidance method that mitigates semantic drift in text-to-image generation without retraining or extra user input. ODC operates in two stages: first it detects drift by comparing the generated image to the prompt in a shared vision-language embedding space, isolates the orthogonal component of the drift, and converts it back into text; second it refines the text conditioning for a second generation pass using an adaptive rank-reduced concept removal module. Experiments show ODC improves prompt-image alignment, raising CLIP scores by up to 5.4% on COALBench-300 and PartiPrompts, and boosting the 3-in-1 score by 4.5% on T2I-CompBench’s complex subset.
By Kasra Kamali, Basim Azam, Tuan Duong Trinh, Jean Honorio, Naveed Akhtar
The paper presents a rubric‑based assessment pipeline that uses lightweight language models and a vision‑language model to evaluate student reports in cybersecurity education. It demonstrates that small models can achieve up to 97% accuracy on text‑only assessments and 94% on rich‑text assessments across various low‑resource hardware setups. The solution is open‑source, locally deployable, and designed to reduce instructor workload while addressing privacy and cost concerns.
By Sandesh Kumar Kafle, Jan Vykopal, Valdemar \v{S}v\'abensk\'y
The paper examines how different preprocessing steps in fine‑tuning data influence the performance and fairness of a generative large language model recommender system. Using a controlled pipeline that includes supervised fine‑tuning, catalog‑constrained decoding, and prompt augmentation, the authors evaluate three preprocessing choices—item‑title representation, demographic balancing, and auxiliary item‑knowledge enrichment—on the MovieLens‑1M dataset. Results show that catalog constraints eliminate out‑of‑catalog items, prompt augmentation improves consistency and utility, and that demographic balancing and knowledge enrichment can reduce utility gaps for certain groups while preserving overall performance, though gender balancing may lower utility without improving gender indicators.
By Furkan Yasir Goksu, Yaren Yilmaz, Sule Gunduz Oguducu
The paper investigates how monitoring the internal representations of Large Reasoning Models (LRMs) during their Chain-of-Thought (CoT) reasoning can predict future behavior. By evaluating a probe at each generated token, the authors construct probe trajectories that capture the continuous evolution of a concept’s probability, and extract signal‑processing features to characterize volatility, trend, and steady‑state behavior. These trajectory features improve predictive performance over static probes, adding an average of 21.7 AUROC points, and the study demonstrates that template‑based training data and pooling operations are essential for effective monitoring across multiple reasoning models and datasets.
By Maciej Chrabaszcz, Aleksander Szymczyk, Marcin Sendera, Tomasz Trzcinski, Sebastian Cygert
SpeechEQ is a new benchmark that evaluates the sociolinguistic reasoning of Speech‑Language Models (SLMs) by using a dataset of 2,265 dialogues covering 15 Emotional Quotient (EQ) subscales from EQ‑i 2.0 theory. The framework introduces a multi‑turn evaluation protocol and a Spoken EQ (SEQ) score inspired by human EQ assessments. Experiments reveal that current multimodal models suffer from a text‑reliant shortcut, an alignment‑induced safety trap, and contextual amnesia, limiting their emotional awareness in spoken interaction.
By Liang-Yuan Wu, Zih-Ching Chen, Tongshuang Wu, Chao-Han Huck Yang, Hua Shen
The paper introduces Multi-View Latent MoE, a sparse mixture‑of‑experts architecture that uses multiple independent routers to generate richer selection signals and to modulate expert computation. By aggregating sigmoid scores from these routers, the model selects experts that are strongly favored by one or several routers and mixes their relative scores into each expert’s input, thereby reusing routing preferences for both selection and computation. Experiments on a 24‑layer 10B‑A1B model trained on 708 B tokens show a 1.18‑point improvement in nine‑task downstream mean over the baseline, with the Multi‑Router submodule alone providing a 0.6‑point gain and significantly better alignment between router scores and expert utility.
whyItMatters":"The approach demonstrates that leveraging multiple routers can both improve expert selection quality and maintain computational efficiency, leading to measurable downstream performance gains on large‑scale language models."
By Zelin Zhao, Junyuan Shang, Tianchi Ma, Shuohuan Wang, Dianhai Yu
The paper presents a multimodal house‑price prediction framework that combines structured attributes, textual descriptions, and property images. It uses a large language model to extract spatio‑temporal and contextual features from text and an unsupervised vision‑language method to cluster images into categories such as indoor, outdoor, mixed, or floorplan without manual labeling. The fused representation is fed to an optimized regressor, and experiments on a large real‑world dataset show improved accuracy over traditional and existing multimodal baselines, with ablation studies highlighting the contribution of the new textual and visual features.
By Khandokar Md. Rahat Hossain, Souvik Das, Tahsina Hashem, Mohammed Eunus Ali, Tanzima Hashem
The paper presents a method for evading Knowledge-Guided Watermarking (KGW) by replacing token‑event evidence with ordinary phrase rewrites. It jointly selects LLM‑proposed alternatives, measures global token‑edit distance, and uses exact coverage search to maximize removal of original scored token pairs within a budget. Experiments on short passages show that about half of detected sources can be rewritten to pass a blinded detector with a moderate token‑edit cost, though the approach does not outperform simpler rewriting strategies.
By Hongliang Liu
The paper introduces Self‑Steered Reasoner (SSR), a method that aligns self‑distillation with the reasoning actions a large language model can actually reach. SSR projects privileged teacher trajectories onto locally reachable actions, filters for correct paths, and iteratively refines sampling using diagnostic guidance. Experiments show that SSR recovers learnable trajectories on hard problems and boosts accuracy by 4.9 percentage points on benchmark datasets.
By Zijian Wu, Lingkai Kong, Lin Yang, Haiteng Zhao, Kuikun Liu, Kai Chen, Dahua Lin, Wenwei Zhang
The paper introduces MedForget, a hierarchy‑aware multimodal unlearning benchmark that models medical data as nested institutional structures, allowing fine‑grained evaluation of retain and forget splits. Experiments reveal that existing unlearning methods struggle to achieve effective hierarchy‑aware forgetting without harming retained utility on clinically relevant tasks. To address this, the authors propose CHIP, a training‑free method that deletes target‑specific weight subspaces while preserving sibling‑shared information, achieving superior forget‑retain performance across all hierarchy levels while maintaining competitive downstream utility.
By Fengli Wu, Vaidehi Patil, Jaehong Yoon, Yue Zhang, Mohit Bansal
The paper introduces an information‑theoretic framework for designing sparse autoencoder‑based steering interventions in large language models. It uses the Jensen–Shannon divergence between empirical feature‑firing distributions to quantify how much a state deviates from a target distribution and to construct steering vectors that guide hidden states toward desired firing patterns. Experiments demonstrate the method’s effectiveness in question‑mode generation, show structured downstream effects of local interventions, and apply the framework to safety‑related feature analysis.
By Hirofumi Morise
The paper introduces a discrete-class framework for detecting human‑AI co‑authorship, distinguishing between superficial AI polishing and deeper AI‑driven discourse changes. It shows that adversarial paraphrasing can obscure detection signals, but augmenting training data with such paraphrases improves detector robustness across multiple datasets and generation models. The study demonstrates that principled data augmentation and a fine‑grained classification approach are essential for reliable co‑authorship detection.
By Ziruo Zhao, Jie Fu, Wendy Hui Wang, Zining Zhu
The paper introduces D&A (Describe & Assess), a prompt-based method that separates progress estimation into a textual description of scene dynamics and a subsequent assessment using explicit rubrics. By requiring Vision‑Language Models to generate an explicit definition of absolute success before scoring, D&A anchors evaluations to an external criterion, reducing confirmation bias and providing fine‑grained, partial progress metrics. Empirical results show that D&A matches or surpasses post‑trained reward models across tasks, embodiments, and benchmarks, and its reward estimates support efficient downstream reinforcement learning.
By Marius Memmel, Shawn Yin, Urvi Rutia, Angad Wadhwa, Eldon Li, Gavin Angeloff, Ankit Goyal, Jesse Zhang, Dieter Fox, Fabio Ramos, Anqi Li, Abhishek Gupta
The paper introduces GAVAGAI, a method that adds a referential alignment variable to vision‑language models by solving a small assignment problem via entropic optimal transport. By allowing words to be assigned to a special NOTHING column and enforcing a capacity constraint on regions, the approach improves word‑to‑picture accuracy and localization on the SAYCam infant video corpus. Experiments show that the alignment loss boosts performance over a standard next‑token captioning baseline, especially under high non‑reference rates.
By Rajdeep Pal, Jotiraditya Banerjee, Saptarshi Pani