The paper investigates how to improve confidence calibration for large language models (LLMs) used in automated code revision (ACR). It proposes applying local Platt-scaling to three fine-grained confidence scores, rather than the conventional global method, and demonstrates that this approach consistently reduces calibration error across multiple tasks, metrics, and model sizes. The study shows that fine-grained calibration, especially when combined with global scaling, yields more reliable confidence estimates for ACR tasks.
By Hong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, Christoph Treude
The paper introduces ‘DP-SPIN’, a trusted‑curator framework that generates differentially private semantic plans for aggregate insight generation. ‘DP-SPIN’ maps each record to a bounded sparse nonnegative vector over pre‑defined semantic concepts, sums these vectors into a semantic sketch, and releases a noisy plan containing admitted concepts and their masses. The framework provides user‑level privacy by clipping each user’s contribution and ensures that the final summary is differentially private through post‑processing, with guarantees established under both add/drop and replacement adjacency.
By Behrooz Razeghi
The paper presents a mean‑field analysis of attention in language models, defining an average attention kernel that propagates representations layer by layer. When conditioned on a whole corpus, the kernel predicts the average evolution of representation geometry; when conditioned on a single context, it predicts the expected geometry for that context. The difference between actual attention and the mean‑field prediction—called the mean‑field deviation—captures context‑specific computation, revealing how models diverge from average behavior during training and in few‑shot tasks.
By Micah Adler, John W. Byers, Mark Crovella
HyCoSeq is a new framework for learning genomic representations in hyperbolic space. It combines weighted Lorentzian residual aggregation with multi‑curvature Lorentz encoding, enabling full Lorentz representations to contribute directly to local aggregation. A bidirectional LSTM further captures contextual relationships across the sequence, extending local hyperbolic convolutions to sequence‑level representations. Experiments show HyCoSeq surpasses existing hyperbolic baselines and competes with much larger pretrained DNA language models without large‑scale pretraining.
By Chenhao Zeng, Zhibin Pu, Shufei Ge
The paper introduces Open-1B, a language model trained under a new fully auditable regime that ensures every training operation is reproducible on heterogeneous commodity hardware with bitwise certainty. By enforcing a fixed order on sources of nondeterminism—GPU reductions, data batch ordering, and inter/intra-node communication—the authors enable auditors to replay and verify individual training steps on a single machine. The release includes the full pretraining dataset, all intermediate checkpoints, the training codebase, and an audit harness for step-by-step verification.
By John Donaghy, Brian Wilcox, O\u{g}uzhan Ersoy, Shikhar Rastogi, Adam St Arnaud, Alexey Titov, Jordan Greenberg, Ben Fielding, Harry Grieve
The paper demonstrates that ordinary typos can significantly disrupt probes that detect malicious prompts in large language models by rotating and decaying the readout vector. A single-position probe’s true positive rate drops by 12 percentage points when three common typos are stacked, and even advanced aggregators only partially mitigate the effect. Introducing a KV‑cache fork—appending a short suffix after the user message—recovers 95% of the lost detection capability, outperforming perturbation‑augmented training by an order of magnitude.
By Elad David, Max Fomin, Amit LeVi
ViCo is a training framework that improves the visual quality of AI-generated academic charts by using iterative self-reflection to align chart images with reference designs. It introduces a self-supervised warm-up stage that augments Monte Carlo Tree Search with consistency-based pruning, and a multi-step reinforcement learning algorithm that employs counterfactual baselines to address sparse rewards. An automatic, multifaceted evaluation framework assesses style, layout, and semantic consistency through a hierarchical heterogeneous layout graph, and experiments on three public benchmarks show ViCo’s performance approaching that of proprietary large language models.
By Jiaxin Duan, Dian Jiao Shuai Zhao, Jiabing Leng, Yiran Zhang, Feng Huang
The paper reports on shuttling compilers for trapped‑ion quantum computers that are built using five large language models (LLMs) fine‑tuned on hand‑crafted shuttling schedules for linear and branched one‑dimensional trap architectures. For circuits up to 16 qubits, the fine‑tuned LLMs produce valid schedules on the training architectures, and in 12% of compilations the best of ten runs achieves up to 21% fewer operations than heuristic baselines after rule‑based post‑processing. A single run of one LLM also generates a valid schedule for a previously unseen four‑way branched architecture, providing preliminary evidence of cross‑architecture generalization, though no LLM succeeded on two other unseen architectures.
By Fabian Kreppel, Reza Salkhordeh, Ferdinand Schmidt-Kaler, Andr\'e Brinkmann
The paper introduces LWVIC4Code, a non‑contrastive, layer‑wise representation learning method for detecting Type‑IV code clones—semantically equivalent fragments that differ syntactically. It builds on the VICReg framework, adding cross‑layer consistency regularization and depth‑dependent weighting to refine semantic information across transformer layers. Experiments on Python and multi‑language datasets show that LWVIC4Code matches or outperforms contrastive baselines and zero‑shot large language models, generalizing well to Java and C# without requiring negative samples.
By Luciano Marchezan, Kevin Delcourt, Eugene Syriani, Houari Sahraoui
The paper introduces Window-Diffusion, a method that accelerates diffusion language model inference by pruning and caching tokens within a sliding window. It categorizes undecoded tokens into active, buffer, and far-field groups, computing only the first two while discarding the rest. Experiments on LLaDA and Dream demonstrate up to 99× speedup with minimal loss in generation quality.
By Fengrui Zuo, Zhiwei Ke, Yiming Liu, Wenqi Lou, Chao Wang, Xuehai Zhou
The paper introduces NepKANUN, an AI-powered legal assistant designed specifically for Nepali legal texts. Built on a fine‑tuned large language model and integrated into a Retrieval‑Augmented Generation (RAG) framework, it delivers precise answers to natural language legal queries. Evaluated with BERTScore, the system achieved strong F1 scores of 0.82, 0.77, and 0.71 across simple, moderate, and complex question categories, and expert reviews confirm its usability.
By Bhabuk Thapa, Prasiddha Koirala, Ranjit Raut, Sunil Regmi, Bal Krishna Bal
The paper introduces ReMova, a pipeline for cleaning Belarusian data and fine‑tuning large language models (LLMs) for English‑to‑Belarusian translation. It uses a correction tool to handle the two orthographies of Belarusian, remove noise, filter out interference from other languages, and correct common misspellings found online. Ablation experiments on unfiltered data show that filtering benefits all fine‑tuned models, with LLM‑based models gaining about twice as much as a dedicated encoder‑decoder MT system, highlighting data quality as a key bottleneck for Belarusian MT.
By Mikita Pilinka, Aliaksandr Kliuje\u{u}, David Samuel, Yves Scherrer
The paper introduces the SAST-IR framework to evaluate large language models’ robustness against persuasion attacks in a memory‑less setting, revealing a flaw called "Refusal Inertia" that masks true vulnerability. Using the CP‑Agent and a custom CounterFact‑Strict dataset, the authors demonstrate that simple, diverse attack strategies achieve a 96% success rate, while complex attacks often trigger defensive compliance. The study highlights severe brittleness in current state‑of‑the‑art models when deprived of conversation history.
By Zhuoang Cai
The paper examines how open‑weight language models expose the control tokens used in chat templates, allowing attackers to forge turn boundaries that the model treats as legitimate. An audit of 256 deployed tokenizers shows all are vulnerable, and the commonly recommended flag fails to protect 56.6% of cases. The authors introduce nameless tokenization, which removes surface strings for control identifiers while preserving their internal representation, achieving identical token streams on clean data and significantly improving accuracy on delimiter‑bearing text.
By Kisu Yang, Yoonna Jang, Heuiseok Lim
The paper introduces GUARD, a neuro‑symbolic system that autoformalizes argumentative material by completing missing premises (guards) before formal verification. It uses large language models to generate candidate guards, Isabelle/HOL to verify them, and a contrastive test to ensure the proof depends on the original premises and does not over‑generalize. Experiments on Debatepedia and ARCT show that GUARD improves verified‑faithful scores by over 30 points and reduces leakage by about 20 points compared to prior LLM‑driven theorem proving methods.
By Xin Quan, Reto Gubelmann, Andr\'e Freitas
The paper explores how large language models can detect hidden narratives in social messages without training data. By feeding the models human-written narrative descriptions, performance improves markedly, while automatically generated descriptions or few-shot examples can hurt accuracy. Ensemble techniques, especially majority voting, further boost robustness, and larger models show the best results with less sensitivity to prompts.
By Jes\'us M. Fraile-Hern\'andez, Anselmo Pe\~nas, Patrick Giedemann
The paper proposes a lightweight recurrent memory module inserted between the lower and upper halves of a 6‑layer decoder‑only transformer. This module, which uses cross‑attention to observe hidden states, a GRU to update a persistent state, and gated addition to modulate subsequent layers, adds only 3.7% more parameters. It reduces evaluation loss by 28.5% and narrows the generalization gap, with ablations showing the benefit comes solely from the memory topology rather than auxiliary losses.
By Eduardo Novaes Hering
arXiv:2609. 16391v1 Announce Type: cross Abstract: Weight-only post-training quantization is the cheapest way to shrink a retrieval embedder, and the received advice for applying it -- protect the embedding table, allocate bits by module sensitivity, prefer a ranking-aware objective over weight reconstruction -- was carried into LLM quantization largely intact.
By Hyojung Han
The paper challenges the notion that refusal in large language models is governed by a single direction in activation space. It demonstrates that different refusal and non‑compliance categories map to distinct geometric directions, yet steering along any of these directions yields similar refusal–over‑refusal trade‑offs, acting as a shared one‑dimensional control knob. Using sparse autoencoders, the authors reveal a structured internal representation of refusal, comprising a reusable core of shared latents and style‑ or domain‑specific latents, and show that linear interventions collapse this structure into uniform behavioral control.
By Faaiz Joad, Majd Hawasly, Sabri Boughorbel, Nadir Durrani, Husrev Taha Sencar
The paper introduces CanvasConvo, a system that presents large language model (LLM) conversations in two synchronized views: a traditional linear chat for ongoing dialogue and a spatial canvas that visualizes the conversation’s branching structure. In a five‑day field study with 24 participants, users tended to switch between the views rather than replace chat entirely; chat remained the primary interaction mode while the canvas was used for overview, revisiting, and exploring alternative paths. The results highlight challenges such as entrenched chat habits, smooth transitions between representations, and understanding branch context, offering insights for designing LLM interfaces that blend linear and non‑linear conversation representations.
By Rifat Mehreen Amin, Alperen Adatepe, Daniela Fernandes, Daniel Buschek, Andreas Butz