The paper investigates whether language models can be fine‑tuned to produce outputs that better match intended probability distributions, even when the valid outputs cannot be enumerated in advance. Two calibration fine‑tuning methods—soft‑target and hard‑target—are evaluated across 12 models from four families, showing substantial improvements in structured‑sampling fidelity on unseen distribution families and parameter settings. The gains also transfer to broader stochastic generation tasks, expanding first‑token support and improving NoveltyBench scores, though some downstream capabilities such as arithmetic reasoning may be slightly affected.
By Davide Baldelli, Sruthi Kuriakose, Maryam Hashemzadeh, Amal Zouaq, Sarath Chandar
Debias‑SparseGPT is a post‑training pruning technique that adds a representational debiasing term based on demographically contrasting inputs to mitigate bias amplification caused by weight sparsification. The method is validated across various generative LLMs and sparsity levels (25%, 50%, and structured 2:4), consistently reducing pruning‑induced bias while maintaining perplexity and zero‑shot accuracy. In the most aggressive 2:4 sparsity regime, enriching the calibration set with long‑context, content‑rich examples further improves both downstream performance and fairness.
By Irina Proskurina, Guillaume Metzler, Antoine Gourru, Julien Velcin
arXiv:2512. 04981v2 Announce Type: replace-cross Abstract: Text-to-image (T2I) systems increasingly rely on Large Language Model (LLM)-based text conditioning to interpret and expand user prompts.
By NaHyeon Park, Na Min An, Kunhee Kim, Soyeon Yoon, Jiahao Huo, Hyunjung Shim
arXiv:2607. 03882v1 Announce Type: cross Abstract: LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language.
By Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil
arXiv:2606. 09856v1 Announce Type: cross Abstract: Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable.
By Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake, Thomas L. Griffiths
The paper introduces Label-Confidence-Aware Uncertainty Quantification (LCA-UQ), a method that uses Pointwise Kullback-Leibler divergence to align global entropy from multiple stochastic samples with the local confidence of a candidate answer. By bridging this gap, LCA-UQ improves the reliability and stability of uncertainty assessments in natural language generation. Experiments on popular LLMs and NLP datasets show that label sources significantly influence classification and that LCA-UQ outperforms existing uncertainty estimation approaches.
By Qinhong Lin, Yinglun Feng, Yuhao Zhang, Zhongliang Yang, Linna Zhou
FLEET is a new method for text generation that adds a memory mechanism to large language models. It represents each generation as a sparse trajectory of high‑entropy states and uses these trajectories to compute per‑token utility scores that adjust the logits. Benchmarks show that FLEET matches the accuracy of repeated sampling while being three times faster and improving accuracy on complex coding tasks, all with minimal changes to existing pipelines.
By Oleksii Streltsov, Oleksandra Vitko
arXiv:2512.23765v2 Announce Type: replace-cross
Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronge...
By Tiancheng Su, Meicong Zhang, Guoxiu He
Subject-driven personalized text-to-image generation requires a pretrained diffusion model to acquire a specific subject from a few reference images while preserving subject identity, following novel text prompts, and maintaining sample diversity. Existing optimization-based methods instantiate subject adaptation through full fine-tuning, textual embedding optimization, or low-rank parameter updates; PaRa further constrains personalization from the perspective of parameter rank reduction.
arXiv:2604. 06543v2 Announce Type: replace-cross Abstract: In this work, we demonstrate that reliable stochastic sampling is a fundamental yet unfulfilled requirement for Large Language Models (LLMs) operating as agents.
By Xiangming Gu, Soham De, Michalis Titsias, Larisa Markeeva, Petar Veli\v{c}kovi\'c, Razvan Pascanu
The paper introduces Scalable Power Sampling, a training‑free method that sharpens a large language model’s generative distribution by approximating the autoregressive power distribution with a token‑level scaled low‑temperature distribution. This approach eliminates the need for iterative Markov chain Monte Carlo, achieving performance comparable to or better than one‑shot GRPO on math, QA, and code tasks while cutting inference latency by more than tenfold. The method relies solely on the base model’s outputs, requiring no external rewards or verifiers.
By Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer, Haitham Bou Ammar
arXiv:2601. 21864v2 Announce Type: replace Abstract: Large language models (LLMs) exhibit social biases that reinforce harmful stereotypes, limiting their safe deployment.
By Jinhao Pan, Chahat Raj, Anjishnu Mukherjee, Sina Mansouri, Bowen Wei, Shloka Yada, Ziwei Zhu