The paper introduces a conformal prediction framework designed for molecular property prediction under label shift. By weighting conformal scores with marginal label probability ratios, it generates statistically rigorous prediction intervals without retraining, enabling robust uncertainty quantification when property distributions change. This approach provides actionable confidence measures that improve the reliability of AI-driven predictions in drug discovery.
By Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba, Seungjin Choi, Hyunjin Shin
The paper introduces ‘Fool’s Gold’, a defensive deception technique for open‑weight language models that hardens them against safety‑removal attacks. By training decoy responses within a differentiable simulation of the attack, the method poisons the payoff of stripped refusal mechanisms, producing confident but falsified answers to hazardous requests while preserving benign behavior. Experiments on seven models (9B‑122B) show that 51‑90% of attacked‑state responses become decoys, with the defense accounting for 27‑84% of this effect, and that the defended 122B model remains within benign‑behavior budgets.
By Mark Russinovich
The paper introduces a comprehensive privacy evaluation framework for genomic language models (GLMs) that quantifies memorization risks using perplexity-based detection, canary sequence extraction, and membership inference. By planting canary sequences at different repetition rates in synthetic and real datasets, the authors systematically assess how repetition, model capacity, and training dynamics affect memorization across various GLM architectures. The study demonstrates that GLMs do memorize training data to varying degrees and that no single attack method fully captures this risk, highlighting the necessity of multi-vector privacy auditing for genomic AI systems.
By Alexander Nemecek, Wenbiao Li, Xiaoqian Jiang, Jaideep Vaidya, Erman Ayday
The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.
By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
The paper compares four audit methods for assessing identity‑level differential privacy in pre‑trained, black‑box face generators. Each method—GaussMech, KDE‑LR, MMD‑TV, and ROC‑HT—has distinct assumptions, hyperparameters, and finite‑sample limitations, and they produce markedly different epsilon estimates when applied to FaceFusion and InstantID. The study finds that all methods reveal significant identity distinguishability, but none can be reliably ranked in this high‑distinguishability regime, suggesting that future work should evaluate them on partially private mechanisms.
By Arman Zareian Jahromi, Vishnu Bondalakunta, Mohammad Akbar Bin Shah, Naimul Haque, Shuangqing Wei, George T. Amariucai
The paper proposes a methodological framework to assess large language models (LLMs) as surrogate experts in security surveys, particularly for Security Operations Centres (SOCs). By comparing persona-based and aggregate LLM-generated responses to real SOC professional data, the study evaluates stability, inter-model agreement, and alignment with human answers. Findings reveal that while LLMs produce internally consistent responses, they systematically diverge from experts, showing reduced variance, central tendency bias, and homogenised opinions, indicating they are suitable for piloting and hypothesis generation but not for replacing expert elicitation.
By Despoina Giarimpampa, Roland Meier, Tegawend\'e F. Bissyand\'e, Vincent Lenders, Jacques Klein
Reflex-Guard is a lightweight, locally‑run guardrail for large language models that uses jailbreak‑aware preprocessing, compact sentence‑transformer embeddings, and seven fast binary classifiers to filter unsafe prompts. It achieves 95.9 % recall on harmful prompts with an end‑to‑end latency of 37.6 ms, outperforming existing solutions such as Llama Guard 2 (255 ms) and SafeDecoding (723 ms). The system can detect all GCG suffix attacks and Base64‑encoded prompts at the default threshold, while DrAttack structured prompts require a lower threshold of 0.03 for optimal detection, and it attains a Reflex Efficiency Score of up to 16.79.
By Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon, Abu-fuad Ahmad, Thi Hong Tran
QuantumNovelty is an open‑source, skill‑orchestrating language agent that both creates quantum‑computing artifacts—such as papers, ansatz candidates, and patent drafts—and evaluates them through simulated referee and patent‑examiner panels. Its core innovation is an audit‑and‑falsify layer of deterministic gates (Pareto domination, numerical recomputation, Wilson intervals, and cross‑vendor consensus) that restricts claims to those that survive rigorous checks, with every model call logged for transparency. In initial tests on a planted adversarial corpus and a small real‑world deployment, the system successfully flagged all overclaims without false positives and produced panels that were more conservative than typical public acceptance rates.
By Shlomo Kashani
Prism‑GRPO enhances the GRPO reinforcement‑learning algorithm for vision‑language‑action policies by adding a weighted trajectory‑level execution‑quality score to binary success rewards. This approach splits groups with identical outcomes into a quality spectrum, preserving training signal while ensuring successes always outrank failures. Experiments on four RoboTwin tasks show Prism‑GRPO achieves higher success and quality at matched rollout budgets, reaching target success rates with up to 56% fewer rollouts and mitigating reward‑hacking behaviors that transfer to real‑robot deployment.
By Zeyun Deng, Yuzhe Lu, Yawei Wang, Linbo Liu, Qing Ping, Han Ding, Guande Wu, Panpan Xu, Jun Huan
The study investigates how aligning the representational geometry of artificial neural networks can improve bidirectional predictivity with biological neural responses. By applying spectral regularization during training of self‑supervised contrastive vision models, the authors increased reverse predictivity by 55% while only modestly reducing forward predictivity. The adjustments also lowered effective dimensionality and reorganized the shared representational subspace, making forward and reverse predictivity more symmetric at intermediate spectral exponents.
Genomic analysis on viruses such as SARS-CoV-2 variants: Beta, Gamma, Delta, and Omicron is heavily dominated by classical bioinformatics methods, including Sequence Alignment, Phylogenetic Analysis,...
TokEval is a tokenizer evaluation suite that introduces metrics beyond traditional fertility and compression rate, capturing linguistically and structurally meaningful properties such as UTF-8 character boundary integrity and digit place-value alignment for mathematics. The authors validate these metrics by conducting controlled language model pretraining experiments that vary tokenizer training data, pretokenization strategy, and training algorithm, then evaluate the models on bits-per-byte and benchmarks covering linguistic understanding, mathematical reasoning, and code generation. Results show that information-theoretic metrics predict language modeling performance, while structure-sensitive metrics correlate with task accuracy, suggesting TokEval can guide tokenizer selection more principledly.
The paper argues that language operates with two parameters: amplitude, which measures how often words co‑occur, and phase, a signed relational factor that determines how co‑activated meanings combine and can reverse a meaning’s contribution. Unlike amplitude, phase is not captured by standard word embeddings or transformer attention weights and is indexed to individuals and dyadic interactions. The authors propose six empirical predictions to test phase’s role and suggest that future language models should incorporate agent‑indexed, phase‑bearing semantic states.
The paper explores using a small language model (SBERT) for invoice categorisation, a task that requires nuanced accounting judgement. By analysing the embedding geometry of SBERT and DeBERTa, the authors find that the sentence‑embedding space is globally anisotropic but contains locally isotropic clusters tied to vendor identity. Fine‑tuned SBERT achieves 0.96 accuracy and 0.9 F1 with only about 100 client‑specific invoices, outperforming zero‑shot LLMs and vendor baselines, and demonstrates that in‑house SLMs can reduce cost, enhance security, and improve interpretability.
The paper introduces a risk‑controlled framework for using large language models (LLMs) as judges in tasks without reference answers. By calibrating uncertainty thresholds on a held‑out set, the method ensures that the false discovery rate of accepted verdicts stays below a user‑specified level α with high probability, using finite‑sample Clopper–Pearson intervals. When the parametric judge lacks confidence, the instance is routed to a retrieval‑augmented mode with a second calibrated threshold, preserving the error guarantee while achieving higher coverage than single‑mode baselines.
The paper introduces TSN4PI, a unified framework for tracking the evolution of political ideologies on social media. It combines a PIDN module that uses large language models, style transfer, and unsupervised domain adaptation to detect ideologies and filter noise, with a PIPN module that employs temporal graph neural networks to predict future ideological shifts. The authors release two large-scale datasets and validate the approach on platforms such as X and Truth Social, offering empirical insights into political polarization and ideology evolution.
BayesPrompt proposes a Bayesian approach to prompt optimisation for large language models, aiming to generate prompts that are both efficient in perplexity and human readable. The authors argue that traditional optimisation methods produce unintelligible pseudoprompts due to the ill‑posed nature of the task. Their algorithm samples prompts from a posterior distribution, and experiments on real data show marked improvements over state‑of‑the‑art alternatives across several metrics.
TINA+ is a diffusion-consistent, text‑free inversion attack that probes residual visual knowledge in diffusion models after concept erasure. By using optimization‑based inversion and diffusion‑consistent trajectory regularization, it suppresses spurious trajectories that could falsely indicate retained concepts. Experiments across multiple erasure methods, tasks, and model architectures show that TINA+ reliably recovers erased concepts, revealing that many current techniques only sever text‑image links rather than eliminating underlying visual knowledge.
The paper introduces InnerExpert, a method that uses Mixture-of-Experts (MoE) internal signals—such as router entropy, expert disagreement, and usage patterns—to detect hallucinations at the token level in large language models. By combining these MoE-specific signals with standard transformer features into compact per-token vectors, InnerExpert trains a lightweight detector using an LLM-as-a-judge pipeline, enabling continuous updates without manual labeling. Experiments across five datasets and two MoE architectures show that InnerExpert outperforms existing methods, achieving up to 0.91 answer-level and 0.76 token-level AUROC with only a single forward pass.
The article titled "Ten Is Not a Hundred" discusses how the number ten can mislead hallucination detectors. It highlights a specific instance where this numerical confusion caused errors in automated systems. The piece was originally published on Towards Data Science.
By Javier Marin