The article examines the distribution of logits from open AI models, both base and instruction-tuned, to determine how many guesses are required to crack an AI’s supposedly random number. By converting the logits into bits, the author quantifies the predictability of the models’ outputs. The analysis highlights the gap between perceived randomness and actual statistical patterns in AI-generated numbers.
The article examines the performance of AI models on a 2,000-question test, finding that the top two models were indistinguishable at the conventional 5% significance level. When the test was reduced to 200 questions, a different model would have claimed first place roughly three times out of ten trials. The piece questions the reliability of declaring a single model as the best based on limited testing.
Let's practice data science thinking through a probability problem The post Solving the 3Blue1Brown String Probability Problem (Without AI) appeared first on Towards Data Science .
Assistant Professor Benjamin Lindquist explores how early AI engineers intentionally added randomness to models, leading to deep‑learning systems that can hallucinate. The article examines the origins of this design choice and its impact on how AI mimics the human mind’s unpredictability.
We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to obtain probably approximately correct (PAC) bounds for this problem.
arXiv:2607. 20286v1 Announce Type: cross Abstract: We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt.
The article explores the effects of removing a search box from an AI agent and instead providing it with typed tools, hard bounds, and a gate that it cannot bypass. It examines how the agent navigates a knowledge graph within strict limits and discusses findings from four models and one incorrect prediction regarding the value of this approach.
arXiv:2608. 19908v1 Announce Type: cross Abstract: Probability estimation over large alphabets under log loss is a well-studied problem, with celebrated methods such as the Good-Turing estimator.
Research-backed cues to detect LLM-generated text along with the mathematical intuition as to 'why' The post Is This Slop? Detecting AI-Generated Content Without a Model appeared first on Towards Data Science .
When the harmfulness of an LLM agent's output can be quantified, a natural jailbreaking objective is to maximize expected harmfulness over admissible input modifications. An alternative approach const...
They aren’t designed, you can’t help perceiving one anyway, and that makes them an engineering problem almost no one is solving. The post Where Does an AI’s Personality Actually Come From?
The article titled "The AI That Learned to Understand Long After It Stopped Trying" discusses a small, strange discovery in machine learning known as grokking. It highlights how this phenomenon involves an AI developing understanding after ceasing to actively try. The piece was originally published on Towards Data Science.
The paper introduces a method for auditing the calibration of large language models (LLMs) that only exposes a logit_bias parameter. By mathematically manipulating this parameter, the authors can evaluate exact probability thresholds with a single query per sample, enabling a provably consistent estimator of True Calibration Error for binary tasks. This approach offers an efficient framework for auditing black‑box foundation models despite limited access to continuous output probabilities.