arXiv AI
Jul 28
What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features
arXiv:2607. 23271v1 Announce Type: cross Abstract: Contrastive vision-language models such as CLIP map semantically opposite phrases (e.
Invariant Pretraining for Robust Code Representations
arXiv:2608. 15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clone detection and code classification, where their small size and low inference cost are decisive.
Related stories
arXiv AI
Jul 17
T5-CSBoost: Adversarial Perturbation Resistant LLM Fingerprinting
arXiv:2607. 14113v1 Announce Type: cross Abstract: While many AI-generated text (AIGT) detectors achieve strong performance on clean inputs, their accuracy degrades significantly under light paraphrasing, word substitutions, character edits, and distribution shifts.
arXiv AI
Jun 24
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
arXiv:2606. 24716v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence.
arXiv AI
2d ago
When Uncertainty Isn't Enough: An Empirical Study of Self-Correction in Code Generation
arXiv:2608. 14659v1 Announce Type: new Abstract: Large language models for code generation often produce incorrect solutions without reliable indicators of failure.
arXiv Machine Learning
Jun 16
Emergent retokenization symmetry in large language models: phenomenology and applications
arXiv:2606. 15521v1 Announce Type: cross Abstract: Tokenization introduces representational redundancy: under a fixed token vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string.
arXiv Machine Learning
Jul 7
LP-SFT: Local-Preserving Supervised Fine-Tuning via Multimodal Entropy Structure
arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.
arXiv AI
Jul 2
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
arXiv:2607. 00784v1 Announce Type: cross Abstract: Vision-language pretraining remains dominated by contrastive objectives, whereas vision-only self-supervised learning has largely adopted non-contrastive methods.
arXiv AI
Jun 6
Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning
arXiv:2606. 05173v1 Announce Type: cross Abstract: Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper semantic structure.
arXiv AI
Jul 21
Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence
arXiv:2607. 17770v1 Announce Type: cross Abstract: Within Explainable Artificial Intelligence, mechanistic interpretability uses Sparse Autoencoders (SAEs) to extract more interpretable features from neural representations.
arXiv AI
Jul 21
Symbolic Augmentation Closes a Canonical-Equivalence Blind Spot in Neural Fact-Checkers
arXiv:2607. 16212v1 Announce Type: new Abstract: Large language models hallucinate numbers and units when summarizing scientific text, a failure mode that can silently invert a scientific claim.
arXiv AI
Jun 11
When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis
arXiv:2606. 11375v1 Announce Type: cross Abstract: Standard linear probing declares a property "encoded" when a classifier on hidden states achieves high accuracy.
arXiv Machine Learning
Jun 2
Reconsidering Positional Supervision in Masked Diffusion Language Model Training
arXiv:2601. 22947v2 Announce Type: replace-cross Abstract: Masked diffusion language models (MDLMs) generate text by unmasking tokens in parallel and have recently emerged as alternatives to autoregressive language models.