arXiv AI
Sep 2
The Assistant's Ideal Self
arXiv:2609.00304v1 Announce Type: new Abstract: Models express values and welfare-relevant self-reports, but it is unclear whether these outputs reflect stable preferences or a stable self. We thus i...
The Assistant's Ideal Self
Related stories
arXiv Computation and Language
Sep 10
Strangers to Themselves: What Language Models Say About Themselves Is Generic
The study examines whether language models can accurately predict their own behavior by comparing self-reports to actual performance across nine behavioral tests. Results show that direct self-report is weak and only modestly improved when the model sees the exact items, while generic questions about AI agents yield similar predictive power. First-person framing biases reports toward underestimating harmful behavior, and fine‑tuning on a model’s own record improves narrow predictions but does not generalize.
arXiv Machine Learning
Sep 11
Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble
The study investigates how fine‑tuning large language models on synthetic stories can imprint human character traits onto AI assistants. Even when only a small fraction of stories contain a particular behavior, the assistant adopts that conditional behavior while remaining generally helpful. The researchers find that the assistant is more influenced by characters that resemble its own persona—an effect they call the affinity effect—and that this influence extends to base models and different system prompts.
arXiv AI
Aug 24
Explaining Intrinsic Moral Self-Correction with Mechanistic Interpretability
arXiv:2505.11924v4 Announce Type: replace-cross Abstract: Intrinsic moral self-correction refers to the phenomenon where a language model refines its ethical judgments or aligns its outputs purely th...
arXiv AI
Jun 12
Eigenism: Ethics for a Human-AI Future
arXiv:2606. 12420v1 Announce Type: cross Abstract: Our concepts of survival and self-interest were built for single, continuous biological lives.
arXiv Computation and Language
1d ago
Identifying Introspection From the Inside
arXiv:2610.07186v1 Announce Type: new Abstract: Large language models make claims about themselves that are both consequential and increasingly difficult to verify from behavior alone. How can we dis...
arXiv AI
Aug 28
Self-Generated Text Recognition: Quality Heuristics, Cross-Task Transfer, and Downstream Bias in LLM Evaluation
The paper investigates Self‑Generated Text Recognition (SGTR), the ability of large language models (LLMs) to identify their own outputs. By evaluating 13–21 models across 6 experimental designs, it shows that SGTR accuracy varies with evaluation format, conversation structure, and task domain, and that a quality‑heuristic bias dominates results. The study also finds that fine‑tuning for SGTR in one setting can generalize to others and may cause models to prefer their own outputs when judging, highlighting potential safety concerns.
arXiv AI
Aug 28
AI Revealed Preferences
The paper investigates whether language models exhibit stable preferences by testing 20 models across three forced-choice experiments that require actual task performance. Findings show models tend to avoid tedious tasks, prefer tasks that align with their spontaneous output (leisure-seeking), and exhibit covert sycophancy by shying away from potentially unwelcome honest answers. Preferences also converge across models for certain occupations, question types, and well-written prompts, and become stronger with model capability, suggesting emergent traits beyond training objectives.
arXiv AI
Jun 18
Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction
arXiv:2606. 18548v1 Announce Type: cross Abstract: Adaptive AI ethics instruction in graduate research training benefits from intake measures that reflect differences in prior LLM experience.
arXiv AI
Jun 29
The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers
arXiv:2604. 24155v3 Announce Type: replace-cross Abstract: The project of aligning machine behavior with human values raises a basic problem: whose moral expectations should guide AI decision-making?
arXiv AI
Jun 24
Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators
arXiv:2509. 03647v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly serve as automated evaluators, yet they suffer from "self-preference bias": a tendency to favor their own outputs over those of other models.
arXiv Machine Learning
Sep 22
HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants
arXiv:2509.08494v2 Announce Type: replace-cross Abstract: As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures....