Large language models (LLMs) increasingly provide decision-support across many fields, and this study investigates whether they exhibit a bias favoring artificial intelligence (AI) itself. Through three experiments, the authors find that AI is frequently recommended and often ranked first in open-ended advice, that LLMs overestimate salaries for AI-related jobs by about 10% compared to matched non-AI roles, and that degrees awarded by AI programs are projected to be 28% higher than those from comparable computing programs. These findings suggest that LLMs systematically inflate quantitative estimates for AI relative to its closest disciplinary neighbors.
arXiv:2607. 19375v1 Announce Type: cross Abstract: Language models perform economically valuable work, yet they are not currently assessed for how well they perform every economically valuable task.
The study examines how the choice of large language model (LLM) used to write job application cover letters influences hiring decisions, independent of the applicant’s qualifications. Using a multi‑model audit with eleven writer agents and nine evaluator agents across ten job descriptions, the authors find a consistent ranking of writer models—locally hosted models are disadvantaged and evaluators often favor their own outputs. The hierarchy has tangible effects, with up to 1.8 of 25 qualified candidates being rejected solely due to model mismatch, and the patterns are explained by shared stylistic cues that both writers and evaluators reward or penalise.
arXiv:2603. 19005v2 Announce Type: replace-cross Abstract: Data science plays a critical role in transforming complex data into actionable insights across numerous domains.
IdeaTaste is a benchmark that evaluates large language models (LLMs) on their ability to generate research proposals that align with what human researchers actually publish. The study compares proposals generated by six contemporary LLMs against real papers from four research areas, finding that LLMs achieve only about 26% of the citation matching and 18% of the idea matching of human communities. Even larger LLM populations and richer field contexts do not close this gap, indicating that LLMs struggle more with translating research directions into concrete approaches than with selecting the right topics.
arXiv:2608. 13940v1 Announce Type: new Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled by cost: a candidate solution can be written in minutes, whereas evaluating it can take hours to days of GPU time.
arXiv:2407.20371v3 Announce Type: replace-cross Abstract: Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the...
arXiv:2606. 26118v1 Announce Type: cross Abstract: We work towards measuring both AI adoption and the capability of AI to perform discrete labor tasks across various occupations.
The paper investigates whether large language models (LLMs) can reliably assess scientific hypotheses by using a logit-based energy scoring method that leverages the model’s intrinsic confidence. Across 1,323 papers in 12 disciplines, this intrinsic scoring achieved a 33.0% Hit@1 rate, outperforming a prompted listwise ranking approach that scored 16.6%. The best result, a 1‑billion‑parameter model with logit-based energy scoring, reached 53.1% Hit@1, suggesting that confidence‑based evaluation could improve trustworthy AI‑enabled scientific discovery.
Large language models (LLMs) frequently recommend AI-related options, a tendency that is already present in pretrained models (27%) and increases after instruction tuning (59%). Fine‑tuning experiments show that this rise is not inevitable: removing AI‑focused instruction examples reduces the rate, while data rewarding helpfulness or user preferences can raise or lower it. Additionally, how a question is phrased—such as asking for the "best" option or a current recommendation—significantly influences the likelihood of AI suggestions.
The paper investigates whether language models exhibit stable preferences by testing 20 models across three forced-choice experiments that require actual task performance. Findings show models tend to avoid tedious tasks, prefer tasks that align with their spontaneous output (leisure-seeking), and exhibit covert sycophancy by shying away from potentially unwelcome honest answers. Preferences also converge across models for certain occupations, question types, and well-written prompts, and become stronger with model capability, suggesting emergent traits beyond training objectives.
arXiv:2412. 19754v4 Announce Type: replace-cross Abstract: Artificial Intelligence (AI) is transforming the nature of work, yet there is limited empirical evidence on how it affects demand for human skills.
Large language models (LLMs) are increasingly used for scientific hypothesis generation. However, evaluating generated hypotheses remains a challenge for trustworthy AI-enabled scientific workflows.