arXiv:2510. 12201v2 Announce Type: replace Abstract: As AI becomes more common in everyday living, there is an increasing demand for intelligent systems that are both performant and understandable.
By Aline Mangold, Juliane Zietz, Susanne Weinhold, Sebastian Pannasch
The article discusses how large language models can now generate user interfaces from natural‑language descriptions, and proposes a framework that turns HCI design knowledge into machine‑readable ‘skills’ that guide on‑demand UI construction. By treating the dialogue between user and AI as a shared cognitive workspace, these skills encode established HCI principles (e.g., Nielsen’s heuristics, WCAG criteria) as editable, version‑controlled artifacts. The authors outline a research agenda that envisions HCI moving from heuristic checklists to a machine‑executable, open‑source craft.
By Nathan Conklin, Miranda Capra, Chris North
The paper introduces GenUI‑Harness, a multi‑agent system that pairs a Tool Agent for data retrieval with a GUI Coder Agent that generates structured interfaces to aid human‑agent interaction. It tackles training challenges by using Dynamic UX for lightweight reward collection and Reward Auditor to monitor reward quality. The authors present UI‑TAU Bench, a benchmark of 1,300 database‑backed tasks, and show that GenUI‑Harness improves Pass@3 scores and reduces dialogue rounds compared to existing models.
By Xiaolong Li, Xiaohan Xu, Jinyang Li, Xinnuo Xu, Ge Qu, Nan Huo, Jack Williams, Reynold Cheng
The paper proposes ‘meta access’—a structured, permissioned view of an interface’s state—to enhance large language model chat UIs. It outlines a ladder of access levels, from feature tutoring to full orchestration, and demonstrates the concept with the Meta Chat prototype, which uses summaries, settings, and message links to enable reversible changes and a browsable tutorial library. Meta access aims to transform chat from a single conversation window into a control layer for the entire workspace while addressing risks such as grounding, privacy, and prompt injection.
By Julian Kunkel, Ali Doosthosseini
arXiv:2607. 20773v1 Announce Type: cross Abstract: Large language models (LLMs) have shifted human--computer interaction from `traditional'' interface journeys toward more conversational exchanges.
By Zeshu Zhu, Natalie Friedman, Kevin Weatherwax, Emily Eiben
arXiv:2608. 07494v1 Announce Type: cross Abstract: AI tools like ChatGPT and DeepSeek, powered by Large Language Models (LLMs), allow users to obtain instant and effective content responses simply by typing requests, such as ``plan a three-day Vienna trip'', ``solve the attached mathematical problem'', ``draft an email to inquire review progress'', etc.
By Yiqun Zhang, Yunfan Zhang, Mingjie Zhao, Sen Feng, Yiu-ming Cheung