English

Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Prompts

Computers and Society 2026-03-03 v1

Abstract

System prompts - instructions that shape the behaviour of generative AI systems - strongly influence system outputs and users' experiences. They define the model's guidelines, `personality', and guardrails, taking precedence over user inputs. Despite their influence, transparency is limited: system prompts are generally not made public and most platforms instruct models to conceal them, leaving users disconnected from and unaware of a key mechanism guiding and governing their AI interactions. This paper argues that system prompts warrant explicit, user-centred design attention and, focusing on large language models (LLMs), asks: what do system prompts contain, how do end-users perceive them, and what do these perceptions offer for design and governance practice? Our results reveal user perspectives on: the benefits and risks of system prompts; the values they prefer to be associated with prompt-design; their levels of comfort with different types of prompts; and degrees of transparency and user control regarding prompt content. From these findings emerge considerations for how designers can better align system prompt mechanisms with user expectations and preferences over these mechanisms that directly shape how generative AI systems behave.

Keywords

Cite

@article{arxiv.2603.00089,
  title  = {Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Prompts},
  author = {Anna Neumann and Yulu Pi and Jatinder Singh},
  journal= {arXiv preprint arXiv:2603.00089},
  year   = {2026}
}

Comments

Accepted to CHI 2026 Paper Track