Conditional generative models such as DALL-E and Stable Diffusion generate images based on a user-defined text, the prompt. Finding and refining prompts that produce a desired image has become the art of prompt engineering. Generative models do not provide a built-in retrieval model for a user's information need expressed through prompts. In light of an extensive literature review, we reframe prompt engineering for generative models as interactive text-based retrieval on a novel kind of "infinite index". We apply these insights for the first time in a case study on image generation for game design with an expert. Finally, we envision how active learning may help to guide the retrieval of generated images.
@article{arxiv.2212.07476,
title = {The Infinite Index: Information Retrieval on Generative Text-To-Image Models},
author = {Niklas Deckers and Maik Fröbe and Johannes Kiesel and Gianluca Pandolfo and Christopher Schröder and Benno Stein and Martin Potthast},
journal= {arXiv preprint arXiv:2212.07476},
year = {2023}
}